Chapter 25 — Case Studies

Two organisations independently built what Parts II and III derive. That is corroboration. It is also survivorship, and the difference matters.

25.1 What this chapter cannot do

The plan for this chapter was to recruit or construct four case studies covering a matrix of organisation types. I can do neither honestly.

I cannot recruit, having no access to organisations willing to open their telemetry. And constructing four detailed narratives — even labelled as illustrative — would produce exactly the artefact this book has spent twenty-four chapters warning about: plausible, specific, memorable material that readers will cite as evidence because it is shaped like evidence. Chapter 3 documented how claims mutate as they propagate; a constructed case study is a claim optimised for propagation.

So this chapter reports what actually exists, says what grade it is, and marks the gaps. That is less satisfying than four tidy narratives and it is the only version worth printing.

25.2 The evidence pyramid

Three kinds of evidence about organisational practice exist, and they answer different questions.

The middle tier is the one to be careful with. An organisation that publishes a detailed account of its harness is, necessarily, an organisation that has a harness worth publishing. Nobody writes up the eighteen months in which it did not work. §25.7 returns to this.

25.3 The one measured case

Chapter 3 used it for productivity, Chapter 16 for review economics. Read as a case study it is the most complete organisational account in existence.

A mid-sized commercial B2B software company, several hundred engineers, described by the researchers as a near-ideal and unusually favourable site: young, AI-forward, permissive, commercial tools with no per-seat caps or token budgets. In June 2025 its CTO set an explicit goal to double engineering productivity through AI adoption over twelve months, naming merged pull requests per engineer per month as the progress metric [B].

What happened, measured across 802 developers and 196,212 pull requests:

The authors' summary is the finding: an enterprise AI mandate is "a process-redesign problem, not a tooling deployment", and the gain "relocates work downstream rather than removing it."

Three things make this the most useful case available, and one makes it limited.

The mandate was documented before the outcome. A CTO email with a date and a named metric, then twenty-six months of telemetry. That ordering is what separates a case study from a retrospective narrative.

The metric was met and the goal is unclear. Chapter 3 §3.7 worked through this: merged PRs per engineer roughly doubled while the unit grew by a third. Goodhart operating without anyone gaming anything.

The organisation invented an inferential sensor under pressure. Automated review overtook human review — bots reaching roughly 38% of review events and half of all comments — without being named, measured, or scoped. Chapter 3 called this the moment Part II stopped being a proposal and became a description.

And the limitation: this is one company, chosen by the researchers as favourable, with no control organisation. Its outcomes are not generalisable and the authors do not claim they are.

25.4 OpenAI: structure, then sensors, then janitors

In February 2026 OpenAI published an account of its own practice, documenting experience building over a million lines of agent-generated code [D]. Their definition of a harness matches Chapter 7's: "the full environment of scaffolding, constraints, and feedback loops that surrounds an AI agent… and lets it perform stable work."

Three elements, and each corresponds to something this book derived from unrelated evidence.

Enforced architectural layering. Dependencies flow in a controlled sequence — Types → Config → Repo → Service → Runtime → UI — with agents restricted to operating within those layers, and structural tests validating compliance and preventing violations.

That is Chapter 10 §10.7's tier one, implemented: making the wrong call graph unrepresentable rather than discouraged, enforced by a fitness function rather than a rule. Chapter 24 §24.6 argued layering is the thing to lock in early precisely because it is expensive to retrofit; OpenAI locked it in and their agents work inside it.

Custom linters with remediation built into the message. Constraints are enforced mechanically by custom linters — themselves Codex-generated — plus a small set of "taste invariants": structured logging, naming conventions for schemas and types, file size limits, platform-specific reliability requirements. And the detail that matters most:

"Because the lints are custom, error messages inject remediation instructions into agent context."

This is Chapter 7's blocking sensor with Chapter 6's insight applied to its output. The sensor does not merely report a violation; it supplies the payload the agent needs to fix it. Böckeler describes this pattern as sensors optimised for model consumption; here it is in production, at scale, generated by the agent it governs.

Standing drift agents. Agents run periodically to find inconsistencies in documentation and violations of architectural constraints — "fighting entropy and decay" — with background agents continuously scanning for divergence and opening refactoring pull requests.

That is Chapter 4's replacement homeostat, running as Chapter 12's Observe phase, staffed by exactly the A4 band Chapter 13 §13.7 identified as genuinely available today. Chapter 4 argued the remedy for the removed homeostat must be a sensor with a standing mandate, because nothing triggers it. OpenAI built one.

25.5 Stripe: the blocking sensor, priced

Stripe published a two-part account of Minions — one-shot, end-to-end coding agents that merge more than a thousand pull requests a week with zero human-written code [D]. (Figures of 1,000 and 1,300 both circulate; the conservative one is used here.)

Two elements matter for this book.

Blueprints, and a deterministic/inferential division of labour. A blueprint is a workflow defined in code that directs a Minion run. Within it, the split is explicit: the agent reasons about the task and writes the implementation; deterministic code runs linters, executes tests, pushes to CI, formats commit messages, and opens the pull request.

That is Chapter 8's ordering rule as an architecture. Chapter 1 promised and Chapter 8 stated it: use the cheapest deterministic sensor that can detect the failure class, and reserve inferential capacity for what genuinely requires judgment. Stripe has partitioned an entire pipeline along that line, giving the model the one job that needs judgment and deterministic code everything else.

Pre-push hooks, and the sentence that confirms Chapter 7. Stripe describes a philosophy of "shifting feedback left" through a three-tier feedback loop, and requires that before a Minion may push a branch it must pass local linters and pre-push hooks inside its own devbox. Their assessment:

Catching lint and type errors before pushing to CI is the single biggest cost reducer at agent scale — pre-push hooks resolve common issues in under a second, and every failure caught locally saves a CI run.

Chapter 7 §7.4 argued exactly this from a 2024 ablation table: sensor placement matters, value falls sharply as you move from blocking to in-loop to out-of-loop, and moving a sensor from level 3 to level 1 is a direct increase in review capacity — the cheapest such increase available. The argument was derived from SWE-agent's linting guardrail, worth +3.0 points on a benchmark. Stripe reports it as their largest cost reduction in production.

Chapter 7 also argued that agents make blocking controls politically available for the first time — that rejecting hooks were always the most effective and least adoptable control because humans resent friction, and an agent does not complain. A Minion blocked by a pre-push hook retries. That is the mechanism, running.

25.6 What the exemplars confirm — and what they do not

Set the two accounts against the book's derivations and the overlap is substantial: enforced layering as an unrepresentability constraint, blocking sensors ahead of CI, sensors that supply remediation rather than reporting violations, deterministic work separated from judgment work, and standing agents that fight structural drift without being asked.

None of that was copied from these accounts. Chapters 4, 7, 8, 10, 12, and 13 derive each from benchmark ablations, repository census data, control theory, and a 1972 paper. Two organisations arriving independently at the same controls is the strongest available support for the model.

But be precise about what it supports. These are [D] self-reported practice descriptions with no outcome data. Neither publishes a controlled comparison, a defect rate, a counterfactual, or a cost per accepted change. They tell you what a well-resourced team believes is worth doing. They do not tell you it worked, because neither measured it against an alternative.

The one account with outcome data is §25.3's, and its outcomes are ambiguous — throughput doubled, review load doubled, quality proxies flat over a window Chapter 4 showed is too short to detect the effects that matter.

25.7 The empty matrix

Chapter 24 §24.2 corrected the axis for this: the variable is not greenfield versus legacy but banked versus unbanked variety reduction, with age a poor proxy. Populate that matrix with what exists.

That gap is the chapter's most important finding, and it has a mechanism: survivorship.

Organisations publish harness accounts when they have a harness worth publishing. The eighteen months during which it did not work, the attempt abandoned after a reorganisation, the team that installed hooks and removed them because the false positives were intolerable — none of these produce a blog post. Taleb's silent evidence in its standard form: the record consists entirely of survivors, and we infer the odds from them.

So the honest reading of §25.4 and §25.5 is these practices are achievable by organisations with these resources, not these practices work. And the census underneath says the field looks nothing like them: Hooks in 0.9–3.2% of repositories, Skills and Subagents in roughly 5%, no repository using persistent memory [A].

25.8 A bill, not permission

The reading error these accounts invite is worth naming, because it is the most expensive mistake available in Part V.

OpenAI's agents produce a million lines inside an enforced layer hierarchy validated by structural tests. Stripe's Minions merge a thousand pull requests weekly behind blueprints and sub-second pre-push hooks. The headline in both cases is the autonomy. The reader takes away the number.

But the autonomy is the output. The harness is the input, and it was built first.

Chapter 13 §13.6 established that A3 and A4 are reachable today only via near-zero blast radius or mechanically complete acceptance. Stripe's Minions operate inside blueprints where a deterministic pipeline handles everything except reasoning, and cannot push until local checks pass. That is not high autonomy with weak controls; it is high autonomy licensed by strong ones, which is what the ladder says is required.

So the useful way to read this chapter is as an itemised bill. Enforced layering, validated by structural tests. Custom linters whose messages carry the fix. Blocking checks that run in under a second. Standing agents that scan for drift and open pull requests. Deterministic code doing everything the model does not need to do.

Then the autonomy. In that order, because the order is the argument.