Preface — Why This Book?
2812 words
They were nineteen per cent slower. They believed they were twenty per cent faster. Both numbers describe the same sixteen developers, in the same experiment, in the same week.
— after METR, arXiv 2507.09089 [A]
- Why the two most-cited numbers in this field contradict each other, and why that is the interesting part
- The evidence stance this book takes, and the four grades it uses
- Which layers of software engineering are dating in months and which have not dated in fifty years
- How to read this book if you are shipping on Monday, and how to read it if you intend to cite it
the disagreement about whether AI makes engineers faster is not a measurement problem to be resolved by better surveys. It is a structural fact about where the bottleneck moved, and it is the reason a new lifecycle is needed rather than a new tool.
0.1 Two numbers
Two figures dominate every argument about AI and software engineering.
The first: 55.8% faster. A controlled experiment at GitHub and Microsoft Research put developers on a scoped task — implement an HTTP server in JavaScript — with and without an assistant. The assisted group finished in roughly half the time.
The second: 19% slower. A randomised controlled trial run by METR took sixteen experienced open-source developers and gave them 246 real tasks in repositories they had contributed to for an average of five years. When AI tools were permitted, the tasks took nineteen per cent longer.
Both studies are competent. Neither is fraudulent. They are cited by opposing camps as though only one can be true, and this is where the discourse fails, because both are true and the gap between them is the most informative object in the field.
Notice what changes between the two experiments. Not the model — both used frontier tooling of their moment. Not the developers' competence — the METR participants were, if anything, more expert. What changed was the task: from a small greenfield problem with a clear specification and no existing code to respect, to a large brownfield problem where most of the work is discovering which of ten thousand existing decisions your change must not violate.
Read Figure 0.1 as a warning about instruments rather than a verdict about tools. Every point on that line is a real measurement. The line slopes down. Any argument that quotes one point and omits the line is not an argument, it is a selection.
There is a third reading, and it is stranger than either camp admits. When METR ran a follow-up in 2026, between thirty and fifty per cent of invited developers declined to submit tasks they were unwilling to attempt without AI — at fifty dollars an hour. That refusal is not an opinion about productivity. It is a price. People were offered money to work the old way and turned it down.
So we have three instruments pointed at one question, and three answers:
| Instrument | What it reports | Direction |
|---|---|---|
| Stated preference — ask developers | "24% faster" beforehand; "20% faster" afterwards | strongly positive |
| Measured outcome — time the tasks | 19% longer | negative |
| Revealed preference — offer cash to abstain | refusal at \$50/hour | strongly positive |
A field that cannot reconcile these has no business making confident claims in either direction. Reconciling them is the work of Chapter 3, and the answer turns out to be neither "the studies are wrong" nor "developers are deluded." It is that the three instruments are measuring three different things, only one of which is the thing organisations actually care about.
0.2 What this book is about
The bottleneck moved.
Software engineering spent seventy years optimising the production of code. Compilers, high-level languages, libraries, frameworks, package managers, IDEs, continuous integration, containers — a long march of tools whose common purpose was to make correct code cheaper to write. The lifecycle we inherited, in all its variants, encodes that assumption. Requirements, design, implementation, test, release: a pipeline built around the premise that implementation is the expensive step.
Implementation is no longer the expensive step. This is not a prediction; it is the plainest reading of the evidence, including the pessimistic evidence. Even METR's slowed-down developers were not slowed because generating code was hard. They were slowed by reviewing, correcting, and reconciling what was generated.
This book's spine, stated once, plainly:
SDLC assumed the bottleneck was producing code. ADLC assumes the bottleneck is verifying and directing it.
Everything else follows from that sentence. If verification is the constraint, then the engineering discipline that matters is the design of the apparatus that verifies and directs — the specifications that state intent, the context that the model sees, the sensors that catch what it got wrong, the structure that limits what it can break, and the graph that tells it where things are. Code becomes something closer to a build output. The durable artefacts move up a layer.
That is a claim about engineering. There is a second claim, about a change of intellectual kind, and it is the one that makes this a book rather than a checklist. When your primary labour is stochastic, you are no longer writing instructions. You are regulating a process whose individual outputs you cannot predict but whose distribution you can shape. That is control systems engineering, and it comes with a body of theory — feedforward and feedback, requisite variety, observability — that software engineering has mostly not needed until now.
Engineers who make this transition well stop asking "did the agent get this right?" and start asking "what would have caught it if it hadn't?"
0.3 The stance
Every empirical claim in this book carries a grade:
| Grade | Meaning |
|---|---|
| [A] | Peer-reviewed, randomised, or large-N empirical |
| [B] | Preprint with credible methodology, not yet reviewed |
| [C] | Analyst research — rigorous, partly proprietary, partly self-reported |
| [D] | Vendor or practitioner claim — useful signal, weak evidence |
The grades are not decoration and they are not symmetric in what they permit. A [D] claim can motivate a practice; it cannot establish one. An [A] finding on sixteen developers still generalises only as far as sixteen developers.
Two commitments follow, and I would rather be held to them than praised for them.
The first: no selective citation. Where the evidence is mixed, all of it appears, including the parts that damage the argument. Chapter 4 presents the case that agent-assisted codebases accumulate persistent complexity while their velocity gains prove transient. That is not a friendly finding for a book proposing a lifecycle built on delegation, and it appears in full because a reader who discovers the counter-evidence elsewhere is entitled to conclude the author knew and hid it.
The second: verified figures only. Writing this book involved checking the numbers in my own research notes against the primary sources. Nine of them were wrong. A churn statistic that had drifted from 5.5%→7.9% into 3.1%→5.7% somewhere in secondary coverage. A Gartner forecast that turned out to be two different forecasts fused into one. A benchmark score from a single-author preprint that could not be retrieved at all. In every case the error had travelled through a chain of summaries, and in every case the drift made the claim more quotable than the truth.
This is worth naming because it is a mechanism, not an accident. Claims in this field mutate in a consistent direction as they propagate: toward roundness, toward drama, toward the shape of a slide. If you take one methodological habit from this book, take this one — cite the paper, not the summary of the paper. The corrections themselves are recorded in references/sources.md, and I would encourage the exact same suspicion toward the numbers I have printed.
Taleb calls the general failure silent evidence: we reason from the surviving, visible, reported cases and forget the ones that left no trace. Software engineering is unusually exposed to it. We count merged pull requests because merged pull requests are countable. We do not count the abandoned branch, the reviewer who stopped reading carefully at four in the afternoon, the plausible-looking function that has been quietly wrong in production for five months because nothing was watching that path. The visible metrics went up. Whether the invisible ones went down is the entire question, and a lifecycle that does not instrument them is measuring its own applause.
0.4 Who this is for
Two readers, and I refuse to choose between them.
If you are shipping software. You are an engineering leader or a senior practitioner and you must decide what changes next quarter. You need instruments, not epistemology. The book gives you: the Autonomy Ladder (Chapter 13) for deciding how much authority to grant on which code, and why the answer is a function of structure and blast radius rather than enthusiasm; the metrics (Chapter 14) that tell you whether delegation is working, including the ones nobody currently measures; the maturity model (Chapter 15) so you can locate yourself honestly; the 90-day plan (Chapter 26); and the anti-patterns (Chapter 21), which is the chapter most readers will recognise themselves in and the one you should probably read first.
If you intend to cite this. You are a researcher, and you need to know what is evidence, what is synthesis, and what is the author's proposal. The book keeps these separate on purpose. Where I am proposing rather than reporting — the metric families, the autonomy ladder, the maturity model, the reconciliation of the three preference instruments — I say so. Chapter 27 collects the open problems, and Chapter 10 contains an admission that the most elegant argument in the book rests on a mechanism with no causal evidence behind it yet. I would rather hand you a falsifiable claim than a persuasive one.
The two readers need each other more than either would like. Industry practice in this field is currently running well ahead of the evidence, which means practitioners are making expensive decisions on [D]-grade material. Research is running behind the tooling, which means findings arrive describing a configuration surface that has since changed. A book that serves only one audience widens that gap.
0.5 How to read a book about a moving target
The obvious objection to this book is that it will be obsolete before it is read. Model checkpoints turn over in weeks. The configuration mechanisms described in Chapter 7 are, in the words of the researchers who catalogued them, "a point-in-time snapshot of a field whose conventions are still consolidating."
The objection is correct about some layers and wrong about others, and the difference is the most useful thing in this preface.
Taleb's Lindy effect says that for non-perishable things, life expectancy grows with age: an idea that has survived fifty years will probably outlive one that arrived last spring. Applied here it is a straightforward editorial instrument. Parnas published On the Criteria To Be Used in Decomposing Systems into Modules in 1972. Fifty-four years later it is not merely still correct, it has acquired a second and unintended application: the information-hiding discipline that was designed to limit what a human must understand turns out to be the mechanism that limits what an agent can get wrong. That is a Lindy bet paying out. AGENTS.md is roughly eighteen months old. It may well become permanent infrastructure — Chapter 7 shows it converging bottom-up across tools and then being donated to a foundation, which is genuinely remarkable for the timescale — but you should hold the two beliefs with different grips.
So: this book excludes benchmark leaderboards and model comparisons; dates every claim about a specific tool's capability, because those claims expire; describes current configuration conventions as evidence of practice rather than as prescription; and proposes at the layer of controls, metrics, and organisation, where the half-life is long enough to justify print.
Where a specific tool appears, it is because a study measured it. When you read "as of the February 2026 snapshot," treat the sentence as carrying an expiry date.
0.6 What is in here
Five parts.
Part I — The Discontinuity establishes that something actually broke, using evidence rather than assertion. Chapter 1 resolves a naming collision that has made most of the existing literature on this subject unusable, and it should be read first. Chapter 2 identifies the single result that justifies treating any of this as engineering. Chapters 3 and 4 present the productivity and quality evidence in full, including the parts that argue against the book. Chapter 5 names which inherited assumptions fail.
Part II — The Four Pillars is the engineering core: context, harness, graph, and structure, presented as nested rings of control rather than a list of techniques, plus verification and security running across all four.
Part III — The Lifecycle assembles those pillars into a process model with two loops, an autonomy ladder, a metric system, a maturity model, and an argument about organisational shape.
Part IV — The Human Transformation is the part I expect to be argued with, and the part that matters most. Ten shifts. The perception–reality inversion and how to defeat it. What delegating effort does to judgment. How to train juniors when the tasks they learned on are the tasks that automated first.
Part V — Modes and Practice places vibe coding fairly, names the moment a prototype becomes a system, addresses the inversion that legacy codebases need this discipline most and can adopt it least, and closes with a 90-day plan and a list of things nobody has solved.
0.7 The bargain
What I owe you: graded evidence, named sources, explicit separation of finding from proposal, and an honest account of what is unresolved.
What I ask in return: read the parts that contradict your position. If you arrived believing this technology is transformative, Chapter 4 is for you. If you arrived believing it is overstated, Chapter 2 is for you. The interesting position is available only to readers who hold both.
One more thing, and it is the sentence I would keep if I could keep only one.
A harness cannot verify correctness against a requirement that was never articulated. Every control in this book — every sensor, every guide, every fitness function, every graph — operates on stated intent. Where intent is unstated, no apparatus helps, and the failure is silent, because generated code that satisfies an unstated requirement incorrectly looks exactly like generated code that satisfies it correctly.
Specification is the last human monopoly. The rest of this book is about building the apparatus that makes that monopoly worth holding.
The bottleneck moved from producing code to verifying and directing it; every practice in this book is a consequence of that single relocation.
- METR RCT — arXiv 2507.09089 [A] ✅ · the 19%/20% inversion, 16 developers, 246 tasks
- METR redesign, Feb 2026 [B] ✅ · −18% returning subset; 30–50% refusal at \$50/hour
- GitHub/Microsoft Research Copilot experiment [A] 🟡 · ~55.8% on a scoped task — primary pending
- DORA, State of AI-assisted Software Development 2025 [C] ✅ · throughput up, instability up
- GitClear 2025 [C] 🟡 · churn 5.5%→7.9%; see
sources.md § CorrectionsC1 - Parnas (1972), On the Criteria To Be Used in Decomposing Systems into Modules, CACM 15(12) ✅
- Taleb — silent evidence, the Lindy effect. Concepts used with attribution throughout.
- Galster et al., arXiv 2602.14690 [A] ✅ · "a point-in-time snapshot of a field whose conventions are still consolidating"
Full entries and verification status: references/sources.md.