Chapter 8 — Harness Engineering II: The Steering Loop

"Whenever an issue happens multiple times, the feedforward and feedback controls should be improved to make the issue less probable to occur in the future, or even prevent it." — Böckeler [D]

"A regulator… can only regulate what it has a model of." — after Ashby, Law of Requisite Variety

Put those two sentences together and something follows that neither states: a harness cannot grow indefinitely, so variety reduction is not a preference — it is forced.

8.1 Fix the control, not the code

The steering loop is the practice that turns a pile of controls into an engineering discipline. Böckeler states it plainly, and the sentence deserves to be memorised:

"Whenever an issue happens multiple times, the feedforward and feedback controls should be improved to make the issue less probable to occur in the future, or even prevent it." [D]

The operative words are multiple times. A one-off defect is a defect; you fix it. A defect that recurs is evidence about your control system, and fixing the instance leaves the generator that produced it exactly as likely to produce the next one.

This is the single most important behavioural change the book asks for, and it inverts a reflex that has been correct for fifty years. When a human colleague made the same mistake three times, the remedy was a conversation — they would learn. An agent will not learn from your conversation. The next session starts fresh. So the only durable place to put the lesson is the harness.

Which gives the diagnostic question that should replace "did the agent get this right?":

What would have caught this if it hadn't?

Chapter 3 gives the loop its economic justification. Review capacity is the binding constraint. Fixing an instance consumes review capacity and returns nothing to the system. Installing a control consumes review capacity once and returns capacity on every subsequent occurrence. The steering loop is how an organisation converts a non-renewable expenditure into a renewable one.

There is a genuinely Talebian property here worth naming, because it is the optimistic half of this book. A system that gets stronger from failures — that converts stressors into structural improvements — is antifragile. The steering loop is exactly that mechanism, and it is the only thing standing between an organisation and Chapter 4's monotone accumulation. Without it, every agent failure is a pure loss. With it, each failure buys a control that pays out indefinitely. The loop is what makes an engineering organisation antifragile with respect to a stochastic generator.

Böckeler adds that the loop can itself be accelerated by agents — writing the tests, drafting the rules, scaffolding the linters. That is right, and it is the highest-leverage use of an agent in this book: point it at the harness, not only at the product.

8.2 The loop is one-directional, and that is a problem

Now the objection, and it is the reason this chapter is not simply a summary of the article that named the loop.

Read the loop again. When an issue recurs, improve the controls. Every traversal adds. There is no clause that removes anything.

Chapter 2 established, from three systems across two years, that the optimal harness shrinks as models improve: harness mass measures what the model cannot yet do for itself, and a control that compensates for a closed deficit is not neutral but overhead. Chapter 6 established that the context budget is scarce and the marginal token can be negative. Chapter 7 established that guides age while sensors persist.

A loop that only adds, operating on a budget that is finite, populated mostly by assets that depreciate, has one terminal state: a harness full of controls nobody can justify, consuming budget, contradicting each other, and teaching everyone who reads it that its contents are decorative. Chapter 2 promised this chapter would name that condition, so: it is Harness Theatre, and Chapter 21 catalogues it alongside its siblings.

So the loop needs a second arm.

The retirement arm needs a mechanism, not an intention, or it will never run — and §8.9 supplies one. First, the reason the constraint is not merely budgetary but formal.

8.3 The Ashby constraint

W. Ross Ashby's Law of Requisite Variety is the foundational result of cybernetics: a regulator must possess at least as much variety as the system it regulates. Böckeler cites it, along with the sharper corollary that a regulator "can only regulate what it has a model of."

Apply it to a coding agent and something uncomfortable falls out.

A coding agent can, in principle, produce any program. Its output variety is effectively unbounded. Your harness is a finite collection of guides and sensors — bounded variety, and by Chapters 6 and 7, bounded quite tightly: a scarce context budget, an expensive build cost, and a limited number of properties anyone will express as checks.

State it as a result, because I think it is the most useful theoretical claim in this book:

The Ashby constraint on harnesses. A harness can only regulate variety it has a model of. An unconstrained generator has unbounded variety. Therefore either the harness grows without bound, or the variety of what may be produced is deliberately reduced. Since the first is forbidden by the context budget, the harness cost gradient, and depreciation, the second is forced.

Böckeler names the move — committing to a defined set of service topologies narrows the possibility space, making comprehensive harnessing achievable, and she calls this a deliberate variety-reduction move. What the Ashby framing adds is that it is not one option among several. It is the only available option, and every practice that looks like a matter of engineering taste is revealed as a technique for satisfying a constraint:

And it yields a diagnosis for a common and expensive organisational contradiction. A platform group that intends to let every team choose its own language, framework, and architecture and to comprehensively harness the result is not attempting something difficult. It is attempting something Ashby forbids. The variety of the regulated system is the union of every team's choices; the regulator is one platform team's finite output. No amount of effort closes that gap.

Note also the direction of the move. Variety reduction is subtractive — it makes things unexpressible. Taleb's via negativa appears here for the third time in this book, and its consistency is not a coincidence: in a system whose failure modes are unbounded, removing possibilities is the only intervention whose effect you can be confident about.

8.4 Harnessability is banked variety reduction

Böckeler's term for how amenable a codebase is to governance is harnessability, and she connects it to ambient affordances — structural properties that make a system "legible, navigable, and tractable to agents." Strongly typed languages support type-checking; clear module boundaries make architectural rules expressible; opinionated frameworks reduce the decisions an agent must make.

Ashby lets us say what harnessability is, rather than merely listing its symptoms:

Harnessability is variety reduction already banked. A strongly typed codebase has spent past effort making most programs unexpressible. A codebase with enforced layering has already forbidden most call graphs. That work is done; the regulator now has less to regulate.

This reframing has teeth, because it makes harnessability an asset with a balance rather than a vague quality. A team inheriting a well-structured codebase inherits accumulated variety reduction and can build a comprehensive harness cheaply. A team inheriting an untyped, unlayered codebase has no balance to draw on, and must either spend to create some or accept that comprehensive harnessing is unreachable.

Which produces Böckeler's inversion, and it is the cruellest sentence in the field:

"the harness is most needed where it is hardest to build."

Chapter 24 is devoted to this. Note here only that it is not a paradox but an accounting identity: the codebases that most need governance are precisely those that never banked any variety reduction, which is the same fact stated twice.

8.5 Keep quality left, made precise

Chapter 5 showed that shift-left assumes the detector outruns the change, and that human review no longer does. Chapter 7 gave the placement axis — blocking, in-loop, out-of-loop — ordered by which resource each spends. This section combines them into the distribution, and pays the ordering-rule debt from Chapter 1.

Two rules govern the distribution.

The ordering rule (promised in Chapter 1): use the cheapest deterministic sensor that can detect the failure class, and reserve inferential sensors for what genuinely requires judgment. A linter that catches a defect every time beats a review agent that catches it four times in five — not marginally but categorically, because you can build a policy on the first and only a hope on the second. A control with a known false-negative rate cannot gate a merge; it can only advise.

The placement rule (from Chapter 7): push every control as far left and as close to the agent's action as its speed allows. Chapter 2 priced the leftmost position — a millisecond check that refused a broken edit was worth three points and prevented a cascade that dropped recovery odds from 90.5% to 57.2%.

The fourth stage deserves emphasis because it is the one organisations do not have. Continuous drift detection has no triggering event. Nothing arrives to prompt it. It is the mechanism Chapter 4 argued must replace the homeostat that delegation removed, and by construction it requires a standing mandate — an agent or a job that runs because it is scheduled, not because someone noticed. OpenAI's reported "garbage collection" scans and Thoughtworks' "janitor army" are both instances [D]. Chapter 12 places this as the outer loop's Observe phase, which is where it belongs organisationally: it is not part of anyone's task.

8.6 Three regulation categories

Böckeler divides the harness by what it regulates, and the division is honest about maturity in a way most frameworks are not.

Maintainability is the solved one. Computational sensors reliably catch duplication, complexity violations, coverage gaps, and style. This is where Chapter 7's cost gradient makes progress easy and where most published harness advice lives.

Architecture fitness is expressible but under-practised. Fitness functions — Ford, Parsons, and Kua's contribution — turn architectural intentions into executable checks: this layer may not import that one, this module's fan-out may not exceed n, this service may not talk to that database. They are ordinary computational sensors pointed at structure rather than at syntax. Böckeler gives working examples: skills feeding forward performance requirements with performance tests providing feedback; logging standards paired with debugging instructions that prompt reflection on log quality.

Behaviour is the one that matters and the one nobody has. It asks whether the software does what was wanted, and Böckeler's assessment is unhedged: current practice uses functional specifications as feedforward and AI-generated test suites as feedback, which "puts a lot of faith into the AI-generated tests, that's not good enough yet." And: "we still have a lot to do to figure out good harnesses for functional behaviour that increase our confidence enough to reduce supervision and manual testing."

That last clause identifies the real cost. Until the behaviour harness works, supervision cannot be reduced — which means Chapter 3's constraint stays binding and Chapter 13's autonomy ladder cannot be climbed. Every ambition in Part III is gated on the bottom row of Figure 8.4.

8.7 Why generated tests cannot be a specification harness

The circularity deserves precise statement rather than a warning, because being precise about it reveals what generated tests are good for — which is a great deal.

The distinction that falls out is clean and, as far as I can find, not stated anywhere:

AI-generated tests are a regression harness, not a specification harness. They detect future divergence from today's behaviour. They cannot detect today's divergence from what was wanted, because they were derived from the same model of intent as the behaviour itself.

This is not a dismissal — it is a scoping, and the scope is genuinely valuable. A regression harness protects against Chapter 4's accumulation, catches the cascade when a later change breaks an earlier assumption, and lets an agent self-correct in-loop at Chapter 7's level 2. Generated tests do all of that well and cheaply. What they cannot do is tell you the feature was right in the first place.

Which is exactly the failure class Böckeler lists as caught by neither column: misdiagnosis of the underlying problem, over-engineering, unrequested features, and correctness against an intent that was never clearly specified. All four are failures where code and tests agree perfectly.

And it explains a phenomenon every reader will recognise: a pull request with comprehensive passing tests, reviewed and merged, that turns out to have solved the wrong problem. Nothing malfunctioned. The harness did its job. Its job was consistency.

8.8 The transplant

Chapter 1 promised this chapter would attempt an import from Meaning A. Here it is, with an honest account of what survives the journey.

In Agent-SDLC nobody would let the system under test author its acceptance criteria. The evaluation set is built first, by humans, from stakeholder-aligned success metrics, before implementation exists. IBM's published framing has stakeholder alignment on success metrics explicitly "to inform an evaluation framework" [D].

What transfers:

The ordering. Acceptance criteria authored before generation, not after. This is not a documentation preference; Figure 8.5 shows it is the only point at which human intent can enter the system uncontaminated by the generator's interpretation.

The held-out set. This is the mechanism, and it resolves the circularity structurally rather than by exhortation. Partition the tests:

The agent can iterate against the first set indefinitely — which preserves everything valuable about in-loop sensors — while the second set is the only thing whose passing licenses a merge. Because the agent never sees the held-out set, it cannot optimise against it, which is precisely the property that makes it evidence rather than a target. Goodhart's law, pre-empted by construction.

What does not transfer:

Distributional evals do not fit a specific feature. Meaning A's eval sets sample from a distribution of user requests, and their statistical logic depends on that. A feature has a particular intent, not a distribution over intents, so the held-out set is small, hand-made, and unrepresentative in a way an eval set is not. It is a gate, not an estimate.

The cost is real and falls on the scarce resource. Someone must write the acceptance criteria by hand, before the work, without the agent's help — and that someone is the reviewer whose capacity Chapter 3 identified as the constraint. This is the honest objection to the transplant, and the honest answer is that it is the correct place to spend the constraint: Chapter 5 relocated human judgment from the instance to the specification, and this is what that relocation costs in practice.

It bounds the elephant; it does not remove it. A held-out set checks the intent that was articulated. Chapter 0's line stands unamended: a harness cannot verify correctness against a requirement that was never articulated. The transplant converts an unbounded problem into a bounded one — you are now exposed only to what you failed to say, not to what the generator failed to infer. That is a genuine improvement and it is not a solution.

8.9 The silent sensor, and harness drills

Böckeler poses an open question that sounds almost rhetorical and is in fact the deepest problem in the chapter:

If a sensor never fires, is that quality or blindness?

By observation of the sensor alone, these two cases are indistinguishable. A test that always passes because the code is correct and a test that always passes because it asserts nothing produce identical output forever. A guide that is followed and a guide that is ignored produce identical artefacts — nothing. A linter misconfigured to check no files reports clean.

This is Taleb's silent evidence in its purest engineering form, and worse than the usual case: absence of failure reports is not merely weak evidence of safety, it is exactly what a broken detector produces. The turkey of Chapter 3 at least had data. A silent sensor manufactures data that looks like safety.

So the harness has no way to audit itself, and Chapter 7's third ceiling stands. Böckeler names the missing instrument directly: how do we evaluate harness coverage and quality, analogous to code coverage metrics? The literature lists this as unclaimed. So let me claim it, because the answer already exists in a neighbouring discipline.

Code coverage tells you a line was executed. Mutation testing tells you something far stronger: deliberately introduce a defect, and check that a test fails. If no test fails, the mutant survives, and you have learned that your suite does not actually constrain that behaviour — regardless of what coverage claimed.

The harness analogue is direct:

Harness mutation testing. Deliberately produce the failure a control was installed to prevent, and verify the control catches it. A failure mode that survives is uncovered, whatever the configuration file says.

And the practice, small enough to actually run:

A harness drill. Take one control. Construct the failure it exists to prevent. Attempt it. Record what happened.

Three outcomes, all actionable, and — this is why the mechanism matters — they serve both arms of Figure 8.1:

The third branch is the one I want to draw attention to, because it closes a debt from Chapter 2. That chapter argued the optimal harness shrinks as models improve, and prescribed re-testing controls against current models on a schedule — but gave no mechanism for deciding when to remove one. The drill supplies it: disable the control, attempt the failure, and if the failure no longer occurs even unguarded, the control is compensating for a deficit that has closed. Retire it. This is precisely mutation testing's own method — remove the test, see whether the mutant survives — applied one level up.

Three notes on making drills real rather than aspirational.

They are cheap in the only currency that matters. A drill costs agent time and almost no human time: the agent constructs the failure, attempts it, reports. This is the pattern from §8.1 — point the agent at the harness.

They should be scheduled, not triggered. Like continuous drift detection, nothing arrives to prompt a drill. It belongs to the outer loop with a standing mandate, and Chapter 12 places it there.

They are the unit Chapter 14 measures. Harness coverage — the proportion of known recurring failure modes with a control that has demonstrably fired — is only computable if drills are running. Without them, harness coverage is a count of configuration entries, which is exactly the vanity metric this book should not propose.

One further coherence problem, and a shorter answer. Böckeler asks how far we should trust agents to navigate conflicting instructions. My answer is: not at all. A contradiction between two guides is a defect in the harness, and delegating its resolution to the agent means accepting a silent, unlogged, per-invocation decision about which of your stated intentions wins. Chapter 7 gave the scale of the exposure — repositories layering CLAUDE.md, AGENTS.md, rules, and instructions, with CLAUDE.md pointing to AGENTS.md in 301 cases. Contradiction detection across layered context files is on Galster's own list of what the field needs and does not have; Chapter 21 names the condition Guide Sprawl and Chapter 27 keeps it open.

8.10 Harness templates

If variety reduction is forced (§8.3), the question becomes how to make it reusable. Böckeler's answer is the natural extension of a practice most organisations already have.

Mature organisations codify common service shapes — a data dashboard, a CRUD business service, an event processor — into service templates. The extension is a harness template: in her words, "a bundle of guides and sensors that leash a coding agent to the structure, conventions and tech stack of a topology."

The Ashby reading makes the value precise. A service template reduces variety once, for one service. A harness template reduces variety and ships the regulator sized to the reduced system. It is the difference between "here is a starting point" and "here is a starting point together with the controls that keep it in shape."

This also connects to the best-validated organisational finding available. DORA's seven capabilities include quality internal platforms [C], and a harness template is what that capability looks like once agents are the primary labour. Chapter 16 takes up who owns it.

Two honest caveats, one of which is a real threat.

Templates inherit every problem of service templates — versioning, drift between the template and its instances, contribution and ownership. These are known problems with known partial solutions, and they are not new.

The threat is portability. Chapter 2 found that the optimal interface differs by model: Claude 3 Opus beat GPT-4 Turbo at retrieval and lost inside SWE-agent's interface. If harness–model fit is genuinely model-specific, then a harness template is partly a bet on a model, and every model migration silently invalidates some of it. The drills of §8.9 are the only defence I can see: they convert an invisible invalidation into a failing check. Whether that is sufficient is unmeasured, and it is on Chapter 27's list.

Böckeler's own extrapolation is worth sitting with: teams may eventually choose architectures partly on the basis of which harnesses exist for them. That is variety reduction becoming self-reinforcing — the well-harnessed topologies get chosen, which makes them better harnessed. It is either a virtuous cycle or a monoculture, depending on your temperament, and Chapter 24 argues it is both.

8.11 Making valve 4 deliberate

Chapter 3 left this chapter a specific debt. A saturated review queue admits four responses; only the fourth — changing the server, mechanising part of verification — raises capacity. The enterprise studied there pulled it: automated review overtook human review, with bots reaching roughly 38% of review events and half of all comments [B]. But they did it under pressure, unnamed, unmeasured, and without deciding how much to trust it.

Everything in this chapter is what "deliberate" means:

• Bounded by the ordering rule (§8.5): inferential review agents take the judgment work, and nothing that a deterministic sensor could have gated.
• Placed by Figure 8.3: as far left and as close to the agent's action as speed allows, so machine review is preventing rather than reporting.
• Trusted in proportion to demonstrated performance — which requires drills (§8.9), because an unfired review agent is Böckeler's silent sensor wearing a more expensive hat.
• Scoped by Figure 8.4: automated review can carry maintainability, can partly carry architecture fitness, and cannot carry behaviour. An organisation that lets bots absorb its review load without a held-out acceptance set (§8.8) has automated consistency-checking and called it review.
• Retired when it stops earning its place (§8.1–8.2).

That last scoping point is the one I would most want an engineering leader to take from this chapter. Substituting machine review for human review is the correct move and the only one that raises capacity. It is also the move most likely to be over-scoped, because the failure it cannot catch — the feature that was built correctly and should not have been built — is invisible to it by construction, and produces no signal at all until much later.