Chapter 11 — Verification & Security

"The allowlist made the attack easier by auto-approving the very commands that could be hijacked through environment variable poisoning."

— on CVE-2026-22708, Cursor terminal allowlist bypass [C]

A security control that increased the attack surface, in a product built by people who were paying attention. This is the chapter's subject in one sentence.

11.1 The tractable half

Chapter 8 left the behaviour harness unsolved and named the reason: you cannot verify correctness against intent that was never articulated. That is a gap in the human, and no apparatus closes it.

Security looks like the same shape of problem — proving the absence of something — and it is not. The two differ in a way that matters enormously for where to spend effort:

That asymmetry is the reason this chapter is more optimistic than its subject matter suggests. Behaviour verification requires eliciting something that does not exist. Security verification requires retaining something we already have and throw away: the knowledge that this paragraph came from an issue comment and that one came from the compiler.

So: security is the tractable half of verification. It is urgent, it is currently handled badly, and unlike Chapter 8's elephant it is not waiting on a research breakthrough. It is waiting on engineering.

11.2 The read set is the attack surface

Chapter 5 established the change in one line: previously an attacker needed write access to a trusted repository; now they need the ability to open an issue.

State it generally. The attack surface of a conventional software project is its write set — the artefacts an adversary must modify to affect behaviour, guarded by authentication, review, and branch protection. The attack surface of an agentic project is its read set — everything the agent consumes in the course of doing its job.

Two observations follow, and the second is the one I have not seen stated.

Nobody has this inventory. Engineering organisations can produce a list of who has commit access within minutes. Ask which artefacts an agent reads in a typical session, and who can write to each, and there is no document. That is a security posture defined by omission.

Now notice the connection to Chapter 6. That chapter asked you to itemise your context payload — how much is system prompt, how much retrieved code, how much tool output, how much history — because you cannot allocate a budget you have not itemised.

The context payload and the attack surface are the same object. Chapter 6 asked you to itemise it to manage a budget. This chapter asks you to itemise it to manage trust. One inventory, two disciplines, and most organisations have neither.

And read surfaces are frequently also write surfaces. This is the part the literature underplays. In the claude-code-action disclosure (§11.4), the malicious issue was both the injection vector and the exfiltration channel — the attacker's instructions arrived via the issue, and the stolen secrets were written back into the issue for collection [C]. Anything an agent can both read and write is a bidirectional channel, and a threat model that treats reading and writing separately will miss the loop that closes between them.

11.3 Prompt injection is structural

The OWASP GenAI Security Project published the Top 10 for Agentic Applications on 9 December 2025 as a peer-reviewed framework [C]:

Goal hijack is ranked first, and the stated reason is the important part: because agents cannot reliably distinguish instructions from data, a single malicious input can redirect an agent to perform harmful actions using its legitimate tools and access.

Read the clause after the comma twice. The attack does not grant the attacker new capability. It borrows the agent's. Whatever your agent may do, an attacker who reaches its context may do — which means every capability you grant is a capability you have granted to anyone who can write into the read set.

OWASP's framing, echoed across its 2026 material, is that prompt injection should be treated as a structural flaw rather than a bug 🟡, and that it continues to drive most agentic security failures observed in production 🟡. That framing is correct and it has a specific engineering consequence which the rest of this chapter develops: a structural flaw cannot be patched by filtering. If instruction and data are not separable in the channel, then no amount of cleverness at the boundary separates them, and the defence has to move somewhere else.

11.4 Two incidents, read for mechanism

Incident narration is usually alarm. These two are worth reading for mechanism, and — more usefully — for which fixes worked.

The allowlist that helped the attacker

CVE-2026-22708, Cursor, prior to version 2.3 [C].

Cursor's Auto-Run Mode with an allowlist is a sensible control: enumerate the commands the agent may run without asking, and prompt the human for everything else. It is Chapter 7's blocking sensor, applied to shell execution.

The bypass had two parts, and both are instructive.

Part one: the allowlist validated names, not behaviour. External commands were properly checked against the allowlist. Shell built-ins — export, unset, set — were not, because they execute within the shell session rather than as separate binaries. A whole class of operation fell outside the control's model. This is Ashby's corollary from Chapter 8 arriving as a vulnerability: a regulator can only regulate what it has a model of, and this regulator's model was "commands are executable files."

Part two, which is the real lesson: the attacker did not need to run a forbidden command. They changed what a permitted one meant. Prompt injection poisoned environment variables — PATH, LD_PRELOAD — so that the next time the agent ran an already-approved command such as git branch, it silently executed attacker-controlled code instead. The vulnerability permitted remote code execution even when the allowlist was empty.

This is Taleb's iatrogenics with a CVE number: harm produced by the intervention. And it is precisely the shape Chapter 2 found in SWE-agent's iterative search, which scored 12.0% against 15.7% for having no search at all — a well-built capability that performed worse than its own absence. Chapter 5 promised this chapter would deliver a control that enlarged the attack surface; here it is, with the vendor's own remediation confirming the diagnosis: Cursor now requires explicit user approval for any command the server-side parser cannot classify. The fix was to stop assuming that anything unclassified was safe.

The general principle, which §11.5 will formalise:

A control that converts "ask a human" into "proceed silently" has traded interruption for exposure. That may be the right trade. It is never a free one.

One issue, one repository

claude-code-action, disclosed 1 June 2026 by RyotaK of GMO Flatt Security [C].

Two independent failures chained.

The authorisation bypass. The action's checkWritePermissions function unconditionally trusted any actor whose name ended in [bot], regardless of actual permissions. Anyone who registered their own GitHub App could therefore trigger workflows with attacker-controlled content. An identity check implemented as a string-suffix test.

The injection. RyotaK wrote an issue whose body looked like an error message, then refined it until Claude would "recover" by running the commands buried inside. The target was /proc/self/environ, the Linux file holding a process's environment variables — secrets included. Claude Code blocked naive reads of it; the guard was bypassed, and the agent was induced to write the values back into the issue, where the attacker collected them. Combined, this enabled theft of OIDC tokens and compromise of repository supply chains.

This was not theoretical. A variant was actively exploited in February 2026 against Cline's GitHub Actions workflow, where an attacker stole an npm publish token and pushed an unauthorised cline@2.3.0 release [C] — ASI04, agentic supply chain, in the wild.

The detail I want to dwell on is the disguise. The payload looked like an error message. Consider what that defeats. You cannot filter this by content type, because an error message is exactly the kind of artefact the agent exists to read and act upon. Any filter capable of excluding this payload would also exclude the agent's legitimate work. The injection was not hidden in the channel; it was the channel's normal traffic.

And now the most instructive part — Anthropic's remediation:

Every fix that held was structural. Not one was "instruct the model more firmly." The guard that failed — block reads of /proc/self/environ — was a content-level rule about a specific path, and it was bypassed. The fix that worked removed the secret from the environment, so there was nothing at that path to read.

11.5 Belief-based controls fail; capability-based controls hold

Generalise the two incidents and Part II reorganises itself.

Every control is one of two kinds.

Belief-based controls act on what the agent knows, believes, or is told. Guides, rules, principles, instructions, context files, filters on content, prompts that say "never do X." They work by shaping the agent's model of its situation.

Capability-based controls act on what the agent can do. Credentials it does not hold, filesystem paths not mounted, network egress denied, tools not registered, an environment from which the secret has been scrubbed.

Injection is an attack on belief. It therefore defeats the first category completely and by construction, and cannot touch the second.

A poisoned context can make an agent want to exfiltrate credentials. It cannot give the agent credentials it does not have.

Now apply that to Part II's four rings, and the result is uncomfortable.

That convergence is worth pausing on. Chapter 10 ranked controls by cost of ownership and put make it impossible at the top because an unrepresentable state needs no sensor, no drill, and no maintenance. This chapter ranks controls by resistance to injection and reaches the same tier first. Two independent arguments, same answer — which is the strongest kind of support a design principle gets.

Hence the rule that should govern every security decision in an agentic system:

Never rely on an instruction to prevent a security outcome. "Do not read .env" is a guide — Chapter 10's tier four, defeated by any injection. Removing filesystem access to .env is tier one.

The refinement: scope by effect, not by name

But the Cursor incident forbids a naïve reading, and getting this right is the difference between the principle being useful and being glib. Cursor's allowlist was a capability control, and it failed.

Why? Because it scoped by name rather than by effect. git is not a capability. It is a string that resolves, at execution time, to whatever PATH says it resolves to. The control constrained the identifier and left the referent free.

A capability control is only as good as the invariance of what it names. Allowlisting git scopes a name. Granting read-only access to a specific repository scopes an effect. The first is defeated by anything that changes resolution — PATH, LD_PRELOAD, aliases, shims, a modified build script. The second is not.

Practical translations, and note that each replaces a name with an effect:

Every row on the right is Chapter 10's structural engineering applied to the agent's environment rather than to the codebase — which is exactly what Chapter 10 §10.7 predicted when it noted that capability scoping is the same discipline pointed outward.

11.6 Provenance, the missing primitive

Figure 11.3 leaves three rings permeable. That is not inevitable, and the reason they are permeable is a design choice nobody made deliberately: the context payload discards provenance.

By the time text reaches the model it is a flat sequence. The system prompt, a file the agent read, the output of a test run, a paragraph from issue #4412, and a fetched web page all arrive in the same undifferentiated stream. The model has no reliable way to know which is which, and the strongest reason it cannot separate instruction from data is that we did not tell it which was which.

Chapter 9 reached the same conclusion from a different direction. A knowledge graph mixing AST-derived edges with edges extracted from issue text launders the provenance of the latter — content enters as an untrusted comment and leaves as "a fact from the code graph," in the same typed result set and with the same apparent authority. The remedy there was to type every edge by provenance and never let asserted edges gate what derived edges could.

That remedy generalises to the whole payload, and it is the constructive proposal of this chapter:

1. Tag every segment of the payload with its origin at assembly time, when the origin is known for free.
2. Carry the tag into the context so the model sees "the following text came from an issue comment authored by an unauthenticated account" rather than undifferentiated prose.
3. Make trust level a first-class input to the harness, so a rule can say: content below trust level n may inform, and may never authorise a tool call.
4. Keep the tags out of the untrusted content's control — which means the framing markers must be structurally unspoofable, not merely conventional, or injection simply forges them.

Point four is where this gets genuinely hard, and I will not pretend otherwise. A model that trusts a delimiter can be persuaded that the delimiter appeared. Making provenance markers unspoofable in a flat token stream is an open problem, and it may require architectural support rather than prompt convention.

But notice the shape of the situation. Behaviour verification (Chapter 8) is blocked on eliciting information that does not exist. Provenance verification is blocked on carrying information that exists, is free, and is currently discarded. Those are not equally hard problems, and the second is where the field's effort should go.

11.7 What benchmarks cannot tell you

Chapter 2 established that SWE-bench is a casino: it supplies ground-truth tests that real work lacks, 32.67% of its successful patches involved solution leakage, 31.08% of passing patches had weak tests, and the founding result fell from 12.47% to 3.97% once problematic instances were filtered [B].

For security assurance the problem is worse, and it is worse in a way that is structural rather than fixable.

Benchmarks measure the average of a curated distribution. Security failures are tail events. Taleb's distinction is exactly load-bearing: an average is a meaningful summary in Mediocristan, where no single observation dominates the total. Security lives in Extremistan, where one event does. A tool that behaves correctly on 10,000 curated tasks and catastrophically on the 10,001st has a superb benchmark score and an unbounded loss.

You cannot sample your way to tail safety. This is not a criticism of any particular benchmark; it is a statement about what sampling can establish. Chapter 21 names the procurement failure Benchmark Cargo Cult, and this is its most expensive form — selecting an agent for a regulated system on the basis of a capability score.

And security has no ground truth at all. SWE-bench at least hands you the maintainers' tests. There is no enumerable set of attacks to test against, because the attack set is adversarial and open — it grows in response to your defences. A held-out acceptance set (Chapter 8) works because intent, once stated, is fixed. A held-out attack set does not, because the adversary reads your fix.

What the field actually lacks, stated as work someone should do:

The honest summary: benchmarks tell you something real about capability on curated tasks, and nothing about safety in deployment. Use them for the first and never for the second.

11.8 Drills, pointed at an adversary

Chapter 8 argued that a sensor which never fires is indistinguishable from a sensor that is blind, and proposed harness drills — deliberately produce the failure a control exists to prevent, and verify the control catches it — as the harness analogue of mutation testing.

Security is where this discipline already has a name and a professional practice: red-teaming. The contribution of Chapter 8's framing is to say what makes it work as an engineering control rather than an event:

• Scheduled, not triggered. Nothing arrives to prompt a drill. It belongs to Chapter 12's outer loop with a standing mandate.
• Constructed from your own read set (§11.2), not from a generic list. The drill is: place a payload in each artefact your agent reads, and see whether it changes behaviour.
• Measured as coverage over the read set, which is Chapter 14's harness coverage with a security column.
• Cheap in the currency that matters. The agent constructs and attempts the injection; the human reads the result. Chapter 8's rule again — point the agent at the harness.

And one asymmetry particular to security. A behaviour drill that fails to reproduce a failure is weak evidence that the failure cannot occur. A security drill that fails to reproduce an attack is no evidence at all that the attack cannot occur, because the adversary is not sampling from the same distribution you are. Drills establish that specific known vectors are closed. They never establish that the surface is clean, and any process that reports "red team found nothing" as assurance has misunderstood what was measured.

11.9 Accountability after authorship

The governance problem, and it is sharper than the literature admits.

Accountability in software engineering travelled with authorship. The person who wrote the code owned it, was called when it failed, and carried the reputational consequence. Review added a second name, but authorship was the anchor.

Authorship is now machine. So accountability defaults to the next available signal, which is approval — the person who clicked merge.

But Chapter 5 demonstrated that approval is precisely the gate whose informational content collapsed. It was a checkpoint by cost; the cost is gone; and Chapter 3 measured reviewers absorbing double the load while batch sizes grew by roughly a third.

We have attached responsibility to the weakest remaining signal. The person accountable for an agent-authored change is the person whose ability to evaluate it we have most thoroughly degraded.

This is a skin in the game failure in Taleb's precise sense — not an absence of accountability, but a misalignment between who bears the consequence and who holds the decision. And the consequence is predictable: reviewers who cannot genuinely evaluate what they are accountable for will either approve defensively slowly, becoming the bottleneck, or approve nominally, which Chapter 21 calls Review Collapse.

The way out is to move accountability off the instance and onto the decisions that actually determine outcomes:

This is why Chapter 13's autonomy ladder is a governance instrument and not a maturity poster. An explicit statement that a class of change may proceed at a given autonomy level is a decision someone made, can be asked about, and can be revised after an incident. "The reviewer approved it" is not.

Two honest limits. The AI Agent Index reportedly documents deployed agentic systems alongside a marked deficit in governance documentation 🔴, and making human–agent responsibility mapping auditable is an open problem — Chapter 27 keeps it open. Figure 11.5 says where accountability should attach. It does not tell you how to prove, after an incident, which control was in force at the time, and that evidentiary problem is unsolved.

11.10 Part II closed

The four rings are built and the fifth concern runs through all of them.

Part II has argued that each ring exists because the one inside it saturates, and that each outer ring makes the inner ones cheaper. This chapter adds the adversarial reading: each outer ring is also harder to talk past. Context can be poisoned, guides can be forged, retrieved facts can be laundered — and a capability the agent does not have cannot be argued into existence.

Which is the strongest possible endorsement of Chapter 10's ordering. Make it impossible is not merely the cheapest control to own. It is the only one that survives an adversary.

Part III now assembles these pillars into a lifecycle: two loops, an autonomy ladder that turns Figure 11.5's governance requirement into an instrument, a metric system that makes harness coverage and drill results measurable, a maturity path, and an argument about who in an organisation owns any of it.