Chapter 6 — Context Engineering: the Budget, Not the Bucket

"Context Engineering, a formal discipline that transcends simple prompt design to encompass the systematic optimization of information payloads for LLMs."

— Mei et al., A Survey of Context Engineering for Large Language Models, arXiv 2507.13334 [B]

"Models do not process context uniformly… reliability drops as input length grows, even for tasks as simple as retrieval and text replication."

— Hong, Troynikov, Huber, Context Rot, Chroma, 18 models [D]

6.1 What this pillar is

Chapter 1 fixed the definition, and it is worth restating because almost everything written under this heading violates it:

Context is the complete information payload assembled for a single model call.

Not the wording of the instruction. The whole payload: the system prompt, the retrieved file contents, the outputs of tools called earlier in the session, the conversation so far, the project's context files, the schema of available tools, the task description. Everything the model sees at the moment it is asked to act.

The survey by Mei and colleagues, synthesising over 1,400 papers, defines the discipline as "the systematic optimization of information payloads" and organises it into three foundational components — context retrieval and generation, context processing, and context management — before treating system implementations: retrieval-augmented generation, memory systems, tool-integrated reasoning, and multi-agent architectures [B].

The distinction from prompt engineering is not a matter of scope but of object. Prompt engineering asks how should I phrase this? Context engineering asks what should be in the payload at all, given that it cannot all fit and that some of it will hurt? One is rhetoric. The other is allocation under scarcity, which is engineering.

Where this pillar sits: Chapter 1 established that harness engineering strictly contains context engineering. Context engineering designs what the model sees. Harness engineering designs the whole control system that decides what it sees, what it may do, and what checks it afterwards. Chapter 7 shows that most teams have built the inner discipline while believing they built the outer one.

And where it sits in the argument: Chapter 5 concluded that intent can no longer be recovered after generation by interrogating an author, so it must be externalised beforehand in a form a machine can consume. The context payload is where externalised intent lives. That makes this the first pillar not by importance but by necessity — if intent does not reach the model, nothing downstream can compensate.

6.2 The evidence that more is worse

Most writing on this subject argues from intuition. We have better than intuition, and it is already in this book.

Chapter 2 reported the SWE-agent ablations: the same model, the same benchmark instances, one interface dial varied at a time [A]. Two of those dials are pure context decisions.

How much of a file to show:

How much history to retain:

Read those tables as what they are: a controlled experiment in which giving the model more relevant information made it worse. The entire file contains everything the 100-line window contains, plus more. The full history contains everything the last five observations contain, plus more. Both supersets scored lower than their subsets — the file window by 5.3 points, the history by 3.0.

Nothing was removed to achieve the better score except quantity.

And a third dial, from the same table, shows the same effect arriving through a tool rather than a setting. Iterative search — page through matches one at a time — scored 12.0%, below the 15.7% for having no search capability at all, because agents exhaustively enumerated every result and filled their own budget with low-value content. A capability that let the model add context degraded it below not having the capability.

This is the empirical foundation of the chapter, and it is unusually solid for this field: controlled, model-held-constant, and — per Chapter 2's argument about symmetric contamination — expressed as a ratio between arms rather than an absolute level, so it survives the benchmark's known defects.

6.3 Two failure modes, opposite remedies

Now the framework, because "more is worse" is not actionable on its own. If more can be worse and less can also be worse, a practitioner needs to know which one they are suffering from.

There are exactly two ways a payload fails, and they are opposites.

Starvation. The payload lacks information the task requires. The model cannot succeed because the answer is not derivable from what it was given. It will not tell you this; it will guess, plausibly, and the guess will be wrong in a way that looks like reasoning. Signature in Chapter 2's table: 30 lines (14.3%) and no search (15.7%).

Dilution. The payload contains everything required, plus a great deal that is not, and the relevant material is no longer reliably attended to. The model has the answer available and does not use it. Signature: the entire file (12.7%), full history (15.0%), iterative search (12.0%).

Here is why the framework earns its place: the two failures present almost identically, and the remedies are opposite.

In both cases the model produces confidently wrong output. In both cases the obvious diagnosis is "it didn't know enough." A starved model genuinely did not know enough. A diluted model knew perfectly well and did not attend. And because the symptom is the same, the default response to both is the same — add more context — which fixes the first and makes the second worse.

I believe this is the single most common error in applied context engineering, and it is structurally invited: adding is easy, cheap, and feels responsible. Removing requires deciding what does not matter, which requires knowing the task well enough to be wrong about it. Taleb's via negativa — the observation that in complex systems removal is more reliable than addition, because you can be confident about the harm eliminated and only hopeful about the benefit introduced — is not a philosophical flourish here. It is the operating procedure, and Chapter 2 gave it a measurement: −5.3 points for adding the rest of the file.

The diagnostic. Distinguishing the two requires one experiment, and it is cheap:

Take a failing task. Cut the payload roughly in half, preserving what you believe is essential. If performance improves or holds, you were diluted. If it degrades sharply, you were starved.

Teams almost never run this, because halving the context feels like sabotage. It is the only test that distinguishes the two, and it costs one run.

6.4 Why the marginal token can be negative

The slogan of this chapter has to be earned rather than asserted, so here is the argument.

In a bucket, the marginal unit is neutral. Water added to a half-full bucket neither helps nor harms; it simply occupies capacity, and the only failure is overflow. Under that model the correct strategy is obvious: fill it, and worry only when it spills. This is precisely how most teams treat a context window, and it is why "we have a million tokens now" is greeted as the end of the problem.

In a budget, items compete, and the constraint is not capacity but what you had to give up. That is closer, but still not right, because a budget's marginal item is at worst worthless.

The context payload is worse than a budget. Its marginal item can be negative. An irrelevant token does not merely occupy space; it competes for attention with the relevant ones, adds candidate patterns the model may match against, and dilutes the instruction. Chapter 2 measured the sign: adding true, relevant, on-topic content — the rest of the very file being edited — cost 5.3 points.

Once the marginal item can be negative, there is no such thing as a free addition, and therefore no defensible default of "include it in case." Every inclusion is a decision with a cost. That is what makes this engineering rather than configuration.

Let me push it one step further, because a decision rule falls out.

Suppose performance depends on the payload's relevance density — the share of the payload that bears on the task — as well as on whether the payload is sufficient at all. Then adding an item raises the numerator only if it is relevant, and always raises the denominator. An item whose relevance is below the current average lowers the density. Which gives:

The inclusion rule. Add an item to the payload only if it is more relevant to this task than the average of what is already in there. Otherwise you are trading attention for coverage at a loss.

Two consequences worth sitting with.

The bar rises as the payload improves. A well-curated context is harder to add to than a sloppy one, because the average it must beat is higher. This is counter-intuitive and correct, and it explains a common experience: the first few additions to a context file help enormously, and later additions seem to do nothing or harm. They are being measured against a higher average.

Sufficiency and density trade off, which is why the optimum is interior. Starvation is insufficiency; dilution is low density. Maximising density alone gives you the 30-line window — beautifully dense, and lacking what the task needs. Maximising sufficiency alone gives you the whole file. The peak is where marginal relevance crosses the average.

I should be clear about the epistemic status of this. It is a model, not a measurement — a mechanism that explains the observed non-monotonicity and yields an actionable rule. It is falsifiable in a specific way: it predicts that adding highly relevant content to an already-dense payload should still help, while adding marginally relevant content to a sparse payload should also help. If instead performance depends only on total token count regardless of content, the model is wrong. Nobody has run that experiment cleanly, and it is on Chapter 27's list.

6.5 The mechanism: context rot

Why should relevance density matter at all? A perfectly attentive reader with unlimited patience would be unharmed by irrelevant material.

Two independent lines of evidence say models are not that reader.

Lost in the middle. Liu and colleagues showed that models use information in long contexts unevenly, with material positioned in the middle attended to less reliably than material at the beginning or end [A] 🟡. The SWE-agent authors cite this directly as the mechanism behind their file-viewer and context ablations — which is worth noting, because it means the 2024 ablation was not a surprise to its own authors; it was a prediction confirmed.

Context rot. Chroma's technical report evaluated 18 state-of-the-art models — including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 — on tasks deliberately kept trivial in order to isolate the effect of length alone: retrieval with semantically rather than lexically matched targets, variations in distractor content, conversational question-answering on LongMemEval, and a synthetic task of replicating repeated words. The finding: models do not process context uniformly, and reliability drops as input length grows even on tasks as simple as retrieval and text replication [D].

A disclosure the field usually omits: Chroma sells a vector database. A finding that long context degrades and retrieval is therefore necessary is commercially convenient for them, and this book's grading treats it as a vendor claim accordingly.

Two things nonetheless make it credible. First, the methodology is built to be hard to dismiss — if degradation appears on word replication, it is not an artefact of task difficulty. Second, it agrees with an entirely independent source that had no such interest: the SWE-agent ablation, from an academic group measuring an interface, found the same shape without looking for it.

That convergence is the strongest evidence available here, and it matters more than either source alone. A vendor finding corroborated by an incidental academic ablation is in much better shape than a vendor finding standing on its own.

The practical consequence, stated as plainly as I can:

Absence of a limit is not absence of a cost. A million-token window does not mean a million tokens is free; it means nothing will stop you.

This is the timely point of the chapter. Every capacity increase is read by the market as making context engineering less necessary. It makes it more necessary, because the distance between what you can include and what you should include is now enormous, and nothing in the tooling will warn you.

6.6 The four levers

Practitioners have converged on four techniques for managing long-horizon context. They are usually presented as a list. They are more useful mapped against the failure mode each addresses.

Compaction is the best known and the most over-relied-upon. It reduces token count while preserving gist, which helps dilution — but it is lossy in a direction you do not control, and the thing summarised away is often the specific detail that mattered.

Tool-output offloading deserves more attention than it gets, because Chapter 2 measured the failure it prevents. Iterative search scored below no search because the agent filled its own budget with search results. The general pattern: any capability that lets an agent add content to its own payload is a dilution risk, and the remedy is to make tool output land somewhere retrievable rather than somewhere resident.

Progressive disclosure is the most important, and the only lever that addresses both failure modes at once. Carry a pointer — a file path, a symbol name, a summary with a reference — and fetch the body only when needed. Density stays high because nothing unnecessary is resident; sufficiency remains achievable because everything is reachable.

This is worth naming as a general principle, because the rest of Part II is built on it: the answer to partial observability is not to show the agent more; it is to make more findable. Chapter 9's graph engineering is progressive disclosure with a typed index. The SKILL.md convention — a short file with references/ and scripts/ directories loaded on demand — is progressive disclosure as a file format. And McMillan's finding is the empirical version: multi-file organisation and retrieval strategy affect agent accuracy more than the choice of serialisation format [B] 🔴 (unverified — retrieve before print). How you arrange and reach material matters more than how you encode it.

Full context reset is the crude lever and sometimes the only correct one. Anthropic's published long-running-agent pattern is the clearest articulation: an initializer agent builds a structured environment — a feature list, a repository, a progress file — and a coding agent makes incremental single-feature progress, with the framing that these are engineers working shifts, each with no memory of the last [D] 🔴. The load-bearing consequence is that compaction alone is insufficient: if the context will be discarded, everything durable must live in artefacts outside it. Which is, once again, Chapter 5's conclusion arriving from a different direction — intent must be externalised, because the thing holding it is temporary.

6.7 Contexts decay

A finding that reframes the whole pillar: the context artefact is not stable. It degrades, and it degrades in a characteristic direction.

The Agentic Context Engineering work treats contexts as evolving playbooks, refined through a generate → reflect → curate loop, and names two failure modes of that loop [B] 🔴 (figures unverified):

Context collapse. Repeatedly rewriting a context artefact degrades it. Each rewrite is a lossy re-expression, and the losses compound toward genericity — the specific, hard-won, expensive detail is exactly what a rewrite drops, because it looks like noise to anything summarising.

Brevity bias. The rewriting process trends toward shorter and more general text, because concision reads as quality to the summariser.

Their remedy is structural rather than exhortative: incremental structured updates rather than wholesale rewrites.

Notice the shape, because Chapter 4 spent itself on it. A codebase under purely additive change grows monotonically in complexity and needs a homeostat. A context artefact under repeated rewriting decays monotonically toward genericity and needs the same thing — a mechanism that senses the deviation and corrects it without being asked.

So the practical rules follow from the mechanism rather than from taste:

• Never let an agent rewrite a context file wholesale. Edits, not regenerations. A regenerated AGENTS.md will be more readable and will have quietly dropped the three lines that were doing all the work.
• Date every guide with the failure it was added for. This is Chapter 2's prescription arriving again — a mature harness shrinks, guides age, and a guide whose failure mode no longer occurs is not free. It costs budget, and worse, it teaches everyone that the guides are decorative.
• Prune on a schedule, deliberately. If nothing is ever removed, Figure 6.1's dilution side is where you end up by default.

Corroborating practice: the Agent READMEs study finds that context files are actively maintained through small, frequent changes [B] 🔴 — which is what incremental updating looks like when a community discovers it by feel.

6.8 What teams actually do

Time for the reality check, because everything above describes a discipline and Chapter 7's evidence describes a practice, and they are far apart.

From the 2,853-repository study [A]:

• Context Files are present in 2,586 of 2,853 repositories (90.6%) and adopted by 61.5%–100% of each tool's users. They are, overwhelmingly, the mechanism.
• 4,768 context files in total: CLAUDE.md 1,640, AGENTS.md 1,508, copilot-instructions.md 1,393.
• Most repositories contain one or two context artefacts. Advanced mechanisms are rare: Skills in 158 repositories, Subagents in 131, both with a median of 2.
• 514 of 601 Skills (85.5%) bundle no additional resources at all — no scripts/, no references/. The authors' conclusion: Skills "function primarily as structured text rather than executable scripts."

And on content, Santos and colleagues' analysis of 328 CLAUDE.md files finds architectural concerns dominate, followed by development guidelines, project overviews, and testing [B] 🔴.

So in practice, context engineering means: one markdown file, describing mostly architecture, maintained by hand.

Two honest observations about that.

It is not obviously wrong. Architecture is exactly what an agent cannot infer from a local view, and Chapter 4 showed why — the cost gradient toward duplication exists because "find every call site" is a repository-wide question answered from a bounded window. A file that states the architecture is attacking the right problem with the cheapest available tool. The practice is thin, not misguided.

The evidence base under the field's most-adopted mechanism is one study. Lulla and colleagues report a controlled study finding lower runtime and token consumption at comparable task completion with an AGENTS.md present [B] 🔴. That is, as far as I can establish, the only causal evidence that context files help — supporting a practice adopted in 90.6% of repositories using these tools.

I want to be precise about what that means, because it is easy to read as debunking. It is not. It is a statement about the field's maturity: the practice raced ahead of the evidence, as Chapter 0 warned it had. The prior is reasonable, the mechanism is plausible, the adoption is near-universal, and the controlled studies number one. Chapter 27 lists this as an open problem, and it is a cheap one to close.

6.9 The budget discipline

The practical output of this chapter. Chapter 14 turns these into metrics; here they are habits.

Know your payload's anatomy. Most engineers cannot say, even roughly, how their agent's context is composed on a typical call — how much is system prompt, how much is context files, how much is tool output, how much is history. You cannot allocate a budget you have not itemised. Itemise it once and the dominant term is almost always a surprise.

Think in tokens the way you think in latency. No competent engineer adds a network call to a hot path without asking what it costs. The same reflex should apply to adding a paragraph to a context file: what is it displacing, and is it more relevant than the average of what is already there?

Default to pointers. Progressive disclosure as the standing convention, resident payload as the exception requiring justification.

Offload tool output. Anything voluminous goes to a file with a reference in context. Chapter 2 priced the alternative.

Edit, never regenerate. Contexts collapse.

Date and prune. Every guide carries what it was added for; guides whose failure mode has stopped occurring get removed.

Load richly, ask narrowly. This one comes straight from the survey's most striking finding, and it is verified: "While current models, augmented by advanced context engineering, demonstrate remarkable proficiency in understanding complex contexts, they exhibit pronounced limitations in generating equally sophisticated, long-form outputs" [B].

That asymmetry has a direct design consequence, and it is not the one people draw. It does not say keep the input small. It says the input side of the ledger is in better shape than the output side — so the constraint to design around is the size of what you ask for, not only the size of what you supply. A payload rich enough to support a narrow, well-specified change is the shape to aim for. A payload rich enough to support "refactor this module" is a payload whose output request exceeds what the model reliably produces. This is the mechanism beneath Chapter 12's Decompose phase: decomposition is not project management, it is compensation for a measured asymmetry.

6.10 Why this pillar saturates

The chapter ends by arguing its own insufficiency, which is the reason Part II has four rings rather than one.

Chapter 4 identified the mechanism driving structural decay: the agent duplicates rather than extracts because extraction requires knowing every call site, and that is a repository-wide question answered from partial observation. The natural inference is that better context would fix it — show the agent the whole repository and the gradient flattens.

Figure 6.1 says no. The whole file scored worse than a window of it. There is no reason to expect the whole repository to behave differently, and every reason from context rot to expect it to behave worse.

So partial observability cannot be solved by expanding the payload. It has three other remedies, and they are the remaining three pillars.

Chapter 9 is progressive disclosure with a typed index: rather than resident context, a navigable structure the agent queries. Chapter 10 is the most radical answer — change the codebase so that the context required to modify any module is bounded regardless of the repository's total size, which converts an unbounded retrieval problem into a bounded one. Chapters 7 and 8 accept that the agent will sometimes be wrong and build the apparatus that catches it.

Read in that order, the concentric rings stop being a diagram and become an argument: each ring exists because the ring inside it has a ceiling.