Context, Memory, and Retrieval for Agents

Meet Scout, a support agent wired to a local knowledge base with no freshness or provenance check — search, read, answer, done. On Tuesday it picked up ticket #5290: a return-window question for order #71204.

Press play. Scout does everything a careful reader would do — it retrieves, it reads, it quotes. Watch for the moment that carefulness stops mattering.

Scout's failure wasn't reading comprehension. It found a real document and reported its contents faithfully — the problem is that nothing in the run treated 'found' and 'trustworthy' as two separate questions.

Retrieval is a trust boundary, not a convenience feature. Anything that crosses it — a chunk, a snippet, a search result — carries the same burden of proof as an external input, because that's exactly what it is.

Here's the contract that would have stopped ticket #5290:

Context is the material available for one decision. Memory is information someone deliberately chose to persist beyond that decision. Retrieval is the selection step that proposes candidates for the context — it has no opinion about whether they're still true. Confusing any two of these is how stale documents end up sounding authoritative.

A bigger context window doesn't fix badly chosen evidence, and it can make things worse: models are measurably worse at using facts buried in the middle of a long context than facts placed near the start or end — the 'lost in the middle' effect. Dumping every candidate document into the prompt and hoping the important one gets noticed is not a retrieval strategy.

How a document gets cut into chunks matters as much as what it says. A chunk stripped of its heading, owner, or date loses exactly the metadata that would have gotten it excluded — which is how a technically-accurate sentence from a superseded file ends up quoted as current fact.

This is ticket #5290 again, retrieval now running under the provenance-and-freshness contract. First run it clean and watch the vet and cite gates do their work. Then switch to Break it and inject three classic retrieval failures yourself.

Everything above ran in the page. Now build it where it counts — your machine, real files, a real (fake) knowledge base. Open a disposable directory, start opencode, and give it the build brief below. Your job is to hold the contract while an agent writes the retriever; specifying what counts as current, authoritative, and citable is the actual curriculum here.

When your run produces a context packet, try the three injections from Break it against your own build. If your vet and cite stages catch all three, you've shipped the chapter's artifact. If they catch two, you've found your first real bug — better here than in a support queue.

What the context packet does not prove matters as much as what it does: it proves this pipeline admitted only scope-matched, freshness-checked chunks and cited only what they literally contain, on these three fixtures. It says nothing about whether your real index gets refreshed on schedule, whether your production chunking strips the dates you're relying on, or how the pipeline behaves on a knowledge base ten thousand documents larger. That gap is exactly why freshness has to be a running operational check, not a one-time contract you write and forget.

You watched an agent trust a stale document because nothing checked its date, wrote the provenance-and-freshness contract that would have stopped it, caught three injected retrieval failures with dumb-on-purpose gates, and built the pipeline yourself with an agent as your hands. The next chapter is what happens when your knowledge base doesn't fit in one page — durable memory across sessions, and the deliberate decisions about what an agent is allowed to carry forward and what it has to forget.