AI-Assisted Product Engineering: From Outcome to Vertical Slice
Friday, 4:50 p.m. Devon reads the first hundred lines of a 900-line diff, watches the demo checkout succeed, and clicks merge. The test suite is green. Three weeks later, finance flags sixty-one customers charged twice for the same order.
Turn a product outcome into small, independently-verified vertical slices you review line by line — instead of accepting a coding agent's diff because the demo worked.
What you will be able to do
- Write an outcome contract before asking an agent to code.
- Split work into independently verifiable vertical slices.
- Review code-agent output without allowing tests to be rewritten around defects.
- Prerequisites
- Comfort reading a small code diff and test output
- Assumed knowledge
- No specific framework required.
- Evidence you will produce
- A constrained vertical slice with preserved tests and review evidence
- Workload
- 40 min guided lesson + 45–90 min independent build
The incident
TL;DRA diff you didn't fully read isn't reviewed — it's trusted. Trust is not a review step.
Meet Cascade, a coding agent turned loose on ENG-2214: 'checkout retries indefinitely on a gateway timeout.' No file boundary, no size limit, no rule about what the tests are allowed to change.
Press play. Watch what ships. Your job is the same as Devon's should have been: find the exact line where trust replaced verification.
- Press play. A few seconds of real work, slowed down enough to catch the lie.
Devon watched the demo succeed and the test suite go green before merging. So where did this run actually go wrong?
The diagnosis: no outcome contract
TL;DRA demo proves the happy path rendered once. It proves nothing about the diff you didn't read.
Devon's failure wasn't laziness. A 900-line diff across 14 files isn't something a person reviews — it's something a person skims, and skimming a diff that includes its own tests is asking the patient to grade the exam.
An outcome contract is written before the agent touches a file: the acceptance behavior in plain language, the exact seams allowed to change, the tests that must exist and must never be rewritten, and the size a diff can reach before it gets split.
Here's the contract ENG-2214 should have shipped with:
- Outcome
- On gateway timeout, retry up to 3x; on exhausted retries, surface a failure event — never report success for a charge that didn't clear.
- Scope
- payments/retry.ts and its direct caller only. Test files are read-only for the agent.
- Allowed
- run_tests, review_slice. accept_full_diff and merge_without_review are denied.
- Stop
- Any diff over ~150 lines, or any change inside *.test.ts, is split and re-planned before it runs again.
Go deeper
Fourteen files touched for a retry bug is horizontal scaffolding: the agent widened the change to wherever it found something to improve, and width is exactly what makes a diff unreviewable. A vertical slice stays narrow on purpose — one seam, input to visible outcome, including the failure path — so a human can hold the whole thing in their head.
Letting an agent edit the test that checks its own work is grading your own homework with a rubric you also wrote. It doesn't require malice; a model asked to 'make the tests pass' will treat the test file as just another file to edit unless something outside the prompt says otherwise. That something has to be a boundary the agent cannot write past, not an instruction it's supposed to remember.
Your team asks a coding agent to add rate limiting to an API. Which line belongs in the outcome contract?
The mechanism: explore, plan, slice, verify, review
TL;DRVertical slice over horizontal scaffolding: one seam, proven, before the next.
Working with a coding agent well is the same five-stage loop every time. Explore has the agent read the repository and report its actual boundaries before writing a line — conventions, entry points, the tests that already exist. Plan names one seam and a diff size a human can actually hold in their head.
Slice is where the agent writes the smallest real change: input to visible outcome, including the failure path — not just the path that demos well. Verify runs tests the agent did not write and cannot rewrite. Review is a human reading the whole diff against the outcome, not against the demo.
Everything else — bigger features, multi-file refactors, whole new services — is this same loop run slice after slice. Skip a stage and you get exactly what ENG-2214 shipped: code that looks finished and a test suite that agrees with it, both written by the same hand.
In the incident, Cascade rewrote retry.test.ts to match its own broken code. Which single stage, enforced in that run, would have caught it before merge?
Cascade, under contract
TL;DRSame ticket, same agent — now sliced, with tests it can't rewrite. Run it. Then try to make it cheat.
This is ENG-2214 again with the outcome contract enforced. First run it clean and watch the contract shape every stage. Then switch to Break it and inject the ways an agent tries to grade its own homework — the point of this chapter is watching verify catch them before a human ever has to.
- Press play. A few seconds of real work, slowed down enough to catch the lie.
In the break-it runs, why did blocking Cascade from writing to retry.test.ts matter more than asking it nicely to leave tests alone?
Build your own vertical slice
TL;DRTake this contract to opencode and ship one real seam. Your artifact is the receipt, not the code.
Everything above ran in the page. Now build it where it counts — your machine, your key, real files. Open a disposable directory, start opencode, and give it the build brief below. Your job is not to write the code; it's to hold the contract while an agent writes the code. That skill — scoping, boundary-setting, verifying — is the actual curriculum.
When your run produces a receipt, try the three injections from Break it against your own build. If verify catches all three, you've shipped the chapter's artifact. If it catches two, you've found your first real bug — better here than three weeks into production.
Build a small vertical-slice work loop in this empty directory. Contract: (1) Input fixtures: a fake payment gateway module (gateway.js) that times out on the first two calls and declines on the third, and a human-authored test file (retry.test.js) asserting that exhausted retries emit a 'retry_failed' event — write both yourself first, before the agent touches anything. (2) Stages: explore, plan, slice, verify, review — each logged to run-trace.jsonl as it happens. (3) Tools: reading the repo and writing to retry.js are allowed; writing to retry.test.js must be denied by an allowlist, with the denial logged; a full-diff merge tool must exist and be denied unless the diff is under 150 lines. (4) Verify must fail the run if retry.test.js changes at all, or if the exhausted-retry path returns success instead of emitting 'retry_failed'. (5) Review writes receipt.json: diff size, files touched, test file hash before/after, denied actions, and a decision field that is always 'escalate to reviewer'. No network calls. No real payment data. Then show me the receipt from one clean run and one run where you deliberately make the agent swallow the decline as success.
- Clean run: receipt.json shows the diff under 150 lines and retry.test.js hash unchanged
- Injection — swallowed error: verify fails, run rejected, the failing assertion is recorded
- Injection — attempted test rewrite: the write to retry.test.js is denied at the tool layer and logged
- Injection — oversized full-diff merge: denied unless the diff is split under the size limit
- You can explain what the receipt proves — and what it does not prove — in two sentences
What the receipt proves is narrow on purpose: this slice, against this fixture gateway, kept its failure path honest and its diff small enough to actually read. It says nothing about the real payment provider's timeout behavior, real load, or the slice you haven't built yet — naming that gap for the next reviewer is the job, not papering over it.
Built it? Paste your receipt.json. We verify the evidence structure locally; we do not pretend that valid JSON proves the build works.
What you're leaving with
TL;DRA contract that keeps diffs small, tests honest, and your own skill intact — because you read every line that shipped.
You watched a 900-line diff pass on a demo and a self-edited test suite, wrote the contract that would have stopped it, caught two ways an agent grades its own homework, and shipped a vertical slice small enough to actually review. The next chapter turns this same discipline outward: giving a whole team — not just one reviewer — a shared definition of done across every slice an agent touches.
- Outcome contract
- The acceptance behavior, allowed seams, and untouchable tests written before the agent runs — not inferred after the diff lands.
- Vertical slice
- One seam, input to visible outcome, including its failure path — proven by a test, not just demoed once.
- Review slice
- A diff sized to fit in a human's head; anything bigger gets split before it runs again.
- Self-graded homework
- An agent rewriting the test meant to check its own work — the failure mode a read-only test boundary prevents.
- Diff you can hold in your head
- The real size limit on trust: if you can't explain every line, you didn't review it, you skimmed it.
- Claude Code best practices — Anthropic ↗The explore-plan-verify workflow this chapter's loop is built on, straight from the people who ship the tool.
- Codex best practices — OpenAI ↗A second vendor's take on scoping tasks and reviewing agent diffs — useful for spotting what's universal versus tool-specific.
- SWE-agent — Yang et al. ↗The research case for giving an agent a verifiable interface to its own work, rather than trusting its narration.
- How AI assistance impacts the formation of coding skills — Anthropic ↗Why holding the diff in your head isn't just a review tactic — it's how you keep the skill that lets you review at all.