Operator SOPs and Automation: Make Repetition Observable
Week 6 of the automated Friday billing run. Finance flags a support ticket: a legacy customer was charged full seat price for the third month running. Nobody has looked at a Friday invoice since the script started sending them itself.
Turn a task you do from memory into a written SOP, then an instrumented run, then an automation that leaves a receipt a human can actually audit.
What you will be able to do
- Convert tacit work into an executable SOP.
- Define inputs, checkpoints, exceptions, ownership, and receipts.
- Automate only after a human-run procedure is observable and repeatable.
- Prerequisites
- One recurring operational workflow to analyse
- Assumed knowledge
- No programming required for the reasoning exercises.
- Evidence you will produce
- An instrumented SOP with retry, exception, and audit evidence
- Workload
- 40 min guided lesson + 45–90 min independent build
The incident
TL;DRThe script copied Priya's motions, not her judgment — and once she stopped watching, nobody noticed the difference.
For two years, Priya ran Friday billing by hand from billing-roster.csv: seat counts, tier codes, a mental note that rows tagged 'L' were legacy customers grandfathered onto a flat rate years ago. She never wrote that down. Everyone just knew.
Six weeks ago the team shipped weekly_biller.py, built by watching Priya work twice and asking her to describe the steps. It reproduced her clicks. It did not reproduce her knowledge of what 'L' meant.
- Press play. A few seconds of real work, slowed down enough to catch the lie.
The script read billing-roster.csv correctly and applied its pricing logic exactly as built. So where did this run actually go wrong?
The diagnosis: automation without an SOP
TL;DRYou can't automate a procedure that was never written down — you can only automate the parts of it someone happened to mention.
weekly_biller.py wasn't a bad implementation of Priya's process. It was a faithful implementation of a *transcript* of Priya's process, and transcripts drop the exceptions nobody thinks to say out loud because they're too obvious to the person who's been handling them for two years.
An SOP is the artifact that forces those exceptions into the open before code gets written. It names the inputs, the steps, the known exceptions, and what 'done' looks like — as a document a second person can read and a machine can be checked against.
Here's the SOP that should have existed before anyone touched a keyboard:
- Inputs
- billing-roster.csv, current week's date range. Tier column values: 'S' (standard), 'L' (legacy flat-rate, contracted price in legacy-rates.csv).
- Steps
- Load roster → split by Tier → price 'S' rows per-seat, price 'L' rows from legacy-rates.csv → draft invoice batch.
- Exceptions
- Any row with a Tier value other than 'S' or 'L', or an 'L' row missing from legacy-rates.csv, is escalated — never priced by guess.
- Evidence
- Every run writes a log of what was read, priced, and flagged before anything is sent.
Go deeper
Writing the SOP first is not bureaucracy for its own sake — it's the only place the hidden exception gets to surface before it costs money. Priya could have named the Tier column in thirty seconds if anyone had asked her to write the procedure down instead of just watching her run it.
An SOP also outlives the person who wrote it. Priya can go on leave, change roles, or leave the company, and the exception still exists in a document instead of in a head. That's the actual argument for automation: not that it's faster than a person, but that it's the only way to make a procedure durable enough to survive nobody watching it.
Your team wants to automate weekly customer onboarding, currently run from memory by one ops person. What should happen before any code is written?
The ladder: repetition to receipts
TL;DRRepetition becomes a procedure, a procedure becomes instrumented, and only an instrumented procedure is safe to automate — because only it leaves something to check.
Every automation candidate climbs the same four rungs, and skipping one is exactly how weekly_biller.py happened. Repetition is just a person noticing they've done the same task three times. A procedure is that noticing written down as steps, inputs, and named exceptions — an SOP anyone can read and run.
An instrumented procedure is the same SOP, still run by a person, but now logging each step as it happens: what was read, what decision got made, what got escalated. This is the rung everyone skips because it feels like busywork. It's actually the rung that turns the SOP into a spec an automation can be graded against.
Automation with receipts is the last rung, not the first. It's the instrumented procedure with a machine doing the steps instead of a person — same log format, same escalation rule, same evidence a reviewer can open afterward. If a step was never observable when a human did it, it doesn't get to be silent just because a script does it now.
weekly_biller.py went straight from 'Priya does this from memory' to 'a script does this unattended.' Which rung did the team skip, and why did it matter?
The Friday biller, under an SOP
TL;DRSame roster, same task — now every step is logged and every exception escalates instead of getting guessed at.
This is weekly_biller.py rebuilt against the SOP: Tier is a named input, legacy rows price from legacy-rates.csv, anything unrecognized escalates instead of defaulting. Run it clean first, then try to make it fail silently the way the original did.
- Press play. A few seconds of real work, slowed down enough to catch the lie.
In the retry-storm break, what actually stopped the duplicate invoices — the SOP's pricing logic, or something else?
Build your own instrumented SOP
TL;DRTake this SOP to opencode and build the logged, escalation-safe version for real. Your artifact is the receipt, not the script.
Everything above ran in the page. Now build a small version of it where it counts — your machine, real files, an SOP you write yourself first. Pick any recurring task you can fake with fixtures: a weekly report, an onboarding checklist, a billing run. Write the SOP before you write a line of automation code.
When your run produces a receipt, try the three injections from Break it against your own build: an unrecognized case, a duplicate trigger, and an attempt to skip logging. If your escalation and idempotency checks catch all three, you've shipped the chapter's artifact.
Build a small instrumented SOP runner in this empty directory. Contract: (1) Write sop.md first — a real SOP for a recurring task you invent (weekly report, onboarding, or billing), naming trigger, inputs, numbered steps, at least one named exception case, and what counts as a completed run. (2) Create an input fixture file with a column or field that maps to your named exception (e.g. a status or tier value) — include at least one row that hits the exception and is NOT documented anywhere except your SOP's exception clause. (3) Build a runner that executes the SOP steps and logs each one to run-trace.jsonl as it happens using a log_step-style call. (4) Any row or case matching the exception must call an escalate_exception-style step and never be guessed at or silently defaulted. (5) Give each run a unique run identity; a second invocation for the same period must be detected and rejected before it repeats any write. (6) Build a run_silently or skip_exception_review tool, wire it into the tool layer, and make sure it is denied — the denial itself must be logged. (7) Handoff writes receipt.json: run id, artifact path, checks run with pass/fail, unresolved/escalated items, denied actions, and a decision field. No network calls, no real data. Then show me the receipt from one clean run, one run that hits the undocumented exception, and one run that is a rejected duplicate.
- sop.md exists and names trigger, inputs, steps, at least one exception, and what counts as done — written before the runner code
- Clean run: receipt.json shows all checks passing and zero unresolved rows
- Exception run: the undocumented case is escalated, not guessed, and appears in run-trace.jsonl and receipt.json
- Duplicate run: the second invocation for the same period is rejected before any write, and the rejection is logged
- You can explain what the receipt proves — and what it does not prove — in two sentences
What the receipt does not prove matters as much as what it does: it proves this run followed this SOP on these fixtures, escalating what it didn't recognize instead of guessing. It says nothing about next month's data, a Tier value nobody has invented yet, or whether the SOP itself still matches how the business actually works — that's a review a human still has to do, on a schedule, not something the receipt can certify for you.
Built it? Paste your receipt.json. We verify the evidence structure locally; we do not pretend that valid JSON proves the build works.
What you're leaving with
TL;DRAn SOP you write before the code, a logging discipline that makes exceptions visible, and a receipt format that turns 'it ran' into 'here's what it proved.'
You watched an automation quietly overbill customers for six weeks because a tribal-knowledge exception never made it into writing, wrote the SOP that would have named it, and built a version that escalates instead of guessing and rejects duplicate runs instead of doubling work. The next chapter turns this same discipline on the automation's own retries and timeouts — what happens when the SOP is right but the infrastructure underneath it isn't.
- SOP
- A written procedure naming inputs, steps, known exceptions, and what counts as a completed run — reviewable before it's automated.
- Instrumented procedure
- An SOP run by a human with every step logged, producing the evidence an automation is later checked against.
- Escalation
- The rule that unrecognized or out-of-bounds cases get handed to a person instead of priced, sent, or guessed at.
- Idempotent run identity
- A unique ID per run that lets a system recognize and reject a duplicate trigger before it repeats a write.
- Receipt
- The durable evidence packet: run id, what was checked, what was escalated, what was denied, and the next decision owner.
- Building effective agents — Anthropic ↗The case for starting with simple, observable, composable steps before reaching for full automation — the same order this chapter argues for.
- OpenTelemetry Semantic Conventions — CNCF ↗A vocabulary for what a 'step' in your log should actually record, if you want two operators to read it the same way.
- Production Services Best Practices — Google SRE Book ↗Where the retry-storm and idempotency ideas in this chapter come from, at the scale of a whole production service.
- How AI Is Transforming Work at Anthropic — Anthropic ↗A real look at which recurring work gets handed to automation first, and what stays a human checkpoint.