We used to argue about whether the agent could write the feature. That argument is over. On the programmes we run, agents now draft most of the diff. The new argument is quieter and more expensive: who pays for proving the diff is shippable.
Opening census
On one busy session we stopped guessing and measured. Browser acceptance alone burned almost two hours of wall clock across 39 Playwright runs. Fifteen of those runs (about 38% of the set, and about 45% of browser wall time) re-tested scenarios that were already green. Merging main had voided every branch record, so the suite paid again for work other PRs had already proved. Sign-in ate roughly 42% of each run. Gates and browser jobs shared two global locks on one laptop, so a heavy day produced load kills. Review and verify stages ate roughly 40% of workflow model output. Writing code was about 8%. Hundreds of main-thread commands were just polling a gate and re-reading a fat context each time.
That is the verification tax. Generation got cheap. The tax did not.
We are not publishing this census as a marketing scoreboard. It is a floor: one real session, one programme, one honest list of where the clock and the tokens went. The rest of this post is what we do about that floor without deleting the gates that protect users.
What the tax is made of
The tax is not one line item. It is a stack.
- CI and acceptance that re-run work nobody changed. Full suites on every agent PR. Browser records voided by unrelated
mainmovement. Flaky reds that agents cannot dismiss the way a human shrugs. - Human review that re-reads the same world five times. Five review lenses on a top-tier model, each burning a huge cached context, each asking for the whole requirements doc again, each polling while a gate runs.
- Session economics that look like “AI cost” but are really waiting cost. Cache reads on polling turns. Subagents inheriting maximum effort into stages that do not need it. Continuity gaps that replay finished work after a limit hit.
Industry writing about agentic coding often stops at (1): CI volume. Our measured session was (1) plus (2) plus (3) on a single programme. The fix is the same idea in every layer: stop treating verification as an infinite fixed cost of every generated line.
A useful way to say it to a client: the agent made the diff cheap. The factory still has to stamp the diff. If your stamp process re-reads, re-signs-in, and re-runs green work, your “AI bill” is mostly a process bill wearing a model label.
The industry signal: CI is the next bottleneck
Anthropic is seeing the same shape at a different scale. In Agentic coding is straining CI, Sachin Malhotra describes CI job volume rising about 25x in six months as engineers shipped more code, tests grew, and agents authored most of the work. Their test impact analysis service needed three patches that bought 70 days, then 29 days, then less than a day, before a redesign that kept state out of a singleton process. The lesson they take is blunt: plan for the exponential, because half-measures buy a fraction of the time they used to.
We cite that article as industry context, not as BlueLupin metrics. We do not need their multipliers to recognise the tax. We need a floor that cuts avoidable verification without deleting the gates that actually protect users.
Malhotra’s post matches what we tell clients after the first month of agentic coding. Writing stops being the constraint. Once review accelerates, CI feels the pressure. Agents also prefer smaller, more granular PRs, which raises job count even when each change is safer. Overnight and weekend pushes raise the floor. Humans still create bursts when they approve.
Their early test selector was a single process with a listener and a selector that had to stay in sync. Listener lag meant stale selection: widespread failures, flakes that should have been known, fixed tests that did not run yet. Bigger machines, then per-package sharding, then daily restarts, each bought less time than the last. The lasting move was an architecture that could scale horizontally: append-only journal, stateless listeners, a small consumer rolling history up, selectors that look up history quickly.
We steal three ideas from that story without pretending we run their stack:
- Assume 10x to 25x verification load within a couple of quarters once agents are normal, not “maybe later.”
- Instrument in equals out. Jobs arriving should match jobs recorded. Lag is a product bug, not a dashboard curiosity.
- Keep mutable history out of a lonely process. The same rule applies to our browser acceptance register and to any impact analysis we build: a singleton that holds the world in RAM will become the next pager.
What we do not steal: their exact multipliers as our results, or a claim that we already run a horizontally scaled selector in every client CI. We cite their post as the public proof that the tax is industry-wide.
Our solution, measured first
We did not start with new servers. We started with a census of one real session, then ranked levers by measured saving. Hardware sits below rules, because the largest wastes were habits: voiding good browser records, signing in over and over, polling gates from the main thread, and paying top-tier rates for every review lens.
Two lanes.
Lane A cuts browser and gate wall time.
Lane B cuts token burn and session thrash in review and orchestration.
Quality gates stay. Security and correctness still get a serious model. A full gate and browser acceptance still exist. We are not deleting the toll booth. We are stopping people from paying the toll twice for the same road.
Lane A: stop paying the browser toll twice
A1. Void a record only when this branch changed something user-facing
The expensive bug was subtle. A browser-acceptance record was treated as stale whenever HEAD moved relative to the tested SHA on any user-facing file. Merging main dragged in files other PRs already proved. Fifteen of thirty-nine runs in the census were green-on-green reruns. That was about 45% of browser wall time.
The rule we want is an intersection: void only when *this branch* changed a user-facing file since the record. Formally, user-facing files in diff(testedSHA, HEAD) that also appear in diff(merge-base(origin/main, HEAD), HEAD). Files only main moved were covered by their own PRs’ records. A file changed by both still voids the record, because that merged state has not been opened.
We also had to fix supersession. With the tighter drift rule, a branch can legitimately hold two undrifted ancestor records. Selection must prefer the newer ancestor, not throw up its hands as ambiguous.
No record format change. Existing records keep working. A stale origin/main only widens the intersection (more reruns, never fewer), which errs safe.
This is our local cousin of Anthropic’s test impact idea: run what the change actually touches, and keep a register that means something. If you only remember one Lane A item, remember this one. Everything else in the lane is smaller than stopping green-on-green.
A2. Fail fast before the long gate
Put the browser-acceptance check first in the full gate. A missing record should fail in seconds, not after a long static gate has already burned ten minutes of laptop and attention. Memoise drift calculation so we do not spawn git hundreds of times per selection.
Fail-fast sounds obvious until you watch an agent wait for a full gate, then discover the acceptance register was empty. The wall clock cost is bad. The session cost is worse: every waiting turn re-reads context, and every compaction throws away working memory you will need when the gate finally returns red for a boring reason.
A3. Sign in once per persona per run
Sign-in was about 42% of each browser run. Snapshot session storage per persona after settle, restore it for later specs in the same run, wipe auth files when the run releases the lock. Specs that test the form itself still force a fresh sign-in. Arithmetic on the census: roughly half the sign-in time back when a handful of personas share a run.
We are careful not to persist auth across runs. Tokens expire. A run is short enough that an in-run session does not need a refresh dance. Specs that exist to prove the sign-in form keep the fresh path. Everything else stops paying the login tax for the privilege of checking a page that was already authenticated five minutes ago.
A4. Parallel reads, exclusive writes, measure the wait
Split the browser suite into a read project and a write project. Reads take a shared lock and can overlap. Writes take an exclusive lock against the shared demo tenant. Append waited= to the verdict line so we know whether a tenant pool is justified later. Do not invent cross-machine write parallelism against one shared tenant with a filesystem lock.
This is where practitioner caution matters. A “read-only” label is not enough if a write spec renames terminology another read fingerprints, or creates rows another read asserts about. Shared versus exclusive locking is the honest answer. Parallel reads with overlapping writes is how you buy flakes and call them speed.
A5. Move saturating CI off the laptop when the numbers say so
Self-hosted runners revive sharded CI and take iterative gates off a laptop that was also running the browser. Until a deliberate policy change, the local final full gate before merge can stay. Servers first host information and iterative gates. Binding a remote result as the merge gate needs an explicit revision, a complete non-cancelled run, and a SHA check. That is a separate decision, not a runs-on one-liner.
Anthropic’s article is partly a warning about underestimating load. Our warning is the inverse: do not buy boxes before you stop green-on-green reruns. Moving waste to a nicer machine is still waste. When the census shows load kills and a dead CI queue, runners earn their place. When the census shows avoidable reruns, fix the register first.
A6. Tenant slots only when waited= proves the queue
A second tenant or persona slot is real engineering: fixtures, seed data, env wiring, and a clear rule about which runner owns which slot. Build it only after write-run waiting shows up in the verdict lines. Otherwise you buy complexity for a queue you have not measured.
Lane A ends with instrumentation on purpose. waited= is how A6 stays honest. Without it, someone will propose a pool because it sounds scalable. With it, you either see a queue or you do not.
Lane B: stop paying a top model to watch the kettle
B1. Review shape before model theatre
One brief agent builds a complete change manifest: tracked hunks, untracked new files (migrations and tests the builder has not staged yet), story text sliced from the requirements by id, a UI touch flag. That brief is the identical cached prefix for every reviewer. Lens text goes at the end. When the full diff is too large, reviewers get a summary and must read listed paths, not invent coverage.
Drop lenses that the gate already ratchets. Fold “minimal diff” into correctness. Run accessibility only when UI was touched. Cap rounds at two. Round two is confirm-only for lenses that had critical or high findings. Incomplete rounds are incomplete, not “no findings.”
On the profiled session, review shape alone was the largest soft saving before any model change. That surprised people who wanted to jump straight to cheaper models. Shape first. Tier second. If five lenses each re-read a half-megabyte brief, the model name is not your main problem.
B2. Tier the models on purpose
| Stage | Model | Why |
|---|---|---|
| Build, fix | Top-tier, high effort | The change still has to be right |
| Review: security, correctness | Top-tier, high | These are the lenses we refuse to soften |
| Review: criteria, accessibility | Mid-tier, medium | Still serious, cheaper, judged when needed |
| Brief | Small / fast | Manifest assembly, not judgment |
| Verify / detail | Mid-tier, low or medium | Check, do not re-architect |
| Judge | Top-tier, medium | Only on mid-tier findings; confirmed / refuted / unresolved |
Refuted requires evidence. Unresolved stays serious. A null judge is unresolved, not a silent drop.
We publish the table as intent, not as a forever brand-name contract. Models move. The doctrine does not: build and security stay expensive; assembly and verify do not; mid-tier findings get a serious second look before they vanish.
B3. Effort is per stage, not inherited superstition
Do not let a session-wide maximum effort infect every verifier. Set effort on each agent call. Inheritance feels convenient until you notice a detail-check agent burning the same effort budget as the builder that just wrote the migration.
B4. One monitor, one verdict line
No main-thread polling. One monitor owns the full gate or the browser run and returns a single line: exit= seconds= head= …. On failure, a short grep or a cheap model extracts the useful tail. Bound the waits so a stuck load loop cannot become a second session.
In the census, hundreds of main-thread commands were gate polling. Each one re-read a large context. That is verification tax dressed as diligence. Diligence is owning the gate once and reading the verdict. Diligence is not asking “is it done yet” two hundred times from a thread that carries the whole programme in its prompt.
B5. Main-thread hygiene
Clear at batch boundaries after the handoff brief exists. Trim MCP connectors the programme does not use before the first turn. Their instruction blocks ride every subagent request. Keep the root instruction file short; put workflow rules in a doc the agent opens when it needs them.
This is unglamorous work. It does not demo well. It is also where a surprising amount of cache traffic hides: connectors nobody opened, still present in every request, still paid for on every verifier turn.
B6. Continuity without account games
Auto-continue at usage limits, usage credits with a cap, resume from the workflow journal with eyes open: a failed agent replays, and so does everything that started after it. Build and fix prompts must be safe to re-run on a dirty worktree. Extra seats go to extra people with their own worktrees, not to one person juggling logins. We do not rotate consumer accounts. That is a policy line, not a tip.
Continuity is not “make the session immortal.” Continuity is “when the session pauses, resume without inventing a second source of truth.” Journals help. Safe re-run prompts help. Pretending one human can hold three logins and call it parallelism does not help, and we will not teach it.
B7. Measure weekly or the savings are folklore
A small journal script plus usage attribution: review share of output, top-tier share, polling command count, builder cache rewrite ratio. Targets are public to the team. If builders rewrite context too often, that is when you consider a longer subagent cache TTL, not before.
Lane B without B7 is a story you tell yourself. Lane B with B7 is a programme you can run for a quarter and still believe.
What we refuse to cut
A cheaper verification path that drops security review is not a win. Neither is a CI that is green because the selector is a week behind. Anthropic’s lag story is a warning: stale selection creates false confidence and false reds at the same time.
So the hard gates stay:
- A serious model on security and correctness until measurement says otherwise.
- Browser acceptance for user-facing flows, with a register that tells the truth.
- A final human on the merge. The person who opened the agent session does not approve their own PR.
- Deny logs and confirm steps for irreversible tools, same doctrine as our earlier posts on tool access.
The tax we cut is the tax on *repeating proof*. The tax we keep is the tax on *first proof that matters*.
We say this out loud because “cut the verification tax” can be heard as “ship faster by trusting the agent.” That is not the programme. The programme is: prove once, prove what changed, prove with a model that matches the risk, and stop paying for theatre.
How this maps to a client programme
When a client asks how to “make agents cheaper,” we do not open with a bigger subscription. We open a census.
- How much wall time is acceptance and CI last week.
- How much of that is reruns of already-green work.
- How much model spend is review versus build.
- How many main-thread turns are waiting on a lock.
Then we apply the order we use on our own floor:
Week 1, no hardware: fix drift rules, fail fast, cache sign-in, parallelise safe reads, kill polling, reshape review with one shared brief.
Then: tier models, measure a week, decide if runners are justified by waited= and laptop load.
Later: tenant pools, binding remote gates as merge authority, longer cache TTLs only when rewrite ratios demand them.
That order matters. Buying boxes before stopping green-on-green reruns just moves the waste to a nicer machine. Anthropic learned a related lesson the hard way: patches on a singleton bought days, not quarters. Architecture and rules beat ritual.
On a client programme the same census questions translate cleanly. You do not need our internal scripts. You need verdict lines that record seconds and waits, a review journal that attributes tokens by stage, and a willingness to change register rules before you change invoices. The conversation stops being “which model is cheapest” and becomes “which proofs are we repeating.”
We also map ownership. One person owns a workstream end to end in their own worktree: build, review rounds, gate, browser run, PR. Shared resources stay the ones Lane A serialises until runners and slots exist. Parallel people scale throughput. Parallel logins for one person do not.
The bill you should watch
If you only watch “tokens this month,” you will optimise the wrong surface. Watch these instead:
| Signal | Healthy direction |
|---|---|
| Acceptance reruns of already-green scenarios | Down, sharply |
| Time-to-fail on a missing acceptance record | Seconds, not minutes |
| Sign-in share of browser wall time | Down without losing form coverage |
waited= on write runs | Near zero, or justified by a slot plan |
| Review + re-review share of subagent output | Down toward ~20% after reshape |
| Top-tier share of subagent output | Down, with security/correctness still on a serious model |
| Main-thread gate polling commands | Zero |
| CI listener lag / selector freshness (if you have impact analysis) | Flat, not sawtoothing every afternoon |
Generation will keep getting cheaper. The teams that win will be the ones who treat verification as a product: impact-aware, instrumented, hostile to avoidable reruns, and honest about which proofs still deserve an expensive model.
The agent writes the diff. The factory still has to stamp it. The stamp is the bill. We intend to stamp once.