Every PR gets an adversary

Every pull request in every one of SplitKit's repositories is reviewed by an adversarial agent before merge. The review fires automatically when the PR opens, the findings post to the pull request itself, and a ledger records every run. That ledger currently holds 189 review runs across 108 unique pull requests over forty-six days, at roughly a minute per review. Thirty-five of those pull requests were re-reviewed automatically when new commits landed, because new code owes a new review whether or not anyone remembers to ask for one.

SplitKit is a race-day experience platform built by one person, no team, with agents doing most of the typing; the sequence that produced it is the subject of "From idea to production with AI in the loop". This entry is about the most load-bearing gate in that sequence. When agents produce most of the diff volume, the person who prompted a change into existence is the wrong last line of defence against it, for the same reason authors make poor proofreaders. So the second set of eyes is also an agent, but a different one, with an adversarial charter and no ability to touch the code.

1.0The gate, as built

Two paths trigger a review.¹ The commit tooling that opens a pull request fires one in the background within seconds. A scheduled daemon sweeps every ten minutes as the backstop, picking up any open pull request whose latest commit has no review on record. Between them, nothing merges unseen, and neither path depends on a human remembering anything.

Each run checks the pull request out into a detached scratch worktree, deliberately outside the workspace, so a review can never disturb a live working session. The reviewer itself is a headless invocation of the same agent tooling that writes the code, pointed at a mid-tier model, with a tool allowlist that permits reading, searching, and posting comments, and nothing else. Its charter is explicitly advisory: tag findings by severity (security and data loss at the top, crashes and correctness next, minor issues last), post them to the pull request, and stop. It never approves, never requests changes, never writes code, never pushes. Authority stays with the committer; the reviewer's only power is to be right in public.

The ledger is the contract. Every run is keyed on the repository, the pull request, and the head commit hash, which is what makes re-review automatic: push a new commit and the ledger no longer has an entry for your code, so the daemon owes you a fresh adversarial read.² My obligation as committer is the mirror image: read the review before merging, and either fix or explicitly disposition every high-severity finding. Both halves of the loop are enforced by tooling rather than by intention.

2.0Economics that fail closed

The whole pipeline runs locally, on subscription credit, with zero metered API spend. That was a deliberate choice against the obvious architecture, and the reasoning transfers.

Hosted CI was the default shape: a workflow per pull request, billed per token against an API key. But per-token billing gives every review a marginal cost, and marginal costs attract optimisation: the moment skipping a review saves money, the gate has a price, and priced gates get negotiated with. Reviews on subscription credit cost nothing at the margin; running one more is never a spending decision.

The subtle half of the choice is what happens when the credit runs out. In headless mode, an API key present in the environment silently takes precedence over subscription auth, which means exhausted credit would not stop the reviews; it would quietly move them onto metered billing, and the first sign would be an invoice. So the invoking script strips the API-key variables from the environment before launching the reviewer. Exhausted credit now fails closed: reviews stop, visibly, and the remedy is a plan decision rather than a surprise bill.

3.0Twenty-one rounds on one pull request

On July 1 I opened what was estimated as a fifteen-to-thirty-minute quick look at a small fix in the code path that gates a purchase. By the time it merged, it was a diff of about 2,300 lines that had absorbed twenty-one adversarial review rounds and roughly sixteen hours of work. The ledger shows twenty-three runs against that one pull request, the most of any PR before or since.

The growth had one cause: while fixing the named bug, the work surfaced three more findings that were each a pull request in their own right, and all three rode the same branch. Every commit addressing one re-opened adversarial review of everything else, which is how a quick look compounds into sixteen hours.

The saga also stress-tested the pipeline harder than anything before it, and broke it three ways. Two reviews of the same pull request at different commits ran concurrently and clobbered each other's scratch worktrees; the scratch paths now carry the commit hash. The safety net that pastes a review's raw output onto the PR when a run fails to post was comparing against a comment count of zero, so on a PR that already had comments a failed post was silently discarded; a thirty-four minute review vanished before the net learned to snapshot the count before each run. And a default time ceiling on background tasks killed one long review mid-run; review runs no longer have one. Each bug was fixed the moment it appeared, mid-saga, because working around your own gate, just this once, is how a gate becomes decorative.

4.0The doctrine that came out

The next day the lessons were ratified as written doctrine, in the rules file every future agent session loads. Four rules survived contact.

Discoveries get their own pull request. A finding with its own root cause, or one that predates the branch, or one that touches files the original fix never needed, goes to a new PR. A fix PR reviews its fix. This is the rule that would have kept the saga at thirty minutes.

Money paths get a scope decision before implementation. Any change where money moves now begins with an explicit, human-answered question: are we fixing the named edge, or excavating the whole family it belongs to? Either answer can be right. Deciding after the diff exists is what was wrong.

The exit condition is the severity trend, never zero findings. An adversarial reviewer pointed at a nontrivial diff will always find something; that is what adversarial means. The loop ends when consecutive rounds stop producing high-severity findings in pre-existing code, and the residual minor findings get written dispositions and a follow-up PR.

Evidence may refute the reviewer. Every finding in every round ended with a written disposition: fixed, refuted with evidence, or acknowledged. The refutations earn their keep, and an evidence-backed decline, posted on the pull request, is a valid disposition. "Fix everything" and "accept everything" are different disciplines.

One more habit generalised from the same stretch: the adversarial stance works on your own conclusions too. A sceptic pass over an audit checklist already signed off as done turned up a real gap; the highest-value findings live behind your own green ticks.

5.0An instrument cluster for the pipeline

The operational pain that emerged was not review quality; it was blindness between states. I would open a pull request and wait, with no way to see whether the review was queued, running, posted, or lost. When agents do the work, the manager's window is instrumentation of the process, so the pipeline got an instrument cluster: a local dashboard called Workshop, on my machine only, strictly read-only. It shells out to the same git and hosting tools I use by hand and reads the artefacts the pipeline already produces.

Its centrepiece addresses exactly that blindness: a state machine for every open pull request (queued, reviewing, posted, merged, plus attention states for a PR whose new commits owe a re-review and one whose findings await my disposition), derived entirely from real artefacts: the hosting platform's API, the review ledger, lock files, the daemon's scheduler status. Nothing on the screen is self-reported. Around it sit a builds view per surface; agent telemetry aggregated strictly to numbers, because transcripts contain full prompts and are never rendered anywhere;³ and a force-directed graph of the project's knowledge base, about 288 nodes.

The restraint rules matter more than the features, because a dashboard is where good intentions go to sprawl. Glanceable beats analytical: the default view answers "is anything wrong?" in one look. Positive state is surfaced, never hidden: a passing check renders as a green row rather than an absence, because an empty section is indistinguishable from a broken collector. There is one attention colour and one alert colour, and no glow. And there are no business metrics on it at all, by design. It is an engineering instrument, and the moment revenue appears on a screen, people start performing to the screen.

6.0If you're doing this yourself

The recipe, as rules and gate questions:

1. Automate the trigger, mandate the read. The review fires on PR creation, a polling daemon is the backstop, and findings post where the merge happens. Rule: no merge until the committer has read the review and dispositioned every high-severity finding in writing.
2. Key reviews on the commit, not the pull request. A re-push is new code. Rule: new commits owe a new review, enforced by the ledger rather than by anyone's memory.
3. Pick economics that make skipping irrational, and fail closed on billing. Gate: what does one more review cost, and what happens the day the credit runs out? Fix any answer other than "nothing" and "reviews stop loudly".
4. Give the reviewer an adversarial charter and no hands. Read, search, post findings by severity, nothing else. Rule: the reviewer never approves, never blocks, and never writes code; authority stays with the human who must disposition its findings.
5. Exit on the severity trend. Rule: merge when consecutive rounds stop producing high-severity findings in pre-existing code, with residual minor findings dispositioned in writing and queued for a follow-up PR. Zero findings is a treadmill, not a standard.
6. Let evidence disagree. Rule: a pinned regression test posted on the pull request is a valid basis for declining a finding, and every finding ends its round as fixed, refuted with evidence, or acknowledged.
7. Split discoveries, and pre-decide money-path scope. Gate: does this new finding have its own root cause or predate the branch? Then it gets its own PR. And before touching a path where money moves: named edge or whole family, decided before the diff exists.
8. Instrument the pipeline itself. Rule: derive every state from a real artefact, aggregate agent telemetry to numbers only, surface positive state, and allow one attention colour. If reading it takes more than a glance, prune it.

What the ledger shows is a merge gate that ran every single time, cost nothing at the margin, and got argued with only on evidence. It fits inside the spare capacity of a subscription.

Entry 04 · Written after the gate's first 108 pull requests