Simon Willison posted a short note on 22 August 2026 that, in three sentences, named the skill everyone who uses a coding agent is supposed to be learning. The skill is twofold. You need to confidently instruct the agent on how to make a change, and then you need to confidently verify that the change was actually applied the way you asked for it. The word that gets missed is the second half. The eyeball-every-line ritual that has been the default in many shops for two years is not verification. It is just a different shape of guessing.
There is a real reason the second half gets skipped. Most code review tools give you two affordances: a diff (the textual difference) and a green checkbox. The diff is the only artifact that fully describes what changed, but reading it takes time and attention. The checkbox is fast. People optimise for what is fast. The result is that a large share of the changes that land in production have been signed off by someone who only skimmed the patch and relied on a testing pipeline to catch what their eyes missed. The pipeline catches the syntax errors. It does not catch the one-character constant change that swaps a comparison order, or the new database query that runs fine on the development dataset but has a different access plan on the 40-million-row production table.
What verification actually looks like
Verification, in the sense Willison is using the word, is the act of confirming that the agent’s diff is the same change the problem statement asked for. There are four flavours, in roughly increasing order of effort.
- Branch policy enforcement is the cheapest. If the agent only edits a file inside a directory you have scoped in advance, you do not need to read the parts of the diff outside that directory. Most pull request bots and pre-commit hooks only enforce this for security-relevant paths, but the same pattern works for any directory that has a single owner. A change to
services/billing/that does not touch the schema migration directory is a meaningful signal by itself. - Behavioural verification sits in the middle. Run the agent’s diff against a test that exercises the changed code, and check that the test outcome matches what the change claims to do. This is not the same thing as running the existing test suite. The existing suite was built to catch regressions in human-written code. It does not know what the agent’s change is supposed to do and has no specific assertion for the agent’s fix. You need a targeted check, written after the change, that would fail if the agent had misinterpreted the request. You can ask the agent to write the test as part of the same pull request. That last step is the discipline.
- Line-by-line review is the most expensive, and most of the time it is the wrong tool. A line-by-line review only catches errors that show up as wrong text in the diff. It misses wrong text that was inside the original code, because the agent may have left an existing bug alone and you will not notice that it is still there. It misses errors that show up only at runtime (the wrong constant in a configuration file). It misses errors that show up only in the data path (the agent updated the controller but not the view, and the view is now stale). A reviewer reading the patch top to bottom will not see any of these because the diff looks fine.
- Tracing logs and audit hooks form a fourth layer that pairs naturally with the other three. Wrap any state-changing agent call with a one-line log that captures the request, the parameters the agent received, and the post-change state of the database row or file. The log becomes the artifact you read during review in place of the diff, and it is the artifact that survives the deployment pipeline run. Most agent frameworks expose this through an explicit hook on every tool call. Use it.
Why the eyeball ritual stays popular
There is a social reason code review defaulted to reading the diff. Before coding agents, the diff was the only thing that existed. The agent did not run before review; the human wrote the change and asked another human to sign off on it. Reading what the human wrote was a reasonable approximation for “does this change do what was asked”, because the change had no other artifact. The reviewer was the integration test.
That approximation broke when a different class of writer started producing the diffs. The model is fluent and gets the syntax right. The diff looks like a human’s diff. The reviewer reads it and, because the syntax is fine and the structure is fine, signs off. Nothing in the review process actually checked the change against the request. So errors that would have been obvious in the request (“please add a guard against negative quantities”) show up in production, where the runtime test would have caught them, but the runtime test was not asked for in the request, so it does not exist.
What to do about it
Stop framing review as the verification mechanism. Review is a check, not the check. The check is the test that runs against the post-change code and exercises the thing the request specified. Build the test as part of the same agent turn whenever the request is anything beyond a typo fix. Read the test, not the diff. The diff becomes supporting evidence for the test outcome.
Next, put a wall around what the agent is allowed to touch for any request that involves a database, a network call, or a stateful system. The wall can be as soft as a comment on the pull request (“this change should not touch migrations/, infra/, or services/billing/billing_model.py“) or as hard as a server-side branch policy that refuses to merge any agent-produced diff that touches a blocked path. Either works. The point is that the reviewer only needs to do a full read of the parts of the diff that fall inside the wall, and not the parts that are guaranteed to be unchanged.
Then, instrument the agent’s output so the reviewer can see what the agent thought the request meant, not just the code it produced. The skill Willison is describing is not “watch the agent carefully”. The skill is “watch yourself carefully when you instruct the agent, and watch the artifact carefully when you inspect it”. The note from the agent about which parts of the request it understood and which parts it had to guess at is the cheapest and most informative artifact you can ask for. Many agents will produce a short summary of their interpretation before they start editing. The summary is easier to read than the diff and catches most of the interpretation errors before they turn into merge conflicts.
Trade-offs
None of these things are free. Adding a targeted test for every agent change adds an asset your team has to maintain and a failure mode your CI (continuous integration, the automated build-and-test pipeline) has to handle. A branch policy blocks legitimate changes when the agent and the policy disagree about where a piece of code belongs. An interpretation summary assumes the agent will produce one, and many agent configurations do not. Each of these costs you time you would otherwise have spent reading the diff.
The honest answer is that the cost is usually worth it on agent-produced changes, and not on human-produced ones. A human’s diff was the change; reading it was the only verification path. The agent’s diff is a candidate for the change; verification is the act of asking which candidate it is. The cost of the test is roughly the cost of a code review, and the cost of a missed-agent-error is the cost of a rollback plus the next two days of debugging. The trade-off is asymmetric in favour of the test.
Bottom line
The skill Willison is pointing at is verification. Verification is not the same as code review. Code review reads what was changed. Verification confirms what was changed against the request. The two overlap for human-written changes. They diverge sharply for agent-written changes, because the agent’s changes look right more often than they are right. Build the test, scope the diff, and read the agent’s interpretation before you read the diff. That is the workflow that scales.