The GitHub blog post that triggered this piece opens with a line I have not been able to get out of my head. The author says they have watched the same one-prompt demo three times in a month, and the room claps, and by Monday morning the magic does not hold up against a real PR review. That observation matches what I have seen on three different teams I have worked with this year, and it is the thing I want to push on here.
A one-prompt demo proves a model can generate code on demand. It does not prove a team can ship code under the same conditions a human reviewer would have to operate in. The two tests look identical from the front row of an all-hands and have almost nothing in common.
Why the demo feels like proof and is not
Demos are designed to flatter the model. The prompt is hand-tuned, the context is a fresh sandbox, the output is one self-contained file with no integration cost. Real shipping is the opposite. Real shipping has a ten-year-old Postgres schema with three deprecation warnings, a CI matrix that runs twenty-eight minutes, a CODEOWNERS file with eleven reviewers, and a Slack channel where someone will absolutely ask why you bumped that minor dependency. The source article makes the same point in different language, and it is worth restating because it is the part the room forgets between the demo and the deploy.
A one-prompt demo tells you the agent can write code. It does not tell you the agent can read your repo, respect your conventions, and survive the merge gauntlet. Until you wire the agent into the same plumbing your human contributors go through, you are not testing the thing you actually care about. You are testing the demo environment.
The trap I keep falling into is treating a clean demo output as evidence that a future workflow will work. It is not. It is evidence that the model handles the slice of code I handed it. Everything outside that slice, the imports, the tests, the conventions, the boundary checks, is still unproven. The source puts this as “demos prove the model can generate code, not that the team can ship code.” Same warning, applied differently.
The workflow I landed on, and why it works
After watching two of those demo-to-Monday collapses play out, I started building agent flows the way I build any other piece of production software. Pick a small trigger, scope the work tightly, capture the output in a pull request, and let the same CI gates a human contributor would face do the talking. No special lane, no exceptions, no trust shortcuts. The source article frames this as a “developer role shift toward wiring pieces together.” My version is the same shift, applied to my own repos.
Concretely, the shape looks like this:
- Pick a boring, contained workflow first. Dependency bumps, doc lint fixes, weekly changelog drafts. Anything where the blast radius is small and the reviewers are forgiving.
- Scope the agent to one job. A single PR, a single intent, a single commit. If the agent wants to do three things, it does three PRs.
- Capture output in a PR, not on main. Bots do not get direct-to-main privileges on any branch I care about.
- Run the same CI you run for humans. Lint, tests, security scan, the works. If those fail, the agent’s PR fails. No exceptions, no manual override.
- Review the first ten PRs by hand, every one. This is the part nobody wants to do. It is also the part that teaches you the failure modes the demo never showed you.
That last bullet is doing most of the work. The first ten PRs are where you find out the agent hallucinates a build flag in three files, or imports a library your repo banned last year, or wraps everything in try-except because it saw that pattern in the training data. None of that shows up in a one-prompt demo. All of it shows up the first week you let it touch a real branch. Treat those first ten reviews as your source of truth for what the model actually does in your codebase, not what the demo suggested it would.
Where the agent earns trust and where it does not
Once I had ten clean PRs in a row on a boring workflow, I started loosening the boundary. Bumped the auto-merge threshold for docs-only changes. Allowed the agent to label its own PRs once I had seen the label taxonomy it produced. Gave it read access to a wider slice of the repo so its context window stopped guessing about file paths. Each loosening was small and reversible, which is the part the GitHub post calls out as “the developer role has quietly shifted.” I would add: the shift only holds if the loosening is reversible.
What I did not loosen, and what I am not going to loosen any time soon:
- Merges into protected branches still need a human reviewer on risky files. The agent can open the PR. It cannot merge into main on a Friday.
- CI failures are still hard stops. If the agent argues the test is wrong, it can leave a comment. It cannot override the check.
- Audit logs stay on. Every prompt the agent ran, every tool it called, every diff it produced, gets recorded. The day something ships that should not, I want to be able to answer how.
- A human fallback exists for every automated workflow. If the agent goes down, the workflow still has to be doable by hand. Otherwise the team builds a dependency on magic.
The principle is simple. The agent is a contributor with a different risk profile, not a different set of rules. Same gates, same review bar, same audit trail. The model earns the right to skip steps the same way a human does, which is by demonstrating it does not need them. The source frames this as “you decide which checks run on that PR and which can block the merge.” My version is the same idea, phrased as earned trust rather than granted trust.
What I would tell the version of me from last month
Three things, in order of how much pain each one would have saved.
- Do not trust a clean one-prompt demo as a shipping signal. It is a model output, not a workflow proof. The two are different products, even when they look the same from across a conference room.
- Wire the agent into the same plumbing your humans use from day one. Special lanes become permanent lanes, and permanent lanes become permanent trust debt. Use the real CI, the real branch rules, the real review queue. Boring is the point.
- Pick the smallest, most boring workflow first and review every output yourself for ten runs. The failure modes are not exotic. They are boring. They only show up under load.
Trade-offs
This approach is not free. Reviewing the first ten PRs by hand on every workflow adds real time to the rollout. For a team running a single weekly maintenance bot, that is roughly ninety minutes per workflow, which is real time I would rather spend elsewhere. The payback is that by PR eleven we have a list of the agent’s actual failure modes, not the ones I was afraid of.
The audit log approach also has a cost. Capturing every prompt, tool call, and diff costs storage, and reviewing the logs after incidents costs attention. We accept that cost because the alternative is debugging an agent-shipped incident with no paper trail, which is worse.
The biggest tradeoff is cultural. Asking the team to review bot PRs by hand for ten runs is friction. Some of them will push back, and some of them are right to. Skip this step and you are back where we started, trusting a clean demo that has not been stress-tested.
If you only ship one of these ideas, ship the first one. Stop treating a one-prompt demo as a shipping signal. The rest of the workflow can wait a quarter. That one change is the thing that separates a developer who uses agents from a developer who gets used by them.