Cheap AI coding is easy. Cheap AI coding that still ships correct code is the actual problem. The teams I have watched waste the most money on AI tooling are the ones who optimized a single lever (usually “use a smaller model”) and called the job done. The teams that actually moved the cost-per-resolved-task needle did three or four things at once, and almost none of those things were a model swap. This is what the playbook looks like once you strip out the marketing.
I am writing this from the perspective of someone running AI coding tools on a real codebase, not from a vendor blog. The patterns below are not specific to any one product. They are strategies that show up in any serious cost-reduction write-up for AI coding assistants, including the engineering posts GitHub has published about its own Copilot infrastructure. Where I am inferring general practice rather than citing a specific source claim, I will label it.
Stop measuring requests, measure resolved tasks
The first mistake is the obvious one. You look at your AI coding bill and see “10 million tokens spent,” and you start optimizing tokens. Token cost is the wrong denominator for this work. Two requests can cost very different amounts, and they might or might not produce something the developer accepts and keeps.
The unit of value is the resolved task, meaning one useful edit (or test, or explanation) the developer actually accepts. Anything that produces an output the human throws away is pure waste, regardless of whether it was cheap to generate. Cost-per-resolved-task is what you want sliding down, not cost-per-request.
Once you have a per-task metric, a bunch of optimization work that seemed urgent stops being urgent. Specific line-completion suggestions that get auto-rejected by the developer are not saving you money even at zero token cost. They are just latency.
Pick the smallest model that clears the bar
This is the lever everyone reaches for first, and the one most often over-rotated. The temptation is to standardize on the cheapest possible model and hope for the best. In practice, the cheapest model that fails your acceptance bar is not a savings, because the developer has to redo the work and that cost is much higher than the token cost.
The practice that has held up across every team I have watched is tiered routing. For routine completions (boilerplate, predictable patterns, type inference against well-typed code), a small model is fine. For changes that span multiple files, touch novel code, or require reasoning about unfamiliar APIs, a larger model earns its cost. The interesting work is in the middle: tasks where a small model has a 30 percent failure rate, and a larger model has a 5 percent failure rate, and the costs are surprisingly close once you include the human review time on the small-model failures.
There is also a hidden version of this. The same model name across providers can have meaningfully different capabilities depending on quantization (how the model’s weights are compressed, which trades some quality for cheaper inference), inference stack, and routing on the provider side. A “small” model from one provider is sometimes a “medium” model in practice. The only way to know is to run your eval suite against the actual API you are paying for, not against the marketing page.
Cache the boring stuff, batch the rest
A surprisingly large fraction of AI coding token spend is repeated context. Open a file, read a function definition, ask for an edit that needs the function definition again. Most assistants handle this with prompt caching (the provider stores the prefix and charges a fraction of the price to reuse it), which can drop repeat-window cost by 70 to 90 percent depending on the provider. If your tool does not cache automatically, you have an opportunity.
Beyond caching, batching is the quieter win. A single large prompt that asks for three related edits is cheaper than three sequential prompts that each carry their own context window. The reason is the prefix cost. Once you have loaded a file into the model’s context (context is the maximum amount of text the model can consider at once), every subsequent message in the same conversation is paying only the marginal cost. Splitting that into three conversations pays the prefix three times.
The realistic pattern in day-to-day work is to keep the conversation scoped to a single task or file pair, even if that means starting fresh often. A sprawling 50-message conversation is usually costing you more than it saves, because every new ask brings in stale context the model has to re-read.
Track where tokens actually go
You cannot optimize what you cannot see. Most teams I have watched skip the cost-telemetry step entirely until they are already bleeding money. The minimum useful setup is a per-developer, per-day cost dashboard, broken out by tool and rough task category. You do not need a custom telemetry stack for this. A spreadsheet exporting the day’s tokens from your billing API is fine for a small team.
A short list of the cost signals worth tracking from week one:
- Cost per resolved task, by tool. Not cost per request, and not gross spend.
- Acceptance rate (the fraction of assistant outputs the developer keeps without rewriting).
- Retry count per session, averaged across the team.
- Top 10 percent of sessions by token cost, tagged with a one-line description of what was being done.
The reason this matters is that token spend is extremely skew-distributed across developers and across days. A small number of developers and a small number of heavy sessions will account for most of the bill. Without the per-developer breakdown, you cannot find those people, and you cannot figure out what they are doing differently. Sometimes they are doing meaningful work. Sometimes they are in a debug loop running the same prompt over and over.
Treat retries as a first-class problem
Retries are the silent budget killer in AI coding tools. A flaky assistant that gets stuck on a tool call, or that produces output that fails a build, will retry the same task until it succeeds or hits a limit. If your retry policy is not tuned, you can easily spend five times as much as you intended on a task that should have been one prompt.
The cost-control pattern is dual: a timeout-and-fallback that forces the assistant to escalate (switch to a different model, or ask the developer for guidance) after a few failed attempts, and a deduplication layer that recognizes “same prompt, same context, three times in a row” and breaks the loop.
Trade-offs
None of these levers are free. Tiered routing requires you to maintain an evaluation suite that actually predicts production quality, and that is real engineering work. Prompt caching depends on the provider supporting it well, and pricing models change. Per-developer cost dashboards get pushback from developers who correctly point out that cost is a poor proxy for value. Retry controls turn a sometimes-helpful “try again” loop into an occasional false negative where the second try would have succeeded.
The honest answer to “how do I make AI coding cheaper” is that you should expect to spend some engineering time on this. The teams that skipped that work and just turned off the largest model paid for it later in two ways: developer time spent correcting bad suggestions, and a quiet loss of trust in the tool that ended up reducing adoption anyway.
What I would tell past me: do not let the cost dashboard be a Saturday-morning project. Build it the same week the team starts using AI coding tools, because by month three the data is already stale and nobody wants to retroactively reconstruct it.
Bottom line: Optimize cost-per-resolved-task, not cost-per-request. Route by difficulty. Cache and batch. Watch the spend distribution. Tune retries. None of these are silver bullets, and the model swap is the smallest of the five wins.