Last week I asked an AI coding agent to reply with a single word: OK. Before it produced that word, it had already sent 31,798 tokens of its own plumbing to a model. Not my test file, not my task. Just the harness that wraps the model: the loop that builds each request, lists every tool it might call, and re-sends the whole conversation on every step.
That number did not come from a badly configured machine with a hundred plugins bolted on. It came from a clean Playwright repo with one config file, one spec file, no agent instruction files, no MCP servers and a git history of exactly one commit. It is the floor, not the ceiling.
So here is my thesis, and I will defend it below. The defining cost problem in agentic coding is no longer the model. It is the harness. We spend our days comparing Opus and GPT and Qwen on leaderboard charts while the loop around them quietly decides how many tokens get billed on every one of the 50 to 90 steps a real task takes.
The benchmark that measures the part you never see
TestDino ran a harness overhead benchmark on 8 October 2026 across 11 coding agents. Every agent opened the same minimal Playwright project, and the prompt was seven words: reply with the single word OK. They ran each agent three times and recorded the input tokens of the very first model call, as the sum of uncached input, cache writes and cache reads. Nothing else is in that number. No file read, no tool call, no reasoning. It is pure harness.
- Kiro CLI shipped 31,798 tokens. Its own debug log itemizes the damage: 31,722 of those tokens are tool definitions, 58 are system prompt, 18 are context files. The harness carries a huge toolbox and the model must read the entire catalogue before it looks at your code.
- Claude Code came in at 31,480 on a clean profile. Add a global CLAUDE.md, auto memory and three MCP servers, as on a working developer machine, and the first call rises to 34,161.
- Copilot CLI used 17,059, Codex CLI 14,768 and Cursor CLI 13,174. On a fresh account, 12,544 of Codex's tokens returned as cache hits, so its genuinely new payload was closer to 2,200.
- Cline ran a frontier model on 5,382 tokens. Pi sent 1,894, with identical prompts across all three runs and zero cache reads. Aider sat near 2,500.
That is a 16.8x spread between the lightest and heaviest harness, in the same repo, on the same day. And because the growing part of a request is only seven words long in this test, the table is essentially the fixed part of each harness, which is the part you pay for on every single step.
One honest caveat: each agent ran its own default model, so the counts span four different tokenizers. Treat gaps of five to ten percent between neighbours as ties. That does not weaken the headline. Kiro and Claude Code both cross 31,000 tokens while Pi stays under 1,900, and no tokenizer explains a gap that size.
Why the catalogue only ever grows
Anthropic's own tool search documentation makes the same point from the vendor side: a typical multi-server MCP setup can consume around 55,000 tokens of tool definitions before the model does any work, and tool selection accuracy starts to degrade once you pass 30 to 50 tools. Everyone involved knows the arithmetic. The catalogue still grows.
There is a comparison that should worry teams more than any leaderboard swing. Claude Code's clean 2.1 preamble measured 16,581 tokens earlier this year. It is now 31,480. That is not a model regression, it is a wrapper regression, and it happened inside a single major version while nobody was looking at the right number.
I do not think this is an engineering accident. A tool catalogue is a capability surface, and capability surfaces win demos, launch posts and comparison tables. Vendors set the price per token; the harness sets the volume. Almost nobody is competing on the volume.
What I would actually fix first
The good news is that harness overhead is one of the few AI cost problems you can measure in a minute without an eval harness, a GPU cluster or a research budget. Here is the checklist I would hand a team tomorrow morning.
- Measure the first call yourself. One seven-word prompt, one flag for JSON usage output, and you have your baseline before you have an opinion about models.
- Count your tools, not your models. If your setup carries 30 or more definitions, assume both the bill and the tool-selection error rate are worse than your benchmarks suggest.
- Match harness weight to task size. A small refactor does not need a full toolbox. A 1,894-token harness and a 31,798-token harness are not competing on the same job.
- Re-measure after every upgrade. Preambles grew 15,000 tokens inside one version. Assume the next release adds more.
There is a second invoice attached to all of this, and it is worse than tokens. When agents write the code, they move work from writing to reading, and reading is the harder half. A twenty-line fix comes back as two hundred lines, the reasoning behind a subtle architectural choice is never written down, and generated code looks clean enough to skim. Every decision the agent made silently is a decision nobody on your team now owns.
Unread code is technical debt the moment it merges, and the interest gets paid at 3 a.m. during an outage in a code path no human actually wrote. So keep the agents, absolutely, but keep making the decisions that shape the system yourself. Delegate the typing. Never delegate the understanding.
The question is not how much code your agent wrote this week. The question is how many tokens it sent to say OK, and whether anyone on your team can still fix the system when it breaks tonight.
Comments