An LLM cost audit traces where each dollar goes. It divides the bill three ways. By feature, it tells you which product surface made each call. By model, it records which price list applied. By token type, it separates uncached input, cached input, output, and the tokens that retries re-send. Most bills show only the model view, because that is the view the vendor dashboard gives. The first week of an audit makes the other two visible. After that, the audit checks the logs against a fixed list of leaks. This article is that list, with prices as of October 2026.
Step 1. Get the numbers
A vendor dashboard shows spend per model per day. It cannot tell you that one feature costs four times another, or that retries sent part of the input twice. Per-feature attribution needs your own tags on every call.
Log these fields per request, at the gateway or in the client wrapper:
| Field | Why it matters |
|---|---|
model |
Which price list applied |
prompt_tokens, split into uncached, cache read, cache write |
Cache hit rate per feature; the split is in the response usage block |
completion_tokens, plus reasoning tokens where reported |
Output is the expensive token; reasoning is billed as output |
feature (or tenant, endpoint) |
Attributes cost to a product decision |
latency_ms |
Timeouts correlate with retries |
attempt and outcome |
Finds duplicate sends |
vendor request_id |
Reconciles your log against the invoice |
What the bill looks like before and after tagging:
| Before tagging | After tagging | |
|---|---|---|
| Rows | one per model per day | one per feature per model per day |
| Input tokens | one number | uncached, cache read, cache write |
| Output tokens | one number | answer tokens, reasoning tokens |
| Retries | invisible, folded into input | counted, with their re-sent tokens |
| Dev and eval traffic | mixed into production | its own key, workspace or project |
| Owner of each line | nobody | the team that owns the feature |
Some vendors give you part of this for free. Anthropic lets you set spend limits per workspace and tracks Claude Code usage in a dedicated workspace. Those buckets are coarse, so the feature-level tags still have to be yours.
Export a week of tagged logs, multiply by list prices, and reconcile the total against the invoice. A gap of more than a few percent means something is untagged. Then start the checklist.
Step 2. The leak checklist
Each leak leaves a signature in the logs and carries a price. The prices below are list prices from the vendor pages in Sources.
Leak A. An uncached system prompt on every call
The system prompt, tool definitions and few-shot examples are the same on every call. Without caching, each call pays the full input price for them.
To spot it, look for cache read tokens near zero on a feature whose prompt starts the same way each time. Another sign is a low hit rate. The cause is something volatile (a timestamp, the user's name) at the top of the prompt, which breaks the prefix.
The cost is the gap between the input price and the cache read price.
| Model | Input per 1M | Cache read per 1M | Cache write per 1M (5 min) |
|---|---|---|---|
| Claude Opus 5.5 | $4.00 | $0.20 | $5.00 |
| Claude Sonnet 5.5 | $2.00 | $0.10 | $2.50 |
| Claude Haiku 5.5 (prompt under 100k) | $0.10 | $0.01 | $0.125 |
| GPT-6 Astra | $10.00 | $1.00 | $12.50 |
| Gemini 3.8 Flash | $0.75 ($1.50 from 1 Jan 2027) | $0.075 ($0.15 from 1 Jan 2027) | plus $0.50 per 1M tokens per hour of storage ($1.00 from 1 Jan 2027) |
Anthropic and OpenAI (GPT-6 Astra) bill cache writes at 1.25x the input price. A prompt that changes on every call gets written to cache each time, and costs more than no caching at all. Gemini explicit caching adds an hourly storage charge. A cache kept warm for content nobody re-reads becomes a cost line of its own. So the audit asks for the hit rate per feature. Knowing that caching is switched on tells you nothing.
With prompt caching, you can reuse a book's worth of context across multiple API requests. This can also reduce latency by up to 85% on long prompts.
— Anthropic (@AnthropicAI) August 14, 2024
Use cases include coding assistants, large document processing, and agentic tool use.
See the prompt caching guide for the mechanics per vendor.
Leak B. A frontier model on small-model work
This leak covers classification, extraction into a schema, reformatting, language detection and yes/no decisions. They run on the flagship model because someone picked one model at the start and nobody revisited the choice.
To spot it, group calls by feature and compare output length and prompt shape. A feature with answers under 50 tokens and a prompt ending in "respond with one of: ..." is a routing candidate.
The cost is the price ratio between tiers inside one vendor.
| Vendor | Top tier (in / out) | Small tier (in / out) | Ratio |
|---|---|---|---|
| Anthropic | Opus 5.5 $4 / $20 | Haiku 5.5 $0.10 / $0.50 | 40x |
| OpenAI | GPT-6 Astra $10 / $50 | GPT-6 Luna $0.10 / $0.50 | 100x |
| Gemini 4 Argon $2 / $10 ($4 / $20 from 1 Dec 2026) | Gemini 3.1 Flash-Lite $0.25 / $1.50 | 7x to 8x (13x to 16x from 1 Dec 2026) |
The fix needs an evaluation set per feature to measure the downgrade. The method is in model routing.
Leak C. Output tokens
Output costs five times input on most current price lists (Sonnet 5.5 $2 in, $10 out; GPT-6 Astra $10 in, $50 out; Haiku 5.5 $0.10 in, $0.50 out). Three habits inflate it. The first is answers that restate the question and end with a closing paragraph nobody reads. The second is JSON returned pretty-printed, with indentation and newlines. The third is reasoning. Anthropic's Claude Code documentation states that thinking tokens are billed as output tokens. It adds that the default budget "can be tens of thousands of tokens per request depending on the model".
To spot it, compare median and 95th-percentile completion_tokens per feature with what the consumer of the answer uses. A classification feature with a 200-token median pays for explanations it discards.
Each output token saved is worth five input tokens. Take a feature on Sonnet 5.5 that makes 1M calls a month. Trimming 100 tokens of boilerplate per answer removes 100M output tokens, or $1,000 a month.
Leak D. Retries that re-send the whole context
A retry re-sends the full prompt. If the first attempt timed out on the client but completed on the server, the vendor bills both. Rate-limit errors (HTTP 429) retried in a tight loop do the same. OpenAI's rate-limit guidance covers traffic above 1M input tokens per minute. It asks you to raise such traffic by no more than 50% every 15 minutes to avoid slow_down errors. A deploy that ignores this produces a burst of retried calls.
To spot it, count requests per attempt value. You pay twice for anything above attempt 1. Then match client timeouts against vendor request_ids that did complete.
The cost is the retry rate times the input bill. A 5% retry rate on a 2,000-token prompt at Sonnet 5.5 prices adds $200 per million calls. That figure leaves out the output of the attempts that did finish.
Leak E. Agents in loops
Each agent step sends the whole history again. Anthropic's Claude Code documentation describes the mechanism. The tool "sends your full conversation with every request, and each time Claude uses tools it sends another request carrying that batch of tool results." A raw tool result in context (a 10,000-line log, a full API response) then rides along on every following step. Subagents send their own requests on top of the main conversation. A subagent's first request does not read the parent's cache, because the prefixes differ. The same page says agent teams with teammates in plan mode use about 7x the tokens of a standard session.
To spot it, plot input tokens per step for one agent run. A straight rising line means the agent re-sends history. A jump means a tool result that should have been summarised. A gap between steps longer than the cache lifetime shows up as a full-price re-read.
The cost is the sum of that series. An agent that runs 20 steps, with context growing from 5k to 60k tokens, sends about 650k input tokens for one task. On Sonnet 5.5 uncached, that is $1.30 per task. At the cache read rate it is $0.065 plus the writes. More detail is in the hidden token tax and budget guardrails for agents.
Leak F. Context stuffing
Context stuffing means attaching whole documents where a retrieved chunk would do. It also covers chat history that nobody trims. A third form is a system prompt or CLAUDE.md that grew to 2,000 lines because nobody deletes anything. You pay for every token, and long prompts also cross price thresholds:
| Vendor | Threshold | What happens above it |
|---|---|---|
| Anthropic Haiku 5.5 | 100k prompt tokens | input $0.10 becomes $0.50, output $0.50 becomes $2.50 |
| OpenAI GPT-6 Astra | 272k input tokens | 2x input and cache price, 1.5x output, for the whole request |
| Google Gemini 3.1 Pro Preview | 200k tokens | input $2 becomes $4, output $12 becomes $18 |
| xAI grok-4.7 | 200k prompt tokens | input $2 becomes $4, output $6 becomes $12 |
To spot it, check the prompt_tokens distribution per feature and count the requests above each vendor's threshold. For chat features, plot prompt length against turn number. A line that never flattens means nobody trims the history.
The cost grows linearly with tokens below the threshold and steps up above it. Anthropic's guidance for Claude Code is to keep CLAUDE.md under 200 lines. It also advises moving specialised instructions into skills that load on demand.
Leak G. Synchronous calls for work that can wait
Nightly summaries, embeddings backfills, evaluation runs and the classification of yesterday's tickets need no answer within two seconds. Teams still often send them through the synchronous endpoint at full price.
To spot it, look for features with no user waiting on the result. Cron-shaped bursts of calls are the usual signature.
The cost is half the bill for those features. The OpenAI, Anthropic and Gemini batch endpoints all list a 50% discount on input and output. xAI lists 20%. Turnaround is within 24 hours, and Anthropic states that most batches finish in under an hour. Anthropic's batch discount also stacks with prompt caching. Batch APIs compared sets the limits side by side.
Leak H. Non-production traffic on production keys
This traffic comes from developers testing prompts, CI running the eval suite on every commit, and observability tools that re-send samples to a judge model. On the production invoice it looks the same as customer traffic.
To spot it, ask the bill "how much did the eval suite cost last month". If it cannot answer, this leak is open. The log signs are volume outside business patterns, repeated identical prompts and bursts that line up with deploys.
The cost stays unknown until you separate the traffic. Give each environment its own key. Where the vendor offers workspaces with their own spend limits, use them too. The guess then becomes a line item. Anthropic's workspace spend limits work this way. Organisation-wide limits still apply, even when the workspace limits add up to more.
Worked example for a support assistant
A support assistant handles 1M calls a month. Each call carries a 1,500-token system prompt and 500 tokens of user text, and gets a 300-token answer. It runs on Claude Sonnet 5.5 ($2 input, $10 output, $0.10 cache read). At about 23 calls a minute, the five-minute cache never goes cold, so cache writes round to zero.
| Line | No caching | Cached system prompt | Cached, and 60% of tickets routed to Haiku 5.5 |
|---|---|---|---|
| System prompt, 1,500M tokens | 1,500M × $2 = $3,000 | 1,500M × $0.10 = $150 | 900M × $0.01 + 600M × $0.10 = $69 |
| User text, 500M tokens | 500M × $2 = $1,000 | $1,000 | 300M × $0.10 + 200M × $2 = $430 |
| Answers, 300M tokens | 300M × $10 = $3,000 | $3,000 | 180M × $0.50 + 120M × $10 = $1,290 |
| Router call on Haiku 5.5 (700 in, 5 out, every ticket) | none | none | 700M × $0.10 + 5M × $0.50 = $73 |
| Total per month | $7,000 | $4,150 | $1,862 |
Caching alone removes $2,850 a month, 41% of the bill, and takes half a day. Routing removes a further $2,288. It needs an evaluation set to prove that Haiku 5.5 handles the simple 60% as well as Sonnet 5.5 does. Caching does not touch output tokens, so once the prompt is cached they become the largest line. That is why Leak C comes next on the list in practice.
Step 3. Rank fixes by payback
Rank fixes by the saving per hour of engineering. The largest saving is rarely the one to start with. The top two rows usually get done in week one, and they make the rest measurable.
| Fix | Effort | What it removes |
|---|---|---|
| Separate keys, workspaces or projects per environment and feature | hours | Dev, eval and CI traffic from the production bill; makes every other row measurable |
| Cache the stable prefix (reorder the prompt so volatile parts come last) | hours | Full input price on repeated tokens; Sonnet 5.5 $2 drops to $0.10 per 1M |
Cap max_tokens, strip whitespace from JSON, lower reasoning effort where not needed |
hours to days | Output tokens at five times the input price |
| Fix retry logic: idempotency, backoff, honest timeouts | days | Full context sent twice |
| Move offline jobs to the batch endpoint | days | Half the price on work with no user waiting |
| Route simple features to a small model, with an eval set per feature | days to weeks | Top-tier price on commodity tasks, 7x to 100x inside one vendor |
| Summarise tool results, trim chat history, give subagents a small model | weeks | Growing history re-sent on every agent step |
Re-run the Step 1 reconciliation after each fix. A fix that does not move the tagged bill was applied to the wrong feature.
What to do next
Add the seven log fields from Step 1 this week and reconcile one week of logs against the invoice before changing anything.
Run the checklist in order A to H against those logs.
Divero runs this audit as a fixed-scope engagement. Email [email protected] if you want the leak list run against your bill.