A million calls a day turns the LLM bill into a line the CFO reads. At 1,500 input and 300 output tokens per call, that is 45 billion input and 9 billion output tokens a month. The same workload costs between $2,700 and $900,000 a month depending on the model and the discounts applied. Prices below are as of October 2026.
Three things change at this volume. Which vendors you can use at all depends on rate limits and spend caps more than on list price. Discounts (cache, batch, off-peak) move the bill more than a switch between similarly priced models does. A self-hosted open-weight model also stops being a side project. It becomes a column in the same spreadsheet.
The reference workload, costed
Monthly bill for 1M calls a day, 1,500 input and 300 output tokens each, over 30 days:
| Model (input / output per 1M tokens) | List price | With cache | With batch | Cache and batch |
|---|---|---|---|---|
| GPT-6 Astra ($10 / $50) | $900,000 | $576,000 | $450,000 | not published |
| Claude Sonnet 5.5 ($2 / $10) | $180,000 | $111,600 | $90,000 | $57,600 |
| Claude Haiku 5.5 ($0.10 / $0.50) | $9,000 | $5,760 | $4,500 | $3,060 |
| gpt-oss-120B on Groq ($0.15 / $0.60) | $12,150 | $9,450 | Developer plan only, discount not published | — |
"With cache" assumes 1,200 of the 1,500 input tokens are a shared prefix (system prompt, tool definitions, examples). Every call reads that prefix from cache at $1 (Astra), $0.10 (Sonnet), $0.01 (Haiku) and $0.075 (Groq) per million. The table ignores cache writes, because at twelve calls a second the prefix never expires. Batch is 50% off at OpenAI and Anthropic. Anthropic states that batch stacks with caching, and this table bills cache reads inside a batch at the cache rate. The Haiku price applies to prompts of 100K tokens or less. GPT-6 Luna lists the same $0.10 / $0.50.
The table supports two readings. First, the gap to the frontier row is large. Astra costs 100 times Haiku on the same traffic, and Claude Opus 5.5 ($4 / $20) would come to $360,000. Second, "hosted open-weight" has no single price. The same gpt-oss-120B costs $0.03 / $0.15 at OpenRouter's cheapest routed provider. That comes to $2,700 a month, a 4.5x spread for identical weights.
Rate limits are the first wall
One million calls a day is 694 requests a minute on average, with 1.04M input tokens and 208K output tokens a minute. Assume traffic peaks at two to three times the average, and plan for roughly 2,000 RPM and 3M input TPM.
| Vendor and tier | Monthly cap | RPM | Input TPM | Verdict for this workload |
|---|---|---|---|---|
| OpenAI Build ($5 in purchases) | $500 | 5,000 | 1,000,000 | cap and TPM both too low |
| OpenAI Launch ($100) | $5,000 | 10,000 | 4,000,000 | TPM fits, cap does not |
| OpenAI Grow ($500) | $200,000 | 15,000 | 40,000,000 | fits Sol-class bills, not Astra |
| Anthropic Start | $500 | 1,000 | 2,000,000 | no |
| Anthropic Build | $1,000 | 5,000 | 5,000,000 | cap too low |
| Anthropic Scale | $200,000 | 10,000 | 10,000,000 | Sonnet and Haiku fit; Opus 5.5 needs Custom |
| Gemini Tier 2 ($100 paid, 3 days) | $2,000 billing cap; $50 per rolling 10 min | shown in AI Studio | shown in AI Studio | cap gone on day one |
| Gemini Tier 3 ($1,000 paid, 30 days) | $20,000 to $100,000+; $200 per 10 min | shown in AI Studio | shown in AI Studio | 10-minute limit fits Flash-class bills; a $20,000 billing cap would not |
OpenAI's RPM and TPM figures are for Astra, Sol and Terra. Luna gets more (30,000 RPM and 180M TPM at Grow). Anthropic's output TPM is 2M at Scale, ten times the average need here. Anthropic's Custom tier has no cap and is arranged with the account team. The published OpenAI tiers stop at Grow.
Every vendor follows the same pattern. By the top published tier the token-per-minute limits are generous, and the monthly spend cap is what blocks you. A Gemini 3.8 Flash bill on this workload is $67,500 a month, or $15.63 per ten minutes. That sits well inside Tier 3's $200 window, but Tier 3 requires $1,000 paid and 30 days of history. OpenAI upgrades tiers automatically on cumulative credit purchases. Anthropic assigns them "based on usage history and account standing". Either way, you earn next month's tier this month.
Five moves work when a tier is the ceiling.
- Start the conversation with the account team before launch rather than after the first 429s. Gemini's 30-day clock cannot be skipped.
- Split traffic across two vendors behind a router, so neither tier is the limit. The AI load balancers article covers the tooling.
- Smooth the ramp. OpenAI asks that above 1M input TPM you raise traffic by no more than 50% every 15 minutes, or expect
slow_downerrors. A queue in front of the API does this for you. - Move the non-urgent share to the batch endpoint, which has its own limit pool. Anthropic allows 4,000 batch requests a minute and 500,000 queued at Scale. Gemini Tier 3 allows 1 billion enqueued tokens for 3.8 Flash. OpenAI lists its enqueued-token cap per model on the Settings page.
- Buy priority where you need latency guarantees and sell it where you do not. Gemini's priority tier costs 1.8x and starts with default limits at 0.3x of standard. Its flex tier is billed at batch price. OpenAI's Flex processing is also at batch price for Astra and Terra.
Today we are rolling out service tiers in the Gemini API! You can now (optionally) set "flex" or "priority". In the case of flex, this will save you ~50% on API costs (with lower reliability). In the case of priority, this will cost ~80% more but give you higher priority!
— Logan Kilpatrick (@OfficialLoganK) April 2, 2026
Four levers, applied in order
The worked example below runs on Claude Sonnet 5.5 at $180,000 a month list. Each lever is applied on top of the previous one.
1. Cache the shared prefix: $111,600
Read from cache, the 1,200-token prefix drops from $2 to $0.10 per million. The 300 variable tokens and the output stay as they were. The bill falls 38%, and the product does not change. The prompt caching guide covers the mechanics per vendor. At volume the one rule is to keep the prefix byte-identical. A single moving timestamp at the top of the prompt defeats the cache.
2. Batch the non-urgent share: $90,000
Not every call needs an answer in two seconds. Nightly enrichment, re-indexing, evaluations, digests and anything a user is not waiting on can go through a batch endpoint at half price. Assume 40% of the traffic qualifies. Anthropic reports most batches finish in under an hour, with a 24-hour ceiling. OpenAI fixes the window at 24 hours. Gemini targets 24 hours and expires jobs after 48. Details and limits are in Batch APIs compared.
3. Route by difficulty: $47,340
Half of a typical production mix is classification, extraction, short rewrites and formatting. Those calls go to Haiku 5.5 at one twentieth of Sonnet's price, with the same cache and batch treatment. The non-batched part of the easy traffic (30% of all calls) costs $57.60 a day on Haiku instead of $1,116 on Sonnet. Model routing shows how to build the classifier that makes this decision.
4. Shorter outputs, trimmed context: $32,220
Output tokens cost five times input tokens on every vendor model in the table (four times on gpt-oss-120B at Groq). Two cuts take another $15,120 off. The average answer drops from 300 to 200 tokens (tighter instructions, stop sequences, no restating the question). The variable context drops from 300 to 200 tokens (fewer retrieved chunks, no chat history older than needed). This lever comes last because it needs product decisions rather than configuration.
The order matters. Caching and batching are configuration and ship in a week. Routing needs an evaluation set to prove the cheap model holds up on the easy half. Token trimming needs product owners in the room.
When self-hosting wins
These open-weight options are worth costing at this scale. The table lists their published hosted prices.
| Model | Licence | Hosted price (input / output per 1M) |
|---|---|---|
| gpt-oss-120B (OpenAI) | Apache 2.0 | Groq and Together $0.15 / $0.60; OpenRouter cheapest route $0.03 / $0.15 |
| Llama 3.3 70B (Meta) | community licence | OpenRouter $0.10 / $0.32; Cloudflare Workers AI $0.293 / $2.253 |
| DeepSeek V4.1 Flash | MIT | DeepSeek $0.30 / $1.20 peak, $0.15 / $0.60 off-peak; Together $0.30 / $1.20 |
| Qwen3.8-27B (Alibaba) | Apache 2.0 | OpenRouter from $0.035 / $1.50, 21 providers |
| Gemma 4 31B (Google) | Apache 2.0 | hosted price not in our data |
Licences for DeepSeek V4.1 Flash, Qwen3.8-27B and Gemma 4 are as reported by DataCamp, LetsDataScience and The Next Web respectively. The Llama 3.3 Community License is Meta's own.
Hosted open-weight at a mid-range provider ($0.15 / $0.60) costs more than Haiku 5.5 on the same traffic. The cheapest routed providers undercut every vendor budget model. The trade-off is unknown hardware, quantisation and uptime. DevelopersDigest reported on 17 October 2026 that the best open-weight model scores 46 on the Artificial Analysis Intelligence Index, against 58 for the leading closed one. A fair comparison is therefore with a vendor's budget tier rather than its flagship.
Self-hosting on your own GPUs changes the shape of the cost. We have not verified hardware prices for this article and will not quote them. The break-even reasoning holds whatever the price, and it rests on four factors.
- Utilisation. A rented or owned GPU costs the same idle or busy. At 1M calls a day, a steady twelve a second keeps a box busy. A daily peak three times the average means you provision for the peak and pay for it at night. Batch work fills the trough.
- Memory. According to TestingCatalog's launch report, gpt-oss-120B runs on one 80 GB GPU and the 20B version fits in 16 GB. The Ollama rule of thumb is 8 GB of RAM for 7B models, 16 GB for 13B and 32 GB for 33B. By that rule, a 27B or 31B dense model needs a serious card before throughput even comes up.
- People. Someone owns the serving stack, the upgrades, the on-call rota and the evaluation when a new checkpoint lands. At a million calls a day that is at least one engineer who would otherwise be shipping features.
- Latency. Your own box has no noisy neighbours and no vendor queue. When self-hosting wins, this is usually the reason.
The usual serving stack is vLLM. The talk below, from a vLLM co-lead at PyTorch Conference 2025, covers the throughput and cost of serving models on your own hardware.
For the full hosted-versus-local arithmetic, including what utilisation rate you need before owning hardware pays, see Local LLM vs cloud API.
Risks that only show up at volume
A 5% price change on a $500 bill is noise. On a $180,000 bill it is a hire. Four risks are worth a line in the budget.
Scheduled price increases. Gemini 4 Argon goes from $2 / $10 to $4 / $20 on 1 December 2026. Overnight, that doubles our reference workload on Argon from $180,000 to $360,000 a month. Gemini 3.8 and 3.6 Flash go from $0.75 / $3.75 to $1.50 / $7.50 on 1 January 2027. GPT-5.6 Sol's $4 / $20 is a promotion running to at least 21 November 2026. After that, the standard $5 / $30 applies. DeepSeek starts charging $0.75 per 1,000 server-side search and fetch calls on 1 November 2026. Discounts also come and go. DeepSeek announced the end of off-peak discounts for 5 September 2025, yet its current pricing page lists a 50% off-peak rate again.
Deprecations. GPT-5.4 nano is marked deprecated on its model page. According to CloudZero, Gemini 2.0 Flash was shut down on 1 June 2026 and Gemini 2.5 Flash reached its deprecation date on 16 October 2026. Google paused the Gemini 2.5 Pro free tier in May 2025 under demand. At a million calls a day, a forced migration means re-running your evaluation set and re-tuning prompts on the vendor's timetable instead of your own. Pin model versions and keep a second model qualified.
Outages. The split-across-vendors move from the rate-limit section is also the outage plan. A queue that holds an hour of traffic and a router that fails over cost less than the incident review.
Batch retention. Batch output does not sit there forever. OpenAI deletes output files 30 days after completion. Anthropic keeps results downloadable for 29 days. Gemini stores them for six weeks. OpenAI cancels unfinished requests at 24 hours and bills the completed ones. Download and store results as part of the job itself.
What to do next
Cost the reference workload above with your own token counts and model. The spreadsheet takes an hour and ends most arguments.
Check which tier you are on today and what the next one requires. The qualification clock (days since first payment, cumulative spend) starts only when you start it.
Ship caching and batching this month. Measure the cache hit rate and the batch share before touching routing.
Run one open-weight model on hosted inference against your evaluation set. If it holds on the easy half of the traffic, the self-hosting conversation has a number in it.
We review high-volume pipelines at Divero, usually starting from the vendor invoice and the tier page.