AI Engineering

What an LLM Call Actually Costs: Six Hidden Levers Beyond the Price Tag

The "price per million tokens" on a pricing page is a starting point, not your bill. Here are the six variables that decide what a request really costs — and how to measure them per task instead of per token.

October 28, 2026 · 9 min read

Most teams start model selection by opening a pricing page and comparing dollars per million tokens. It is a reasonable first move, and it is almost never the number that shows up on your invoice. Real spend is decided by the ratio of input to output, cache hit rates, how your context lengths are distributed, which reasoning tier you run in, and how you schedule work.

Below are the six levers that move cost the most, with arithmetic you can redo against your own traffic. Figures are illustrative at the time of writing and rounded for readability — always check the live model page before you budget.

Lever 1: Input and output are not symmetric

Output tokens typically cost three to six times more than input tokens. On one current flagship, input runs about

2 per million tokens and output about

12 — a 6x gap. Put differently: letting a model write 500 more words costs roughly what feeding it 6,000 more words of context costs.

That flips a common instinct. Teams spend days compressing prompts and seconds specifying output format. In practice, capping output is usually the cheaper win:

Ask for structured output (JSON with fixed fields) instead of prose plus a summary.

Set max_tokens to something honest, not to the model maximum.

Put length constraints in the instruction, and validate them in code.

Avoid "explain your reasoning in the answer" unless a human actually reads it.

Lever 2: Cache reads are cheap, cache writes are not

A cache read is often priced at roughly one tenth of the input rate. A cache write carries a premium: the short-TTL tier is usually around the input price, and the long-TTL tier can be 1.6–2x.

This is where most cost bugs live. Caching only pays if your prefix is byte-stable. The system prompt, tool definitions, and few-shot examples belong at the very front and must not change between requests. Anything volatile — a timestamp, a request UUID, a randomly ordered list of retrieved documents, a user name injected at the top — invalidates the prefix and silently converts a cheap cache read into a full-price input charge, plus a write.

A practical rule: stable content first, volatile content last, and never interpolate anything into the top of the prompt. Add a lint check in CI that hashes the first N characters of each prompt template and fails when two identical inputs produce different hashes.

Lever 3: Context tiers are cliffs, not slopes

Many models price in tiers. A common pattern: prompts up to some threshold (say 272K tokens) use a lower rate, and anything above it moves to a higher one.

The important detail is that the higher rate usually applies to the whole request, not just the excess. A 280K-token request can cost more than two 140K-token requests carrying the same information. If your documents are naturally chunked, chunking is often cheaper than one long call — and frequently more accurate, because the model is not hunting for a needle across a haystack it was never asked to search.

Lever 4: Thinking tokens are billed as output

On reasoning models, thinking is metered as output. The same question at different reasoning effort levels can produce anything from a similar amount of visible text to many times more hidden computation.

Treat reasoning effort as a dial with a default, not a switch you leave on maximum. Start low, run your evaluation set, and raise it only for task families where the quality delta is measurable. Two habits help:

Log reasoning token counts separately from visible output tokens, so you can see where spend goes.

Keep a per-task quality floor. "Better" without a threshold becomes an argument; a threshold makes it a decision.

Lever 5: Time, priority, and speed are priced

Price lists increasingly sell the same model at different speeds and urgencies:

Option	Typical price relationship
Batch / offline queue	~50% of standard
Standard	1x
Priority tier	~1.8x standard
High-speed variant of the same model	up to ~2x standard
Off-peak window (where offered)	~50% of peak

None of this is a coupon trick; it is product design. Nightly re-indexing, bulk classification, and backfill jobs have no human waiting on them and should never run on a priority lane. Interactive chat and agent steps do have a human waiting, and paying for latency there is rational. The costliest mistake is letting an offline batch job inherit the interactive configuration.

Lever 6: Measure per task, not per million tokens

The only unit that survives contact with a finance team is cost per completed task:

task_cost = (input_tokens x input_rate)
          + (cached_tokens x cache_read_rate)
          + (cache_write_tokens x cache_write_rate)
          + (output_tokens x output_rate)
          + (tool_calls x tool_unit_price)

Worked example A: document summarization

A 60,000-token input with a 1,500-token output, priced at

2 /

12 per million with a $0.20 cache-read rate:

No cache: (60,000 x

2) + (1,500 x

12) per million = $0.138

55,000 tokens cached, 5,000 fresh: (55,000 x

0.20) + (5,000 x

2) + (1,500 x

12) per million ≈ **

0.039**

A 72% reduction with no change of model — only a change of prompt structure.

Worked example B: a 20-step agent loop

Each step sends 40,000 input tokens (8,000 of fixed preamble, 32,000 of growing history) and receives 800 output tokens:

No cache: 20 x (

0.080 +

0.0096) ≈ $1.79

8,000-token stable prefix cached: 20 x (

0.0016 +

0.0640 +

0.0096) ≈ **

1.50**

Prefix cached and history trimmed to 16,000 tokens: 20 x (

0.0016 +

0.0320 +

0.0096) ≈ **

0.86**

Note which lever did the heavy lifting. Caching saved 16%; context discipline saved more than half. Trimming history costs information, so it must be validated against quality — but it is usually the largest single line item in agent workloads.

A checklist you can run this week

Tag every request with feature, model, input tokens, output tokens, cached tokens, and a computed cost. Untagged spend cannot be optimized.

Report cost per feature per task, not total spend. Totals hide the one endpoint that costs 60% of the bill.

Lint your prompt prefixes for timestamps, UUIDs, unordered serialization, and injected user data.

Set explicit output budgets per endpoint and enforce them in code, not in documentation.

Route by difficulty: cheap model first, escalate on low confidence or validation failure. Measure how often escalation actually fires.

Schedule offline work onto batch lanes and off-peak windows; keep interactive lanes clean.

Re-run the numbers monthly. Prices, cache rules, and thinking defaults change faster than most teams assume.

The takeaway

Cost engineering is mostly translation work: converting "per million tokens" into "per completed task," then attacking the largest component. The ordering is rarely the intuitive one — caching and context discipline usually beat model downgrades, and model downgrades usually beat prompt micro-optimization. Measure first, then swap.

Prices and billing rules cited above are illustrative and change frequently. Verify against current model pages before committing a budget.