AI Engineering
The English-Only Illusion: Shipping LLM Features That Work in Every Language
Most LLM features are prototyped in English, then shipped in twenty languages without ever treating language as a variable. Six angles on tokenizer tax, uneven capability, prompt-language choice, real localization, biased evaluation, and per-locale unit economics.
October 29, 2026 · 9 min read
Most LLM features are born in English. The prompt is English, the test cases are English, and the acceptance bar is set by whoever happened to try it first. Then you add a language selector and find that the same product behaves differently: the German answer is three times longer, the Japanese summary quietly drops a clause, the Arabic layout breaks around punctuation, and the cost chart grows a second staircase nobody planned for.
None of that is a model bug. It is a design bug: one feature was built and shipped in twenty languages without ever making language a variable.
Here are six angles worth designing for, ordered from cheap to fix this week to decide before you launch.
1. Tokenization is a hidden tax — measure it, don't assume it
Every model sees text through its tokenizer. A tokenizer tuned mostly on English compresses English well and compresses other writing systems less well, sometimes dramatically. The consequence is not cosmetic: the same meaning consumes a different share of your context window, a different amount of money, and a different slice of latency depending on the language.
That single fact propagates into four places that all get misconfigured:
Context budget. A retrieved-document window sized on English will truncate the same document in a more token-hungry language, and truncation looks like a reasoning failure to users.
Cost. Your blended cost per request is really a weighted average over your traffic mix. If one locale grows, so does your bill, with no price change anywhere.
Latency. More output tokens means longer generation, which is felt as "the product is slow here."
Caching. If you rely on cached prefixes to cut cost, remember that only the stable part benefits; a language that inflates tokens fills the window sooner and leaves less room for the cached portion to pay off.
Measure it with your own tokenizer and your own samples. Produce a table like this once per model family, and re-run it whenever you change models — a tokenizer swap moves cost and truncation points even when the price list is untouched.
The numbers above are placeholders on purpose. The point is the habit: treat tok/char as a per-locale configuration value, not a blog fact.
2. Capability is asymmetric across languages
Leaderboards rank models with English-heavy evaluation. A model that sits first overall can be third or fifth in the language your customers actually use, and mean scores hide exactly that variance. Two practical consequences:
Stop selecting models with a global score. Select with a locale matrix. Decide which languages are Tier A (launch-critical, must be tested on every change), Tier B (supported, monitored, sampled for quality), and Tier C (best effort, clearly disclosed). Then choose the model per tier, not per planet.
Test language-specific failure modes, not just general quality. A golden set of twenty English-style questions will not catch:
Formal/informal address levels (Korean, Japanese) drifting between turns;
Grammatical gender and agreement (French, Spanish, German, Arabic) producing subtly wrong references to people;
Right-to-left scripts breaking numbers, dates, and mixed-script strings;
Code-switching — users mixing two languages in one sentence, which is normal for a large share of the world and rare in most test sets;
Untranslated leakage: the model answering in the prompt's language instead of the user's.
3. Decide deliberately what language the prompt is in
There are three defensible strategies, and choosing one by instinct rather than by measurement is where teams lose months.
Strategy | How it works | Where it tends to win | Where it hurts |
|---|---|---|---|
English instructions | System prompt and rules in English, output language specified | Complex rule-following, tool schemas, agentic flows | Instructions and user text collide; tone drifts in output |
Same-language instructions | Everything in the user's language | Consumer tone, cultural nuance, short tasks | Inconsistent rule compliance; harder to maintain 20 prompt copies |
Hybrid | English instructions + few-shot examples in the target language | Most production systems | Requires curated examples per locale |
The honest answer is that "English prompts are always better" is folklore. It is often slightly better for strict reasoning and formatting, and often worse for register and naturalness. Run an A/B on your own traffic, scored by your own rubric, before standardizing.
One rule that survives most of these debates: keep the language of your few-shot examples identical to the language you want back. Mixed-script examples produce mixed-script answers.
4. Localization is rewriting, not translating
A model that speaks a language still does not know your market. Locale is a data field, and it has to be passed explicitly rather than inferred from the text:
Dates, currencies, decimal and thousands separators, measurement units;
Name order and forms of address, honorifics, formal vs. informal "you";
Week start, calendar system, public holidays (they change your SLAs and your support load);
Legal and regulatory qualification: the same marketing claim can be acceptable in one market and require disclaimers in another;
Regional content policy and data-handling requirements, which may force a different provider — or a different deployment region — for a specific market.
Put
locale in your output schema and log it. When something goes wrong, the first question is always "which market, which language, which prompt version" — and if you did not store it, you will be reconstructing it from memory at 2 a.m.5. Your evaluation is biased too
Three habits separate teams that scale to many languages from teams that keep re-discovering the same bugs:
Small, real golden sets per locale. Thirty to a hundred cases reviewed by a native speaker beats two thousand machine-translated ones. Include the ugly inputs: typos, mixed scripts, emoji, pasted tables, screenshots-of-text.
Calibrate your judge. LLM-as-judge is not language-neutral: judges tend to prefer verbose answers and output in their own language, and they are stricter in low-resource languages. Measure judge-human agreement per locale, and keep a written rubric so the score means something across languages.
Watch production metrics per locale, not in aggregate. The six that catch most regressions:
- Output-format failure rate (JSON, schema, required fields);
- Refusal rate — a jump usually means a policy or prompt mismatch, not a model improvement;
- Language-consistency rate (did the answer come back in the requested language?);
- Length ratio versus English for the same task;
- Tokens per request and cost per resolved task;
- p95 latency and time-to-first-token.
And gate rollouts by locale. A model upgrade that improves English can quietly regress a Tier B language; if you roll out globally on day one, you will learn that from customers instead of from dashboards.
6. Run unit economics per locale
Cost discussions usually stop at "price per million tokens." For a global product that number is not actionable, because your traffic is not one language.
Report cost per resolved task, per locale. It is the only figure a product manager can act on.
Watch tiered pricing. Providers that price long contexts in tiers charge the higher tier for the whole request once a boundary is crossed, so a verbose language can double a single request's cost rather than raise it slightly. Test with your longest Tier A prompts, not your average one.
Recheck verbose-language prompts first when a provider changes tokenizer, token accounting, or reasoning-token billing; those changes hit the most token-hungry locale hardest.
Consider routing, not just model choice. Short, high-volume Tier C tasks and long, low-volume Tier A tasks usually want different models — and that decision should be visible in one place, not scattered across services.
A checklist you can run this week
# | Action | Owner | What it prevents |
|---|---|---|---|
1 | Measure tok/char per locale for every model you ship | ML eng | Silent cost and truncation surprises |
2 | Classify languages into Tier A/B/C | Product | "Supported" languages nobody tests |
3 | Set context and output budgets per locale | Backend | Clipped answers that look like model failure |
4 | Build a 30–100 case golden set per Tier A language | QA / native speakers | Regressions invisible in English tests |
5 | A/B three prompt-language strategies, judge with a written rubric | ML eng | Inherited folklore about English prompts |
6 | Pass and log locale in every request and response | Backend | Unanswerable incident reviews |
7 | Add per-locale dashboards: format failures, refusals, language consistency, cost/task | Data | Slow, silent quality decay |
8 | Roll out model changes by locale, starting with Tier C | Release | Global regressions from an "upgrade" |
The takeaway
A global product is not an English product with a language menu. Every language you support is a slightly different product: a different token budget, a different failure profile, a different cost line, and a different set of cultural and legal expectations.
You do not need twenty perfect localizations. You need the discipline to treat language as a variable — measured, tiered, tested, and visible in your dashboards. Teams that do this ship in their twentieth language with the same confidence as in their first. Teams that don't keep shipping the same English bug, translated.
