The verbosity trap: when the cheaper model costs 29% more

Every LLM cost guide says the same thing: use a cheaper model for simple tasks. We say it too. So we measured it properly, and the result changed how our product gives advice.

The experiment

Five task classes, eight representative requests each, replayed through an expensive/cheap model pair. Every output judged by a strict LLM judge against the premium model's answer. Every dollar counted from real token usage, not price-sheet arithmetic.

| Task class | Blanket downgrade | Escalation routing | Verdict | |-----------|-------------------|--------------------|---------| | Summarization | 100% quality / +30.6% saving | same | Downgrade freely | | Classification | 87.5% quality / +34% | 100% quality / +22% | Route | | Extraction | 62.5% quality / +35% | 100% / −5.9% | Keep premium | | Code generation | 87.5% / −11.5% saving | 100% / −33% | Keep premium | | Creative/nuanced | 87.5% / −29.4% saving | 100% / −34% | Keep premium |

Why the cheap model can cost more

Output tokens are typically 4-5x the price of input tokens, and on code generation and creative tasks, the cheaper model wrote longer outputs. Longer code, wordier prose. The per-token discount was smaller than the token-count increase. The "saving" was negative: 11-29% more expensive than the premium model, before you even ask about quality.

Price-sheet math cannot see this, because it assumes output length stays constant across models. It does not.

Where the quality actually breaks

Extraction surprised us most: 62.5% quality on a blanket downgrade. The cheap tier mangles structured-output edge cases: null handling, date formats, field omissions. And the failures cluster on the hard tail: requests with negations ("summarize without spoilers"), tone constraints, format constraints. The routine 90% is handled identically, which is exactly what makes the failures dangerous — you will not see them in a casual spot-check.

What we changed

Only one of five task classes is a free downgrade. So our recommendation engine now refuses to recommend downgrades our own eval falsified: extraction, code-gen and creative call sites never get "use a cheaper model" advice from erabot, classification gets cheap-first routing with escalation (100% quality at +22%), and every recommendation carries its measured verdict and the eval date.

If your cost tool recommends model downgrades without measuring your task classes, ask it how it knows. Ours knows because it replayed the traffic. Methodology and the full ladder: /eval.