LLM cost reduction
How to reduce LLM costs without hurting quality
Short answer
The biggest LLM savings come from not calling the large model at all when you don't need to. Filter spam and irrelevant inputs first, route simple requests to smaller models, and replace generate-then-parse classification with a cheap decision model. Then trim context, cache repeated prompts and batch offline work.
LLM bills usually grow faster than usage, because every new feature adds another prompt to every request. Teams then attack the price per token, when the larger lever is the number of tokens and calls you send to expensive models in the first place.
The techniques below are ordered roughly by impact for a typical product. Measure before and after each change: cost per resolved task is a better metric than cost per call.
Get Early Access to ClassifierHub: 2× credits in your first paid month.
1. Filter before you generate
A surprising share of inputs don't need an answer from a large model: spam, duplicates, out-of-scope questions, auto-replies, empty form submissions. A yes/no gate in front of your pipeline removes them for a tiny fraction of the cost of a generation.
2. Replace generate-then-parse classification
If your code asks an LLM to answer with a category and then parses the text, you're paying for generation to get a label. A decision model returns the label and probabilities directly, charges no output tokens at the model level and needs no retry logic for malformed JSON. Classification, scoring, yes/no checks and routing all fit this pattern.
curl https://classifierhub.com/v1/classify \
-H "Authorization: Bearer $CLASSIFIERHUB_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input": "I was charged twice for my subscription this month.",
"instructions": "Which team should handle this message?",
"options": {
"billing": "Payments, invoices, refunds, charges",
"technical": "Bugs, errors, outages",
"sales": "Buying, upgrading, pricing questions",
"other": "Anything else"
}
}'
# -> { "label": "billing", "confidence": 0.93,
# "probabilities": { "billing": 0.93, "technical": 0.03, "sales": 0.02, "other": 0.02 },
# "usage": { "credits": 1, "latency_ms": 420 } }The ClassifierHub API opens to Early Access members first.
3. Route to the smallest model that works
Classify each request by difficulty and send simple ones to a small model. Keep a conservative fallback to the strong model when the router is unsure. See the LLM routing guide for a worked example.
4. More techniques
Once the big levers are in place:
- Trim context: send the latest message instead of the whole thread, and retrieve fewer, better chunks in RAG.
- Cache: use provider prompt caching for long, stable system prompts, and cache full responses for repeated questions.
- Batch offline work: most providers discount asynchronous batch jobs, and nightly enrichment rarely needs real-time answers.
- Cap output: set max tokens and ask for concise formats. Output tokens usually cost several times more than input tokens.
- Stop agent loops early: add a check that decides whether the task is done instead of letting agents run to their step limit.
Where ClassifierHub fits
ClassifierHub is the decision layer for steps 1-3: gates, classification, scoring and routing, billed in predictable credits (one decision is 1 credit, even with several questions about the same input). Your large models then only see the requests that need them.
Frequently asked questions
Related guides
Last updated . ClassifierHub is an independent product built on top of the Jev decision model, accessed through OpenRouter. It is not affiliated with or endorsed by TypeSafe or OpenRouter.