blog / reduce-openai-api-costs
How to reduce OpenAI API costs: a practical checklist
OpenAI caches automatically, charges no write premium, and discounts cached input up to 90% on the GPT-5.x family. So why do so many OpenAI bills look untouched? Because automatic caching still only rewards prefix-stable traffic, and most production prompts are not prefix-stable. The levers, in order:
1. Make your prefixes byte-stable
The cache matches on exact prefixes. Injected timestamps, request IDs, randomized few-shot ordering, or per-user data early in the messages array mean the discount never triggers. Put stable content (system prompt, instructions, schemas, examples) first and dynamic content last, and make serialization deterministic: the same logical prompt must produce the same bytes.
2. Use prompt_cache_key for cache routing
On workloads with many similar-but-distinct prefixes (multi-tenant apps, per-project system prompts), prompt_cache_key helps requests with the same key land on the same cache, improving hit consistency. It is one parameter; most codebases have never set it.
3. Know your input/output split
Caching discounts input tokens only. If reasoning and output tokens are half your spend, the theoretical ceiling of any caching work is the other half. Compute the split from your usage data before investing effort, and size expectations accordingly.
4. Batch the async work
The Batch API runs at roughly half price for anything that tolerates delayed completion: evaluations, embeddings backfills, content pipelines, scheduled jobs.
5. Right-size models per route
The gap between flagship and mini-tier pricing is an order of magnitude. Classify request difficulty and route accordingly, with quality guardrails and an override on critical routes.
6. Verify the discount actually landed
The response usage block reports cached token counts. Compute your realized discount against list price over a week of traffic (method here). Teams routinely discover their "automatic" hit rate is a small fraction of what their traffic could support, and the fix is almost always item 1 on this list.