blog / automatic-prompt-caching-2026
Automatic prompt caching in 2026: what OpenAI, Anthropic, and Gemini do for you, and what they cannot
"Just turn on caching" stopped being advice in 2026, because on every major provider it is already on, or one flag away. That is genuinely good news and it also creates a new failure mode: teams assume automatic means optimal, stop looking, and leave the majority of the advertised discount unrealized. Here is the current state, provider by provider, and the precise list of what automatic modes still do not do. Pricing and mechanics change frequently; verify against provider docs before hardcoding anything.
OpenAI
Fully automatic since October 2024, no separate write charge, cached input discounted up to 90% on the GPT-5.x family (materially less on older models). Retention runs from minutes up to 24 hours on the newest models, and the prompt_cache_key parameter lets you improve cache routing for workloads with many similar prefixes. Cache hits also cut time-to-first-token substantially. The catch: hits remain prefix-based, so none of this helps a prompt whose first differing byte arrives early.
Anthropic
Two modes since early 2026. Automatic: a single top-level cache_control field, and the API places and advances breakpoints itself. Explicit: up to 4 per-block breakpoints, which remains the most powerful surface for RAG and tool-heavy agents because you can cache a mid-conversation document block separately from the system prompt. Distinctive economics: reads at 0.1x input, but writes at 1.25x (5-minute TTL) or 2x (1-hour TTL), a minimum cacheable block of about 1,024 tokens, and per-organization (and since February 2026 per-workspace) cache isolation. The write premiums mean automatic caching can raise a low-reuse workload's bill, which no other major provider's pricing does.
Gemini
Implicit caching on Gemini 2.5+ at roughly 90% off cached tokens (older writeups still cite 75%; verify per model), plus an explicit caching API with storage fees for pinned contexts. Bedrock, for completeness, enables prompt caching by default on InvokeModel with automatic prefix checking.
What automatic modes do not do
ProjectDiscovery, after evaluating Anthropic's automatic mode for their agent platform, chose manual control, and their stated reason is the whole story: automatic caching has no awareness of which parts of your prompt are stable versus dynamic. Concretely, no automatic mode today will:
- Restructure your prompt. If per-user variables or working memory sit before your stable instructions, every provider's cache misses on the content that matters most. Relocating dynamic content to the tail is the single highest-leverage fix in the field (7% to 74% hit rate in ProjectDiscovery's published case), and it is yours to do; the provider will not rewrite your request.
- Decline to cache. On Anthropic's write premiums, caching a route whose traffic never recurs within TTL is a net loss. Automatic modes see one request at a time and cannot reason about expected reuse.
- Choose TTLs per traffic pattern. Whether your recurrence supports 5-minute or 1-hour economics is a per-route measurement, not a global default.
- Stabilize agent scaffolding. Tool schemas that re-serialize differently per request, and conversation prefixes that mutate mid-history, defeat prefix caching in ways only visible across requests.
- Tell you what you realized. No provider dashboard reports your effective discount against list price, and none reports what a better-structured version of your own workload would realize. You cannot manage a gap you cannot see.
The honest summary
Enabling caching is free and mostly done for you; that part of the problem is solved and nobody should pay for it. Realizing the advertised discount on a real workload is a structure and measurement problem, and it is exactly as unsolved as your hit rate says it is. Measure your realized rate first; every decision after that gets easier.