honest LLM cost engineering.
the mechanics of provider prompt caching, the math that decides whether it saves or costs you money, and how to measure the result without lying to yourself.
01 deep dives
Realized Discount Rate: why your 90% prompt-caching discount is not what you actually get
Providers advertise up to 90% off cached input tokens. Most workloads realize far less. RDR measures the gap between the headline discount and your effective rate, and it is the number that decides whether caching is saving you money.
pillar · 2026-07-06Anthropic cache write premiums: the break-even math nobody runs
Claude cache writes cost 1.25x (5-minute TTL) or 2x (1-hour TTL) the input rate. A cache written and never re-read within its TTL raises your bill. Here is the exact break-even derivation and why reuse-aware write decisions matter.
pillar · 2026-07-06Automatic prompt caching in 2026: what OpenAI, Anthropic, and Gemini do for you, and what they cannot
Caching is now automatic or one flag away on every major provider. What automatic modes still do not do: stable/dynamic segmentation, reuse-aware write decisions, TTL policy, and honest measurement. A provider-by-provider guide.
02 practical guides
How to reduce Anthropic API costs: a practical checklist
Six concrete levers for cutting a Claude API bill: prompt structure and the relocation trick, breakpoint placement, TTL choice, write discipline on low-reuse routes, batch and model-tier selection, and measuring the result honestly.
guide · 2026-07-06How to reduce OpenAI API costs: a practical checklist
OpenAI caches automatically, so why is your bill flat? Prefix stability, prompt_cache_key routing, the input/output split, batch jobs, and model selection: what actually moves an OpenAI invoice in 2026.
guide · 2026-07-06Prompt caching for agent loops: why agents miss the cache and how to fix it
Agents are the most cacheable workload in existence and the most commonly miscached. Tool schemas, working memory, and conversation prefixes each break the cache in a different way. How to structure an agent loop for real hit rates.