Prompt caching explained: how it works and how much it saves

Prompt caching lets an LLM API reuse the work it already did on the start of your prompt, so repeated instructions, documents or tool definitions are processed once and then billed at a fraction of the normal input price. Every major provider now offers it: OpenAI, Google and DeepSeek turn it on by default, and Anthropic needs a cache_control setting, which can be added in one line. This guide explains how prompt caching works, compares the rules and prices of the four providers as documented on October 6, 2026 (Anthropic's re-checked on October 26, 2026), and shows how to structure a prompt to get cache hits.

What is prompt caching?

When a model reads your prompt, it builds intermediate results called key-value (KV) states, which let it refer back to earlier tokens while it reads the rest and writes the answer. OpenAI explains that prompt caching saves that state for a reusable prefix, "the unchanged tokens at the beginning of a prompt", and that the cache "stores key-value tensors, not the tokens themselves". When a later request starts with the same prefix and finds a matching entry, the model skips that work and processes only the new part.

Two consequences follow from how it works:

• Only the beginning of the prompt can be cached. The prefix must match exactly: Anthropic says cache hits require "100% identical prompt segments", and OpenAI says reuse requires the entire rendered prefix to match. Anything that changes early in the prompt, such as a timestamp or a reordered tool definition, makes everything after it miss.
• Caching changes the price and the speed, not the answer. The model sees the same prompt. What you save is the cost of re-reading it. For the unit behind these prices, see what a token is.

How do the four providers compare?

Table: prompt caching rules by provider (October 6, 2026)

Read the table with these caveats: older OpenAI models use different retention settings (in_memory or 24h) and different minimums, and the rates for each model are on each vendor's pricing page, summarized in LLM API pricing compared. Anthropic and OpenAI both say caches are isolated between organizations, and Anthropic adds that they are isolated per workspace on its API.

How much can caching save?

OpenAI's own arithmetic shows the shape of it. At a 0.1x read rate and a 1.25x write, writing a prefix once and reusing it fully once costs 1.35 times its ordinary input cost, against 2 times for processing it twice without caching. Across ten requests, one write and nine full reads cost 2.15 times, against 10 times without caching.

Our cost model gives concrete monthly figures. These are our calculations from the providers' published prices; they include cache reads but not cache writes or storage, and they apply no tokenizer adjustment.

Table: monthly cost with and without caching (our calculation)

The first workload saves less (38% at 80% cached) because output tokens, which caching does not touch, are half of its bill. The second, a long document reread on every request, is where caching matters most: the bill falls by about 81%. The two columns match because both models read from cache at 0.05x the input price; on a model that reads at 0.1x, such as Claude Haiku 4.5, the same share saves less. Run your own numbers in the LLM API cost calculator.

How to structure a prompt to get cache hits

Because only an identical prefix is reused, the layout of the prompt decides your hit rate:

1. Put stable content first. System instructions, tool definitions, examples and reference documents go at the start. The question, the user's name, the date and anything that changes per request go last.
2. Do not change the early part. Reordering tools, editing a tool description, toggling a feature or adding a timestamp at the top invalidates everything after it. Anthropic's cache follows the order tools, then system, then messages, and a change at one level invalidates that level and every later one.
3. Place the breakpoint at the end of the static prefix. With Anthropic's explicit mode, put cache_control on the last block that is identical across the requests that should share a cache, not on the block that varies.
4. Keep requests close together. Caches expire: 5 minutes at Anthropic by default (refreshed on each use), at least 30 minutes at OpenAI on current models. Sending similar requests within that window keeps the entry alive.
5. Mind the minimum length. A prompt shorter than the minimum is processed without caching and no error is returned. If yours falls just short, Anthropic notes that expanding the cached content to reach the threshold is often worthwhile.
6. Measure it. Anthropic reports cache_creation_input_tokens and cache_read_input_tokens, OpenAI reports cached_tokens and cache_write_tokens in its usage details, and Google reports cached tokens in its response. If the cached count is zero, the prefix did not match or was too short.

What about privacy?

Caching stores model state, so read the retention terms. OpenAI says prompt caching may store encrypted key-value tensors in GPU-local storage, that caches are not shared across organizations and that, to prevent users from probing whether a prompt was cached, you can use separate cache keys per customer. Anthropic documents isolation between organizations and workspaces and points to its data retention page for how zero data retention applies to this feature. If you handle personal data, check the provider's terms for how long cached state is kept; our guide to the Regulation (EU) 2024/1689 covers other rules that may apply when you deploy language models in the EU.

When is caching not worth it?

When your prompts are short (below the minimum), when every request is different at the start, or when requests are too far apart for the entry to survive. Caching also does nothing for output tokens, which usually cost several times more than input. For an agent that loops, the opposite holds: each step repeats the same long prefix, which is why agents benefit so much, and why their costs are worth modeling before you build, as in our agents for business example.