Prompt caching
Prompt caching is an AI API feature in which the vendor briefly stores the computed state of an unchanged prompt prefix and reuses it for later requests.
The cache only kicks in when the start of the prompt stays the same, character for character.
Many AI applications send the same material with every request: long instructions, descriptions of the available tools, a manual or the conversation so far. Only the new question at the end changes. Without a cache, the model processes all of it every time. With prompt caching, the vendor keeps the intermediate results for that prefix for a while. When a request arrives with exactly the same prefix, the model only computes the new part.
No answers are stored. For each token, the model computes intermediate values called keys and values (the KV cache) that it draws on as it continues. Those values for the prefix get reused. According to Anthropic, the response is identical to one produced without the cache. Researchers described the principle in 2023 under the name Prompt Cache.
The effect shows up on the invoice and in wait times. At the major vendors, cache reads cost a fraction of regular input tokens, and with long prompts the first part of the answer arrives sooner. Whether an application benefits depends on how its prompts are built.
Chapter 1 of 6
Only a character-for-character match counts.
The cache only kicks in when the start of the prompt stays the same, character for character. Anthropic requires 100 percent identical segments for a hit; similar text isn't enough. Everything after the first difference is recomputed.
That leads to the rule for building prompts: stable content first, changing content last. Tool descriptions and fixed instructions go first, then documents many requests need, then the question. Anthropic uses the order tools, system prompt, messages, so a change to the tools invalidates everything after them.
A common mistake: if the application writes today's date or a session ID at the top of the system prompt, everything from that line on is recomputed on every request. The same happens when tools are passed in a different order on each request. The Model Context Protocol specification therefore asks servers for a fixed tool order, explicitly to improve prompt cache hit rates.
Chapter 2 of 6
Each vendor handles it differently.
At OpenAI and Google, prompt caching is on by default for supported models. At Anthropic, the application sets a marker called cache_control, either once for the whole request or at up to four points in the prompt. Since GPT-5.6, OpenAI also allows custom breakpoints. Google also offers explicit caching for Gemini 2.5 and 3, where the application manages the cache, but not for Gemini 4 Argon and later.
Every vendor sets a minimum length. At Anthropic it ranges from 512 to 4,096 tokens depending on the model, at OpenAI it's 1,024 visible input tokens from GPT-5.6 on, and at Google it's 2,048 or 4,096 tokens depending on the model. Anthropic skips caching for shorter prefixes without returning an error.
Each response's usage data shows whether the cache hit. Anthropic reports cache writes and cache reads separately, OpenAI reports reads as cached_tokens. Self-hosted servers such as vLLM offer the same technique as automatic prefix caching.
Chapter 3 of 6
Pricing multipliers are listed per model.
As of November 9, 2026, Anthropic lists these multipliers on the regular input price: writing to the cache costs 1.25 times with a five-minute lifetime and twice the price with one hour. Reading costs 0.1 times, or 0.05 or 0.025 times on some newer models.
For GPT-5.6 and later, OpenAI charges 1.25 times for writes with a 30-minute minimum lifetime, 2.5 times with twelve hours, and usually 0.1 times for reads. For older models there's no write surcharge, and the read discount depends on the model. Google doesn't guarantee savings for automatic caching; explicit caching adds a charge for storage time.
OpenAI's own example: if a prefix is written once and fully read nine times, the total comes to 2.15 times the regular input price instead of ten times. If it's used only once, all that's left is the surcharge.
Chapter 4 of 6
The cache lasts minutes to hours.
An entry expires when nobody uses it. At Anthropic it lasts five minutes by default, and each use refreshes it at no extra cost; one hour is available at the higher write price. OpenAI sets a minimum lifetime of 30 minutes for newer models, or twelve hours; that governs read pricing but doesn't guarantee a hit. With Google's explicit caching, the application sets the lifetime; if it doesn't, one hour applies.
Caching pays off when the same prefix is needed several times within that window. An assistant with many requests per hour hits the cache often. A report that runs once a day pays only the surcharge. Anthropic also lets applications warm the cache before the first real request arrives. In conversations and agent runs, appending new messages instead of rewriting earlier ones lets each request reuse the history so far.
Chapter 6 of 6
Where the term stops working
Prompt caching saves compute for the start of a prompt; it makes the answer neither better nor worse, and an unnecessarily long prompt stays expensive. Response caches and semantic caching store finished answers in your own application. The context window, meaning how much text a model can process at once, isn't affected by the cache.
Frequently asked questions
What is prompt caching?
Prompt caching is a vendor feature that briefly stores the computed state of an unchanged prompt prefix. Later requests with the same prefix are processed faster and at lower cost, and the answer is still generated fresh each time.
Does prompt caching store the answers?
No, it only stores intermediate values for the start of the prompt. A response cache in your own application stores finished answers to identical questions, and semantic caching does the same for similar ones. Both skip the model call but can serve outdated answers.
How many tokens do you need for prompt caching?
Each vendor sets the minimum per model. As of November 2026, it's between 512 and 4,096 tokens at Anthropic, 1,024 at OpenAI from GPT-5.6 on, and 2,048 or 4,096 tokens at Google.
How long does a prompt stay in the cache?
Minutes to hours, depending on the vendor. Anthropic defaults to five minutes, newer OpenAI models to at least 30 minutes, and both offer longer lifetimes at a higher write price.
How much can prompt caching save?
It depends on how much of the prompt stays the same and on the hit rate, so there's no fixed figure. At Anthropic and on newer OpenAI models, cache reads usually cost a tenth of the regular input price, while writes carry a surcharge.
What is the difference between implicit and explicit prompt caching?
With implicit caching the vendor caches automatically; with explicit caching the application decides where the cached prefix ends. Explicit breakpoints let you cache parts that change at different rates separately. Google guarantees savings only for explicit caching.
Is prompt caching worth it for short prompts?
Usually not. Below the minimum length it doesn't apply at all, and with infrequent calls the entry expires before the next request. Where writes carry a surcharge, the application then pays more than without the cache.
Sources
- Anthropic: Prompt caching, documentation, as of November 9, 2026
- OpenAI: Prompt caching, API documentation, as of November 9, 2026
- Google: Context caching, Gemini API documentation, as of November 9, 2026
- Model Context Protocol: Specification 2026-07-28, Server Features, Tools
- vLLM: Automatic Prefix Caching, documentation, as of November 2026
- Gim et al.: Prompt Cache: Modular Attention Reuse for Low-Latency Inference, arXiv:2311.04934, November 2023
- Gu et al.: Auditing Prompt Caching in Language Model APIs, arXiv:2502.07776, February 2025