---
title: "Prompt caching: definition and example – Carl-Uwe Würfel"
description: "Prompt caching explained: how AI vendors store the unchanged start of a prompt, what that does for cost and wait time, and where it doesn't help."
url: https://carl-wuerfel.de/en/glossary/prompt-caching
author: Carl-Uwe Würfel
---

1. [Home](https://carl-wuerfel.de/en)
  2. [Knowledge](https://carl-wuerfel.de/en/knowledge)
  3. [Glossary](https://carl-wuerfel.de/en/glossary)
  4. Prompt caching

# Prompt caching

Prompt caching is an AI API feature in which the vendor briefly stores the computed state of an unchanged prompt prefix and reuses it for later requests.

## On this page

  1. Only a character-for-character match counts
  2. Each vendor handles it differently
  3. Pricing multipliers are listed per model
  4. The cache lasts minutes to hours
  5. A shared cache can reveal what others asked
  6. Where the term stops working
  7. Frequently asked questions
  8. Sources

The cache only kicks in when the start of the prompt stays the same, character for character.

_Figure: Three requests to a machinery manufacturer 's maintenance assistant shown as bars, each read left to right in prompt order: fixed instructions, maintenance manual, question. On the first request, the prefix is stored. On the next request within the lifetime, the instructions and manual come from the cache and only the new question is computed. If the date sits at the top of the system prompt, every request differs from that point on and everything after it is recomputed. The cache holds intermediate values, not answers._

Many AI applications send the same material with every request: long instructions, descriptions of the available tools, a manual or the conversation so far. Only the new question at the end changes. Without a cache, the model processes all of it every time. With prompt caching, the vendor keeps the intermediate results for that prefix for a while. When a request arrives with exactly the same prefix, the model only computes the new part.

No answers are stored. For each [token](https://carl-wuerfel.de/en/glossary/token), the model computes intermediate values called keys and values (the KV cache) that it draws on as it continues. Those values for the prefix get reused. According to Anthropic, the response is identical to one produced without the cache. Researchers described the principle in 2023 under the name Prompt Cache.

The effect shows up on the invoice and in wait times. At the major vendors, cache reads cost a fraction of regular input tokens, and with long prompts the first part of the answer arrives sooner. Whether an application benefits depends on how its prompts are built.

Example

A machinery manufacturer runs an internal assistant for service technicians' questions about maintenance manuals. The [system prompt](https://carl-wuerfel.de/en/glossary/system-prompt), tool descriptions and the manual for the relevant product line sit at the front and stay the same; the technician's question comes last. The date used to sit in the first line of the system prompt; the application now places it at the end. Since then, usage data shows the fixed prefix as cache reads, as long as the next question arrives within the lifetime.

Chapter 1 of 6

## Only a character-for-character match counts.

The cache only kicks in when the start of the prompt stays the same, character for character. Anthropic requires 100 percent identical segments for a hit; similar text isn't enough. Everything after the first difference is recomputed.

That leads to the rule for building prompts: stable content first, changing content last. Tool descriptions and fixed instructions go first, then documents many requests need, then the question. Anthropic uses the order tools, system prompt, messages, so a change to the tools invalidates everything after them.

A common mistake: if the application writes today's date or a session ID at the top of the system prompt, everything from that line on is recomputed on every request. The same happens when tools are passed in a different order on each request. The [Model Context Protocol](https://carl-wuerfel.de/en/glossary/mcp) specification therefore asks servers for a fixed tool order, explicitly to improve prompt cache hit rates.

Chapter 2 of 6

## Each vendor handles it differently.

At OpenAI and Google, prompt caching is on by default for supported models. At Anthropic, the application sets a marker called cache_control, either once for the whole request or at up to four points in the prompt. Since GPT-5.6, OpenAI also allows custom breakpoints. Google also offers explicit caching for Gemini 2.5 and 3, where the application manages the cache, but not for Gemini 4 Argon and later.

Every vendor sets a minimum length. At Anthropic it ranges from 512 to 4,096 tokens depending on the model, at OpenAI it's 1,024 visible input tokens from GPT-5.6 on, and at Google it's 2,048 or 4,096 tokens depending on the model. Anthropic skips caching for shorter prefixes without returning an error.

Each response's usage data shows whether the cache hit. Anthropic reports cache writes and cache reads separately, OpenAI reports reads as cached_tokens. Self-hosted servers such as vLLM offer the same technique as automatic prefix caching.

Chapter 3 of 6

## Pricing multipliers are listed per model.

As of November 9, 2026, Anthropic lists these multipliers on the regular input price: writing to the cache costs 1.25 times with a five-minute lifetime and twice the price with one hour. Reading costs 0.1 times, or 0.05 or 0.025 times on some newer models.

For GPT-5.6 and later, OpenAI charges 1.25 times for writes with a 30-minute minimum lifetime, 2.5 times with twelve hours, and usually 0.1 times for reads. For older models there's no write surcharge, and the read discount depends on the model. Google doesn't guarantee savings for automatic caching; explicit caching adds a charge for storage time.

OpenAI's own example: if a prefix is written once and fully read nine times, the total comes to 2.15 times the regular input price instead of ten times. If it's used only once, all that's left is the surcharge.

Chapter 4 of 6

## The cache lasts minutes to hours.

An entry expires when nobody uses it. At Anthropic it lasts five minutes by default, and each use refreshes it at no extra cost; one hour is available at the higher write price. OpenAI sets a minimum lifetime of 30 minutes for newer models, or twelve hours; that governs read pricing but doesn't guarantee a hit. With Google's explicit caching, the application sets the lifetime; if it doesn't, one hour applies.

Caching pays off when the same prefix is needed several times within that window. An assistant with many requests per hour hits the cache often. A report that runs once a day pays only the surcharge. Anthropic also lets applications warm the cache before the first real request arrives. In conversations and agent runs, appending new messages instead of rewriting earlier ones lets each request reuse the history so far.

Chapter 5 of 6

## A shared cache can reveal what others asked.

Cached prefixes are processed faster, which is measurable from outside. A research team showed in 2025 that when several users share a cache, an attacker can tell from fast response times whether a given text was requested shortly before. At the time, the team found signs of cache sharing across users at seven API vendors.

Today, Anthropic and OpenAI state that caches aren't shared between organizations; on the Claude API, Anthropic also separates workspaces within an organization. According to Anthropic, the intermediate values are held in memory only and deleted promptly after they expire.

Within an application that serves many customers, OpenAI recommends separate cache keys per customer or user, so nobody can work out what others asked by probing. Personal data doesn't belong in a prefix that several users share.

Chapter 6 of 6

## Where the term stops working

Prompt caching saves compute for the start of a prompt; it makes the answer neither better nor worse, and an unnecessarily long prompt stays expensive. Response caches and semantic caching store finished answers in your own application. The [context window](https://carl-wuerfel.de/en/glossary/context-window), meaning how much text a model can process at once, isn't affected by the cache.

## Frequently asked questions

What is prompt caching?

Prompt caching is a vendor feature that briefly stores the computed state of an unchanged prompt prefix. Later requests with the same prefix are processed faster and at lower cost, and the answer is still generated fresh each time.

Does prompt caching store the answers?

No, it only stores intermediate values for the start of the prompt. A response cache in your own application stores finished answers to identical questions, and semantic caching does the same for similar ones. Both skip the model call but can serve outdated answers.

How many tokens do you need for prompt caching?

Each vendor sets the minimum per model. As of November 2026, it's between 512 and 4,096 tokens at Anthropic, 1,024 at OpenAI from GPT-5.6 on, and 2,048 or 4,096 tokens at Google.

How long does a prompt stay in the cache?

Minutes to hours, depending on the vendor. Anthropic defaults to five minutes, newer OpenAI models to at least 30 minutes, and both offer longer lifetimes at a higher write price.

How much can prompt caching save?

It depends on how much of the prompt stays the same and on the hit rate, so there's no fixed figure. At Anthropic and on newer OpenAI models, cache reads usually cost a tenth of the regular input price, while writes carry a surcharge.

What is the difference between implicit and explicit prompt caching?

With implicit caching the vendor caches automatically; with explicit caching the application decides where the cached prefix ends. Explicit breakpoints let you cache parts that change at different rates separately. Google guarantees savings only for explicit caching.

Is prompt caching worth it for short prompts?

Usually not. Below the minimum length it doesn't apply at all, and with infrequent calls the entry expires before the next request. Where writes carry a surcharge, the application then pays more than without the cache.

## Further reading

  * [**AI glossary**](https://carl-wuerfel.de/en/glossary)
  * [**Tracking AI costs per call**](https://carl-wuerfel.de/en/notes/controlling-ai-api-costs)
  * [**Token**](https://carl-wuerfel.de/en/glossary/token)
  * [**Context window**](https://carl-wuerfel.de/en/glossary/context-window)
  * [**LLM gateway**](https://carl-wuerfel.de/en/glossary/llm-gateway)
  * [ Term**System Prompt**](https://carl-wuerfel.de/en/glossary/system-prompt)
  * [ Term**AI inference**](https://carl-wuerfel.de/en/glossary/inferenz)

## Sources

  1. [Anthropic: Prompt caching, documentation, as of November 9, 2026](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)
  2. [OpenAI: Prompt caching, API documentation, as of November 9, 2026](https://developers.openai.com/api/docs/guides/prompt-caching)
  3. [Google: Context caching, Gemini API documentation, as of November 9, 2026](https://aistudio.google.com/docs/caching)
  4. [Model Context Protocol: Specification 2026-07-28, Server Features, Tools](https://modelcontextprotocol.io/specification/2026-07-28/server/tools)
  5. [vLLM: Automatic Prefix Caching, documentation, as of November 2026](https://docs.vllm.ai/en/latest/features/automatic_prefix_caching/)
  6. [Gim et al.: Prompt Cache: Modular Attention Reuse for Low-Latency Inference, arXiv:2311.04934, November 2023](https://arxiv.org/abs/2311.04934)
  7. [Gu et al.: Auditing Prompt Caching in Language Model APIs, arXiv:2502.07776, February 2025](https://arxiv.org/abs/2502.07776)
