Prompt caching
Reusing the model's processed form of a repeated prefix so it is not paid for or recomputed on every request.
When many requests share a long identical prefix — a system prompt, a tool catalogue, a document — the provider can store the processed form of that prefix and reuse it. Subsequent requests that match the prefix exactly are cheaper and faster. The match is on an exact prefix, so anything that changes early in the prompt invalidates everything after it.
Key facts
- Cache matching is prefix-based and exact: changing one character near the start of the prompt invalidates the whole cached prefix.
- Putting variable content — a timestamp, a user name, a request id — at the top of a prompt defeats caching entirely.
- Cached input is billed at a reduced rate rather than free, and writing to the cache can cost more than an ordinary request.
- Caches expire after a period of inactivity, so a low-traffic agent may never hit a warm cache.
Frequently asked questions
Why is my prompt cache never hitting?
Almost always because something variable sits early in the prompt. Move timestamps, session identifiers and per-request context to the end, and keep the stable system prompt and tool definitions at the very beginning.
Does caching change the model output?
No. It changes how the prefix is processed and billed, not what the model produces. An identical request should return the same distribution of outputs whether or not it hit the cache.
Machine-readable copy of this page:
/glossary/prompt-caching.md