Prompt caching
Also called context caching, cached prompts, kv caching
A feature that reuses the processed start of a prompt across requests, making repeated calls cheaper and faster.
Prompt caching lets an AI provider store the processed form of a prompt's opening section, such as a long system prompt, a large document or a set of examples, and reuse it when the next request starts with the same content. Instead of processing those tokens again, the model picks up from the cached state. Cached input tokens are typically billed at a fraction of the normal price, and responses start faster.
Caching works on prefixes: the matching part must be at the start of the prompt and identical, character for character. That shapes how you build prompts. Put stable content (instructions, reference documents, tool definitions, examples) first and variable content (the user's question, today's data) last. Some providers cache automatically; others need you to mark where the cacheable section ends. Caches expire after a period of inactivity, often minutes.
The savings can be large in applications that reuse the same context many times: chatbots with long system prompts, question answering over the same document, agents that resend tool definitions on every step, and multi-turn conversations.
Example
A legal assistant sends a 50,000-token contract with every question a lawyer asks. With caching, the first question pays full price for the contract; the next twenty questions within the cache window pay a small fraction for those tokens and get answers noticeably faster.
Common mistake
Putting changing content, such as a timestamp or the user's name, at the very start of the prompt. Any difference in the prefix breaks the cache match. Keep the beginning of the prompt stable and move anything that changes to the end.