Context window
Also called context length, context size
The maximum amount of text, measured in tokens, that an AI model can read and write in one go.
The context window is the limit on how much a model can consider at once. It includes everything: the system prompt, the conversation so far, any documents or search results added, and the reply being written. It is measured in tokens, roughly three quarters of an English word each. Current models range from tens of thousands to over a million tokens.
When a conversation or document exceeds the window, something has to go. Chat apps typically drop or summarise older messages, which is why a model may forget details from early in a long chat. APIs usually return an error if the input is too long.
A bigger window is not the same as perfect memory. Models can be worse at using information buried in the middle of a very long input than information near the start or end, and long inputs cost more and run slower. For large collections of documents, retrieval-augmented generation, which fetches only the relevant parts, is often better than pasting everything.
Example
A 300-page contract is about 120,000 tokens. It fits in a model with a 200,000-token window, leaving room for questions and answers. In a model with a 32,000-token window, you would need to split it into sections or use retrieval.
Common mistake
Pasting a huge document and assuming the model read every line carefully. Even within the window, details can be missed. Point the model to the sections that matter, ask for quotes to back up claims, and check anything important against the source.