Max tokens

Max tokens is an API setting that limits the length of a model's response. When the model reaches the limit, it stops, even mid-sentence. It applies to output only; the input has its own limit set by the context window. In many APIs the parameter is literally called max_tokens or max_output_tokens.

The setting exists to control cost and latency. Output tokens are usually the most expensive part of an API call and take the longest to generate, so a sensible cap prevents runaway responses. It is also a safety net when a prompt goes wrong and the model starts repeating itself.

Max tokens is a ceiling, not a target. Setting it to 500 does not make the model write 500 tokens; it writes as much as it decides to, up to 500. To control length, describe it in the prompt ("under 100 words", "3 bullet points") and use max tokens as a backstop. For reasoning models, the limit may also need to cover hidden thinking tokens, so set it generously.

Example

An app generating product descriptions sets max tokens to 300, while the prompt asks for 80 to 120 words (about 110 to 160 tokens). Normal responses finish well under the cap, and a rare runaway response is cut off before it costs much.

Common mistake

Setting max tokens too low and getting truncated output, especially JSON that is missing its closing brackets and fails to parse. Check the API's stop reason field: if it says the response stopped because of the length limit, raise the cap or shorten what you ask for.