Choose model settings for an LLM API call
Recommends temperature, top-p, max tokens, stop sequences and caching for a specific LLM task, with the reasoning for each.
I am calling a large language model through an API for this task:
{{TASK}}
Constraints (cost, speed, output format): {{CONSTRAINTS}}
Recommend settings for:
- temperature
- top-p (or say to leave it at the default)
- max output tokens
- stop sequences, if any
- which part of the prompt to cache, if the provider supports prompt caching
- whether to use structured output or JSON mode
For each, give the value, one sentence of reasoning, and what symptom would tell me to change it.
Finish with a short checklist for testing these settings on 20 real inputs before going live.Example output
Temperature: 0.2 (extraction needs consistent output). Max tokens: 600 (largest expected JSON is ~400 tokens). Stop sequence: "</result>". Cache: the 3,000-token system prompt, reused on every call.
How to use
Describe the task by its output, such as "classify support tickets into 8 labels" or "write product descriptions". Treat the numbers as a starting point and test with real inputs.
Why it works
Each setting comes with the symptom that means it is wrong, so you can tune with evidence instead of folklore. The testing checklist matters because the right values depend on the model and your data.