LLM temperature: what it does and why the newest models ignore it

LLM temperature is the setting that controls how random a language model is when it picks each next token: lower values make it stick to the most likely words, higher values make it more varied. For years it was the first knob developers turned. On October 23, 2026 that has changed: OpenAI's newest models reject it while they reason, Anthropic's API version of October 21, 2026 returns an error if you send it, and Google strongly recommends leaving Gemini 3 at the default. This guide explains what temperature does, what top_p and top_k add, which APIs still accept it and what to use instead, based on each vendor's documentation read on October 23, 2026.

How temperature works

A language model writes one token at a time. At each step it gives every possible next token a probability, and a sampling step picks one. Temperature reshapes those probabilities before the pick. Google's documentation puts it simply: lower temperatures suit prompts that need a more deterministic, less open-ended answer, higher temperatures can give more diverse or creative results, and a temperature of 0 always selects the highest-probability token. OpenAI's API reference gives the same picture with numbers: values like 0.8 make the output more random, values like 0.2 make it more focused.

Two related settings filter the candidates before temperature does its work:

• top_k keeps only the k most likely tokens. A top_k of 1 always picks the single most likely token, which Google calls greedy decoding.
• top_p keeps the most likely tokens until their probabilities add up to p. Google's example: if tokens A, B and C have probabilities 0.3, 0.2 and 0.1 and top_p is 0.5, the model chooses between A and B and never C. Gemini's default top_p is 0.95.

OpenAI recommends changing temperature or top_p, but not both.

Which APIs still accept it

Table: temperature by API (official documentation, checked October 23, 2026)

The pattern is clear. Temperature still works in local tools and on models that answer without reasoning, but the vendors have moved away from it on their newest reasoning models, where the model spends tokens thinking before it answers. Google's Gemini 3 guide warns specifically that setting temperature below 1.0 can lead to looping or degraded performance in math and reasoning tasks, and recommends removing the parameter from code that sets it low.

Temperature 0 was never a guarantee

Many developers set temperature to 0 to get the same answer every time. Anthropic's migration guide for Claude Opus 5.5 is blunt about it: temperature 0 never guaranteed identical outputs, even on earlier models. A low temperature makes outputs more consistent, not identical.

If you need repeatable results, use the tool built for it. Locally, Ollama's seed parameter makes the model generate the same text for the same prompt. In the cloud, if what you need is a consistent format rather than identical wording, structured outputs force the answer to follow a schema regardless of temperature.

What to use instead

With temperature gone or fixed on the newest models, the vendors point to other levers:

• Prompting. Anthropic's guidance after removing sampling parameters is to use prompting to guide the model. Say how creative or how literal you want the answer to be, give an example of the tone and set limits on length. Our prompt engineering basics show how.
• Reasoning effort. OpenAI's reasoning effort and Anthropic's effort parameter control how much the model thinks before answering, which changes quality, speed and cost.
• Several candidates. For brainstorming, ask for five different options in one answer instead of raising the temperature.
• Validation. For extraction and classification, check the answer in your code. A lower temperature never stopped a model from hallucinating a confident wrong value.

Practical settings where you still can

On models and tools that still accept it, our rule of thumb is:

• 0 to 0.3 for extraction, classification and code that must follow rules.
• 0.7 to 1 for general chat and writing, close to most defaults.
• Above 1 only for brainstorming, and check the output, because it drifts quickly.

Change one setting at a time and test it on the same set of prompts. And before you tune temperature on a new model, check its documentation: on the newest models from OpenAI, Anthropic and Google, the best setting is usually the default.