Structured outputs: getting valid JSON from a language model

Structured outputs make a language model return JSON that matches a schema you define, so your code can parse every answer without retries. On October 23, 2026, OpenAI, Anthropic, Google and Ollama all support it, and the three cloud APIs enforce the schema during generation (constrained decoding) rather than just asking nicely in the prompt. This guide explains how structured outputs differ from the older JSON mode, how each API turns them on, which parts of JSON Schema they accept and the limits that still break real projects, based on each vendor's documentation read on October 23, 2026.

JSON mode is not the same thing

JSON mode only promises valid JSON. Structured outputs promise JSON that follows your schema: every required key present, every enum value allowed, every type correct. OpenAI describes structured outputs as the evolution of JSON mode and recommends always using them when possible. JSON mode also has a trap: OpenAI warns that if you forget to tell the model to produce JSON, it may emit an unending stream of whitespace until it hits the token limit.

The difference matters in production. Without a schema, a model can skip a field, invent a value outside your list or wrap the JSON in prose, and every one of those breaks a parser. With constrained decoding, the model can only choose tokens that keep the output valid against the schema. Anthropic describes the result as no more JSON.parse() errors and no retries for schema violations.

How to turn it on in each API

Table: structured outputs by vendor (official documentation, checked October 23, 2026)

For tool calls the same idea applies to the arguments: OpenAI's strict mode in function calling and Anthropic's strict tool use guarantee that the inputs match the tool's schema. Our explainer on function calling covers when the model should call a tool instead of answering in a fixed format. Google's own rule of thumb is short: structured outputs format the final answer, function calling takes an action before the answer.

Locally, Ollama accepts a JSON schema in its format field and recommends also putting the schema in the prompt and lowering the temperature to 0. Its documentation notes that Ollama's cloud does not support structured outputs. If you run models on your own machine, our guide to running an LLM locally covers the setup.

What JSON Schema features are allowed

None of the vendors accepts the full JSON Schema standard. The rules that trip people up most:

• Every field required (OpenAI). All fields must be listed in required. To make one optional, give it a union type with null.
• additionalProperties: false. OpenAI requires it on every object, and Anthropic only accepts false.
• Root object. OpenAI requires the root to be an object, not an anyOf, which is what a Zod discriminated union produces.
• Recursion. OpenAI supports recursive schemas; Anthropic lists recursive schemas as not supported.
• Ignored constraints. Gemini strips minLength, maxLength, minProperties, maxProperties and uniqueItems instead of enforcing them; Anthropic's SDK helpers move unsupported constraints such as minimum into the field description and validate them in your code.

Size limits differ too. OpenAI allows up to 5,000 object properties with up to 10 levels of nesting and up to 1,000 enum values per schema. Anthropic limits each request to 20 strict tools, 24 optional parameters and 16 parameters with union types across all strict schemas, and answers "Schema is too complex for compilation" when the grammar would be too large. Gemini rejects very large or deeply nested schemas with a 400 INVALID_ARGUMENT error.

Latency, caching and cost

The schema has to be compiled the first time. OpenAI says the first request with a new schema has extra latency and later ones do not. Anthropic caches compiled grammars for 24 hours from last use and invalidates the cache when the schema structure changes, but not when only names or descriptions change. Changing the output format on Claude also invalidates the prompt cache for that conversation. There is no separate price: the JSON is billed as ordinary output tokens, and OpenAI lists structured outputs as supported in its Batch API, so the 50% discount of the batch API applies.

What can still go wrong

• Refusals. If the model declines for safety reasons, OpenAI returns a separate refusal field and Claude sets stop_reason: "refusal"; in both cases the output may not match your schema.
• Truncation. If max_tokens runs out, the JSON is incomplete. Raise the limit and retry.
• Wrong but valid. The schema guarantees the shape, not the truth. OpenAI notes that when the input has nothing to do with the schema, the model may fill it with invented values, and Google tells you to validate values in your application. Our guide to model hallucinations explains why.
• Enum casing. Anthropic warns that Claude may return an enum value that differs only in capitalization. Compare enum values case-insensitively.
• Incompatible features. On Claude, structured outputs cannot be combined with citations (400 error) or with prefilling the assistant message.

A setup that holds up

Define the schema once in code with Pydantic or Zod and let the SDK generate the JSON Schema, as OpenAI recommends to avoid the two drifting apart. Keep the schema flat, make fields required wherever a default exists, describe each field in one plain sentence and tell the model what to return when the input does not fit, for example an empty list. Then validate the parsed values in your own code, because a valid shape is only half of a correct answer.