Batch APIs: how to pay half for model requests that can wait

A batch API lets you send a large set of model requests in one job, wait for the results instead of getting them in real time, and pay about half the normal price. On October 23, 2026, OpenAI, Anthropic, Google and Mistral all document a 50% discount for batch jobs, in exchange for asynchronous processing that can take up to 24 hours. This guide explains how batch processing works in each API, what it costs, what the limits are and when it is the wrong tool, based on each vendor's documentation read on October 23, 2026.

How a batch job works

The flow is almost identical everywhere. You prepare a list of ordinary requests, each with your own identifier, usually as a JSONL file where every line is one request. You submit the batch, the provider queues it and processes it when it has capacity, and you poll the job (or wait for a notification) until it ends. Then you download the results and match each one to its request through your identifier.

That identifier matters more than it looks. OpenAI warns that the output order may not match the input order, so you have to join results on custom_id, and Anthropic requires a custom_id of 1 to 64 characters for the same reason. Nothing in a batch is a conversation: each line is an independent request with its full prompt, so if you want the discount on a long shared system prompt, combine batches with prompt caching.

Batch processing in four APIs

Table: batch processing by vendor (official documentation, checked October 23, 2026)

Two differences change the design of a pipeline. Anthropic lets one batch mix request types and models, while OpenAI and Mistral want a single model per batch. And the retention periods differ, so download results as soon as a job ends instead of treating the provider as storage.

What it costs

Batch pricing is the standard per-token price cut in half. Anthropic publishes the batch rates in its documentation: Claude Sonnet 5.5 costs $1 per million input tokens and $5 per million output tokens in batch mode, against $2 and $10 at standard rates, and Claude Opus 5.5 costs $2 and $10 in batch. Claude Haiku 5.5 is listed at $0.05 per million input tokens and $0.25 per million output tokens in batch for prompts up to 100,000 tokens.

The same arithmetic applies to the others: whatever you pay per token at standard rates, a batch request costs half. Our LLM API cost calculator gives the standard figures to start from, and the vendor pages on OpenAI API pricing, Claude API pricing and Gemini API pricing list the rates by model. Batches are a strong fit for reasoning models, whose output tokens make up most of the bill: Anthropic even recommends batch processing for thinking budgets above 32,000 tokens.

Failed work is not charged the same way everywhere. OpenAI says that when a batch expires you are charged only for the requests that completed. Anthropic does not bill requests that end as errored, canceled or expired.

Limits and gotchas

• Not every feature is available. Gemini's batch mode only supports generateContent, and Anthropic excludes streaming, among other options, from batch requests. OpenAI lists the endpoints it accepts in a batch file.
• Separate rate limits. OpenAI runs batches on their own rate limits, so a large batch does not eat into your real-time quota. Anthropic scopes batches to the workspace that created them.
• Caching still helps, but it is best effort. Anthropic suggests the 1-hour cache duration for batches that share context, and notes that cache hits inside a batch are not guaranteed. Gemini supports context caching in batch mode too.
• Expiry is real. A batch that does not finish inside its window ends as expired, and you get back only what completed. Plan to resubmit the rest.
• Validate before you submit. A malformed line can fail a request or the whole file. Test with a small batch of a few requests first.

When a batch is the wrong choice

Anything a person is waiting for does not belong in a batch: chat, support replies, autocomplete or an agent loop that needs the answer of one step before the next. Those need the real-time API. Batches pay off for work that can wait: classifying or tagging a backlog of documents, running evaluations across a test set, generating summaries or embeddings for a whole archive, translating a catalog or filling a database overnight.

A simple rule: if the result is needed in less than a day and nobody is staring at a spinner, price the job both ways. At 50% off, a nightly batch often costs less than optimizing the prompt, and you can combine both. If you are new to the unit these prices are quoted in, start with our guide to what a token is.