What is a context window in a language model?

A context window is the amount of text, measured in tokens, that a language model can take into account at one time: your prompt, the conversation so far, any documents or tool definitions you send, and the answer it is writing. Google's documentation compares it to short-term memory. This guide explains what a context window is in a language model, how large the windows of the main models are as of October 6, 2026, what happens when you fill one, and why a bigger window is not automatically better. If the word "token" is new, start with what a token is.

What is a context window?

Anthropic defines the context window as "all the text a language model can reference when generating a response, including the response itself". It is different from the large body of data the model was trained on: it is the "working memory" of one conversation or request. The model does not remember anything outside it unless a product adds its own memory feature.

Everything in the request counts toward the window. For the Claude API that means the system prompt, every message including tool results, images and documents, the definitions of the tools you offer and the output Claude generates, including its extended thinking. Cached prompt prefixes also count: prompt caching changes what you pay for them, not whether they occupy space.

How big are context windows in 2026?

Table: context windows of popular models and products (October 6, 2026)

The same company can show different numbers for its API and its app, so check which product you are using. Anthropic notes that chat interfaces such as Claude's web app can also manage the window on a rolling "first in, first out" basis, which means the oldest parts of a long chat may drop out. Google puts 1 million tokens in perspective: about 50,000 lines of code at 80 characters per line, eight average-length English novels or more than 200 podcast transcripts.

What happens when you fill the window?

It depends on the product. In the Claude API, if the input alone is larger than the window the request fails with a 400 error ("prompt is too long"). On newer models, if the input plus the maximum output you allow goes beyond the window, the API accepts the request and, if the answer hits the limit, stops with the reason model_context_window_exceeded. In chat apps, older messages may be dropped or summarized so the conversation can continue.

To manage this, Anthropic offers server-side compaction, which summarizes earlier parts of a conversation on the server (in beta for Claude 4.6 and later models), and context editing, which clears old tool results or thinking blocks. You can measure a request before you send it with the token counting API.

Does a bigger context window mean better answers?

No, and the vendors say so. Anthropic writes that "more context isn't automatically better": as the token count grows, accuracy and recall degrade, a phenomenon it calls context rot. Google's long-context guide recommends putting your question at the end of the prompt, after all the other context, and warns that when you look for several specific facts in a huge input, the model "does not perform with the same accuracy".

In practice, a larger window lets you paste a whole report or code base, but you should still curate what goes in. Sending only the relevant pages is usually cheaper and often more reliable than sending everything.

How much does a long context cost?

You pay for every input token, so a full window costs more than a short prompt. The pricing rules differ by vendor. For Claude models with a 1M window, Anthropic says the full window is billed at standard rates: a 900K-token request costs the same per token as a 9K-token request. OpenAI states that for GPT-6.1 Sol, requests over 272K input tokens pay 2x on input and cached input and 1.5x on output for the whole request. Google recommends context caching to cut the cost of repeated context. Our LLM API pricing guide and calculator show the effect of caching and batch discounts on a real workload.

Context windows on your own computer

When you run a model locally, the context window is limited by memory. The model keeps a key-value cache that grows in a straight line with the number of tokens: for Llama 3.1 8B at 16-bit precision, our calculation (checked against the published model configuration) gives 1 GiB at 8,192 tokens, 4 GiB at 32,768 and 16 GiB at its full 131,072 tokens. The details are in how much VRAM an LLM needs.

Ollama sets a default context length by the memory you have: 4K tokens below 24 GiB of VRAM, 32K between 24 and 48 GiB and 256K at 48 GiB or more. You can change it with the slider in the app or with OLLAMA_CONTEXT_LENGTH=64000 ollama serve, and Ollama recommends at least 64,000 tokens for tools such as web search and coding tools. After you raise it, run ollama ps and check that the model still shows "100% GPU", which means nothing was pushed onto the processor.

How to work with a limited context window

• Keep the useful part: paste the relevant section, not the whole file.
• Put the question last: Google advises placing the query after the context.
• Start a new chat for a new task: old, unrelated messages still take up space.
• Summarize long threads: ask the model for a summary and continue from it, or use a product's compaction feature.
• Cache repeated context: both Anthropic and Google document caching to lower cost.
• Bring in documents on demand: for large knowledge bases, the usual answer is a system that retrieves only the passages needed for each question (retrieval-augmented generation), and agents do something similar by calling tools to fetch what they need.

What the context window is not

It is not the model's training data, and it is not permanent memory. It is not the same as the maximum output length either: Claude's 1M models can produce up to 128K tokens in a single response, a much smaller number. And it is not a quality guarantee: a model can accept a million tokens and still miss a detail buried in the middle.