What is an LLM? How large language models work, in plain English

What is an LLM? A large language model is a neural network trained on enormous amounts of text to predict what comes next, one small piece of text at a time. Anthropic's glossary defines LLMs as language models "with many parameters" that can "generate human-like text, answer questions, summarize information, and more". ChatGPT, Claude and Gemini are products built on top of LLMs: the model predicts text, and the product adds the chat window, search, memory and tools. This explainer covers how an LLM produces an answer, how it is trained, what "large" means, what it cannot do and how it relates to chatbots and agents, citing the original papers and the companies' own documentation.

What an LLM actually does: predict the next token

An LLM reads and writes tokens, not words. Google's machine learning course puts it simply: a token "can be a word, a subword (a subset of a word), or even a single character", and LLMs predict "a token or sequence of tokens, sometimes many paragraphs worth". OpenAI's rule of thumb is that one token is about four characters of English, or three quarters of a word. Our explainer on what a token is shows why the same sentence costs more tokens in Spanish than in English.

Generating an answer is a loop:

1. The model reads your prompt, plus anything the product adds, such as instructions, earlier messages or search results.
2. It computes a probability for every possible next token.
3. One token is picked, added to the text, and the loop starts again with the longer text.
4. It stops at a special end token or at a length limit.

The temperature setting controls how that pick is made. Anthropic's glossary explains that higher temperatures give "more creative and diverse outputs" and lower ones "more conservative and deterministic outputs", and warns that even at temperature 0 the results "will not be fully deterministic". That is why asking the same question twice can give two different answers.

Everything the model can take into account in one go, your prompt plus its own answer so far, has to fit in its context window: Anthropic calls it the model's "working memory", as opposed to what it learned in training. Current Claude models accept up to 1 million tokens. Our guide to context windows explains what happens when a conversation outgrows it.

How an LLM is trained

Training happens in stages, and each one changes what the model is good at.

Pretraining. The model reads a huge collection of text and learns to predict the next word. Anthropic's glossary notes that a model at this stage is "not inherently good at answering questions or following instructions". It has absorbed grammar, facts and styles, but it continues text rather than helping you.

Fine-tuning. The pretrained model is trained further on examples of the behavior wanted, such as good answers to questions. Google's course notes that fine-tuning can work with "just a few hundred or a few thousand training examples", because the base model already knows so much.

Learning from human feedback (RLHF). People rank several answers to the same prompt, and the model is trained to prefer answers like the higher-ranked ones. The best-known demonstration is OpenAI's InstructGPT paper (March 2022): in human evaluations, answers from a 1.3 billion parameter model trained this way were preferred to those of the 175 billion parameter GPT-3, "despite having 100x fewer parameters". Anthropic says Claude has been trained with RLHF to be a more helpful assistant.

Training stops at a cutoff date, so a model does not know what happened after it unless the product gives it search or documents. Anthropic lists June 2026 as the training data cutoff of its current models.

The Transformer: the architecture behind LLMs

Google's course calls the Transformer "the most successful and widely used architecture for building LLMs". It was introduced in June 2017 in the paper "Attention Is All You Need" by Ashish Vaswani and colleagues, which proposed an architecture "based solely on attention mechanisms, dispensing with recurrence and convolutions entirely" and showed it was better at translation while "more parallelizable and requiring significantly less time to train".

The key idea is self-attention. For each token, Google's course explains, the model asks: "How much does each other token of input affect the interpretation of this token?" In the sentence "The animal didn't cross the street because it was too tired", attention lets the model link "it" to "animal"; change "tired" to "wide" and "it" should point to "street" instead.

The original Transformer had two halves, an encoder that reads and a decoder that writes. Google's course also describes decoder-only models, which "generate new tokens from the text already generated" and can "create continuations of dialog histories and other prompts", which is exactly the job of a chat assistant.

What "large" means: parameters, experts and smaller copies

The "large" in LLM refers to parameters, the numbers the model adjusts during training. Google's course says LLMs beat older language models because they "contain far more parameters" and "gather far more context". More parameters usually mean better predictions, but also more memory and more computing per token.

Three ideas complicate the simple "bigger is better" picture:

• Mixture of experts. Many recent models only use part of their parameters for each token. DeepSeek describes its V4.1 Flash as a 552 billion parameter mixture-of-experts model that uses "just 8B active parameters for input, 16B for output".
• Training quality. The InstructGPT result shows that how a model is trained can matter more than size.
• Smaller copies. Distillation trains a smaller "student" model on the outputs of a larger "teacher". Google notes the result is faster and cheaper to run, though its predictions are "generally not quite as good". Quantization shrinks a model further by storing each parameter with fewer bits, which is how people run LLMs on their own computer; see what quantization is.

Not every company publishes these numbers: Anthropic's models page, for example, lists context windows and prices but no parameter counts.

What LLMs can't do on their own

• They can state false things confidently. OpenAI defines hallucinations as "plausible but false statements" and argues that training and evaluation reward guessing over saying "I don't know". Our guide to model hallucinations covers how to reduce them.
• They don't know recent events beyond their training cutoff, unless a search tool or your own documents are added. Feeding documents into the prompt is called RAG; see what RAG is.
• They don't act by themselves. A model only produces text. Products let it call tools, such as search, code execution or your apps, and an LLM that plans and uses tools in a loop to reach a goal is what people call an agent.

So when you compare ChatGPT, Claude and Gemini, you are comparing both the models underneath and everything each company builds around them; our Claude vs ChatGPT comparison shows how much the surrounding product matters.