How LLMs Work, in Plain English

Predicting the next word, at massive scale

A large language model is trained to predict what comes next in a sequence of text. Given millions of examples of human-written text, the model learns which words tend to follow which other words, and in what contexts.

When you type a prompt, the model generates a response one token at a time, each time predicting the most useful next piece of text given everything that came before.

Why they sound confident even when wrong

The model is optimized to generate fluent, coherent text, not to be accurate. Confident-sounding language appears more in well-written training data, so the model learns to produce confident-sounding language. This is why LLMs "hallucinate": they generate plausible-sounding text that happens to be factually wrong.

Hallucination is not a bug about to be patched. It is built into how these systems work. The solution is not to wait for a better model. Design your use of AI to catch errors.

Context windows: the AI's working memory

A context window is how much text the model can "see" at once. Early models could handle a few thousand words. Current models can handle hundreds of thousands. This matters because:

• The more context you give, the better the response
• Long conversations eventually push early content out of the window
• For complex tasks, how you structure your prompt matters enormously

The practical implication

LLMs are not databases. They store patterns, not facts. That makes them excellent at tasks requiring language, structure, synthesis, and style. They are unreliable for tasks requiring precision, recency, or calculation.

Use AI for drafting, rewriting, summarizing, brainstorming, and structuring. Use traditional tools, search, databases, calculators, for facts, numbers, and current information.