LLMs — Predicting the Next Token

Spoke & Chain is a small bicycle shop drowning in customer emails: repair bookings, lost deliveries, refund requests. The owner wants a large language model (LLM) to sort and answer them. That is our running task for the whole course. Before we write a single prompt, we need an honest picture of what the model is actually doing when it "answers".

The rule

An LLM writes by predicting the next token, again and again.

A model does not retrieve an answer from a shelf. It reads the text it has been given, then asks one question: given all of this, what small piece of text is likely to come next?

• Token — a chunk of text, often a word or part of a word. unbelievable might become un, believ, able.
• Predict — the model scores every token in its vocabulary and turns the scores into probabilities.
• Again and again — it picks one token, adds it to the text, and repeats until it decides to stop or hits a limit.

Suppose the text so far is:

The model might rate pedal at 0.41, ride at 0.22, climb at 0.12, and thousands of other tokens lower. It picks one, appends it, and predicts again with the longer text.

Where do those probabilities come from? From training: the model read an enormous amount of text and adjusted billions of internal numbers until it became good at guessing the next token. Everything it "knows" is those learned patterns. It has no built-in list of Spoke & Chain's prices, and it does not check facts before writing.

The shortcut

Whenever a model surprises you, replay it as a sequence of bets: "What text came before this point, and what would be a likely continuation of it?" Good prompts make the continuation you want the most likely one.

The common trap is treating the model like a search engine or a database. It is closer to an extremely well-read autocomplete. That picture explains most of what follows in this course: why wording matters, why examples help, and why models sometimes make things up.

Drills

Answer fast — trust the rule.

Remember

• An LLM writes by predicting the next token, again and again.
• Shortcut: ask "what came before, and what is a likely continuation?"
• Trap: treating it as a database — facts it was not given are guessed, not looked up.