The Context Window

A customer has been chatting with the Pedal Point assistant for half an hour about a wheel upgrade. Suddenly it forgets which bike they own, or the request fails with a "too long" error. Nothing is broken. The conversation has outgrown the model's context window.

The rule

The context window is the model's entire world for one call.

Every model can handle a fixed maximum number of tokens per request. That budget is shared by everything:

• the instructions (system prompt)
• the conversation history the app resends
• retrieved documents and tool results
• the new question
• and the reply the model writes

The model sees nothing outside this window. If something is not in it, as far as the model is concerned it does not exist.

Window sizes vary a lot between models, from a few thousand tokens to very large ones. Bigger is not free, though. Longer inputs cost more and take longer, and models can overlook a detail buried inside a huge input. A focused context usually beats a stuffed one.

When a conversation grows, you have two main tools:

• Context compaction. Replace older turns with a short summary ("Customer owns a 2021 gravel bike, wants tubeless wheels, budget about 300"). The key facts survive; the tokens shrink.
• Context isolation. Give a side task its own fresh, clean window. For example, send a long spec sheet to a separate call that extracts just the tyre sizes, then pass only that short result back to the main conversation.

The shortcut

Write the budget as a sum: window ≥ instructions + history + retrieved + question + reserved reply. If the left side loses, cut the least valuable part first, usually old history.

The trap is forgetting to reserve room for the reply. A request that exactly fills the window leaves the model no space to answer.

Drills

Answer fast — trust the rule.

Remember

• The context window is the model's entire world for one call.
• Window ≥ instructions + history + retrieved + question + reply.
• Always reserve room for the reply; compact old history first.