Context Windows — The Model's Desk

Asha has been chatting with Tripmate for an hour: three trips, dozens of train searches, a long list of hotel options. Suddenly Tripmate "forgets" that she is vegetarian, which she said at the very start. Nothing broke. The conversation simply outgrew the model's context window.

The rule

The context window is everything the model can see at once — nothing more.

A model has no memory between calls. Each time you call it, you send a bundle of tokens, and it answers using only that bundle. The context window is the maximum size of that bundle, measured in tokens. It must hold:

• the system instructions ("You are Tripmate, a travel assistant…"),
• the tool descriptions the agent may use,
• the conversation so far, including tool results,
• and the answer the model is about to write — output tokens share the same window.

Window sizes vary a lot between models and keep growing, but every window has an edge. When the bundle would overflow, something must go: the application trims, summarises or drops older material, or the call fails. Even inside a big window, models tend to use details in a very long context less reliably than details that are short and clearly placed, so "it fits" is not the same as "it will be noticed". Every token in the window is also paid for on every call, which the next lesson turns into numbers.

The shortcut

Think of the window as a desk. Anything not on the desk does not exist for the model. Before blaming the model for forgetting, ask: "Was that fact on the desk for this call?"

The trap is assuming a bigger window solves everything. More room means more cost per call, slower replies, and important details buried in noise. Good agents decide what goes on the desk.

Drills

Answer fast — trust the rule.

Remember

• The context window is the whole bundle the model sees in one call, answer included.
• Shortcut: "Was it on the desk for this call?"
• Trap: treating a bigger window as a cure — curate what goes in.