The context window, explained
What a model can hold in mind at once.
A model can only hold so much in mind at once. That budget is the context window, and absolutely everything has to share it.
A language model does not read words. It reads tokens: chunks of text roughly three-quarters of a word long. “Understanding” begins by cutting your text into these pieces.
One budget for everything
The context window is the number of tokens the model can attend to at a single moment. Your prompt, the conversation so far, any documents you paste, and the model’s own reply all draw from that one shared budget. Spend it on a long document and you have left yourself less room for a long answer.
Run out of room and the oldest words quietly fall off the edge.
Why attention is the magic
Inside the window, a mechanism called attention lets every token weigh every other token. That is why a model can connect a clause on the first page to a question on the fiftieth, and why the cost of the window climbs so steeply as it grows, since every token must consider all the rest.