What is Context window?
The amount of text an LLM can process in one pass, measured in tokens. Larger windows (200k+) let agents keep more of a project in memory; smaller windows force summarization. Directly impacts cost per call.
What counts toward it
The context window is everything the model can reference while it writes a response, including the response itself. Anthropic describes it as the model's working memory, separate from the data it was trained on. In the Claude API the system prompt, every message (tool results, images and documents included), the tool definitions and the output Claude generates all count, as do thinking tokens when thinking is on.
Bigger is not automatically better
Each turn is added to the history and sent again, so a long conversation or agent run fills the window steadily. Anthropic notes that as the token count grows, accuracy and recall degrade, a phenomenon it calls context rot, so curating what is in context matters as much as how much room there is.
When it runs out
If the input alone is larger than the window, the API rejects the request with a 400 "prompt is too long" error. For long-running work, Anthropic recommends server-side compaction, which summarises earlier parts of the conversation so it can continue, and context editing, which clears old tool results or thinking blocks.
Example
An agent that reads fifty files and runs a dozen commands keeps every tool result in the conversation. By the end, most of the window is old output; clearing stale tool results or compacting the history frees room and often improves the answers.
Sources
See also
← Back to the full AI agent glossary.