What is a context window? The AI limit that explains everything

The plain explanation

A context window is the total amount of text a model can hold in view at one time. Everything the model knows about your current conversation has to fit inside it: your instructions, every message either of you has sent, every file you uploaded, and the answer it is about to write.

A useful mental picture: the model has no memory of your conversation at all. Each time you send a message, the entire conversation so far is handed to it fresh, it reads all of it, and it writes the next reply. The context window is the size of the desk that transcript has to fit on. When the desk is full, something has to come off it.

That is why an AI can seem sharp for twenty messages and then start contradicting itself, forgetting a constraint you set at the beginning, or asking for a file you already gave it. It did not get confused. The early part of the conversation slid off the desk.

One distinction to get right immediately, because it causes a lot of confusion: the context window is not the same as memory. Products like ChatGPT and Claude have a separate memory feature that stores facts about you between conversations and quietly inserts them into new ones. That is a product feature built on top of the model. The context window is a hard property of the model itself, and no memory feature makes it bigger.

Tokens, and how to estimate them

Context windows are measured in tokens rather than words, because models do not read words. They read chunks: common words are usually one token, longer or unusual words split into several, and punctuation and spaces count too.

The rough conversions worth remembering:

• 1 token is about 4 characters of English, or about three quarters of a word.
• 1,000 tokens is roughly 750 words, about a page and a half of normal prose.
• Code is denser. Expect closer to one token per three characters because of symbols, indentation, and unusual identifiers.
• Other languages are less efficient. Text in languages that are less represented in the tokenizer can take two or three times as many tokens for the same meaning, which is worth knowing if you are paying per token.

That gives you a way to picture the numbers you see in marketing material:

Context window	Roughly equal to
8,000 tokens	A long article
32,000 tokens	A short research paper with notes
128,000 tokens	A 300 page book
200,000 tokens	A dense technical manual, or a mid-size codebase
1,000,000 tokens	Several books, or a year of meeting transcripts

The important thing this table hides: the limit covers the whole conversation, not each message. A 128,000 token window does not mean you can send 128,000 tokens repeatedly. It means the running total of everything, including the model's own replies, has to stay under that number.

What actually fills up your window

People are usually surprised by how fast the window fills, because most of what goes into it is invisible. In a typical chat session, the model is reading all of the following on every single turn:

1. The system prompt. Instructions the product sends before you type anything: how to behave, what tools exist, today's date, safety rules. Often thousands of tokens, and you never see it.
2. Your custom instructions or memory. Anything the product has stored about you and injects automatically.
3. Every previous message, yours and the model's, in full. The model's long answers count too, and they are usually the biggest contributors.
4. Every uploaded file, or the extracted portions of it. A 40-page PDF is roughly 20,000 to 30,000 tokens.
5. Tool and search results. A web search that pulls five pages can add more than your entire conversation so far.
6. The reply being generated. Output shares the same budget as input.

This is why a conversation that felt short can be near its limit. You sent eight short messages, but the model wrote eight long answers, you attached two documents, and it ran three searches. The visible part of that is maybe ten percent of the total.

It is also why the fix for a confused conversation is so often "start a new chat". You are not resetting the model's mood. You are clearing the desk.

Advertised size versus useful size

This is the part that matters most and gets discussed least. A model's quality does not stay flat right up to its limit and then fall off a cliff. It degrades gradually, and it starts degrading well before the limit.

The best documented version of this is often called the lost in the middle effect. Put a specific fact at the start of a long document and the model finds it. Put it at the end and the model finds it. Bury it in the middle of 200 pages and retrieval accuracy drops noticeably. Attention is not evenly distributed across a long input, and the edges get more of it.

It gets harder still when the task requires combining several facts scattered across a long input. Finding one needle in a haystack is a solved problem. Finding four needles and reasoning about the relationship between them is not, and this is exactly the task people use big context windows for.

So treat the advertised number as a maximum capacity, not a comfortable working range. A practical rule of thumb from real use: you get reliable behaviour up to roughly half the advertised window, and you should verify anything beyond that rather than trust it. If a model tells you a 300 page contract contains no termination clause, check, particularly if the clause would be in the middle.

The comparison implication is that a model with a one million token window is not automatically better at long documents than one with 200,000. It is better at accepting them. Whether it reasons well across all of it is a separate question, and the only way to know is to test with a document where you already know the answer.

What happens when you run out

Different products handle overflow differently, and knowing which one you are using explains a lot of odd behaviour.

Behaviour	What you see	Where it happens
Hard error	The request is rejected with a message about length	Most direct API use
Silent truncation	Oldest messages are dropped without telling you	Many chat interfaces
Rolling summary	Old messages are compressed into a summary	Increasingly common in chat products
Retrieval	Only the relevant parts of your documents are fetched per turn	Document and knowledge-base tools

Silent truncation is the one that causes real problems, because nothing announces it. The symptom is a model that suddenly ignores a rule you set at the start, reverts to a tone you corrected an hour ago, or asks a question you already answered. Nothing has gone wrong with the model. Those instructions are simply no longer on the desk.

Rolling summaries are better but lossy. A summary keeps the gist and drops the specifics, so the constraint "never use the word synergy" survives as "the user has style preferences", which does not help you at all.

Practical tactics that actually work

These are ordered by how much difference they make relative to effort.

Start a new chat when the topic changes. The single highest-value habit. A long conversation carries the cost of everything before it, including tangents that are no longer relevant. Long conversations are not more informed, they are more expensive and more diluted.

Put your question after the long material, not before. If you paste a document and then ask, the question is close to where the answer gets generated, which measurably improves accuracy on long inputs. Paste first, ask second.

Re-anchor important constraints. In any conversation over about fifteen exchanges, restate the rules that matter in the message where they matter: "as a reminder, British English, no bullet points, under 400 words". It costs you one line and it survives truncation.

Send the relevant pages, not the whole book. If you need the model to check the indemnity clause, give it the indemnity clause and its neighbours. Precision beats volume, it is faster, it is cheaper, and it is more accurate.

Summarise and restart deliberately. When a working session gets long, ask for a structured handoff: current state, decisions made, open questions, constraints. Paste that into a new chat. You keep the substance and drop the noise, and you will notice the model gets sharper immediately.

Use retrieval for anything that does not fit. If your material is genuinely larger than any window, the answer is not a bigger window, it is fetching the relevant chunks per question. That is what document tools do under the hood.

Watch for degradation, not just errors. If answers get vaguer, hedge more, or start ignoring format instructions, you are probably deep in the window. Start fresh before assuming the model has got worse.

How much window do you actually need?

Match the window to the job rather than shopping for the biggest number.

Your work	What you need	Why
Emails, drafting, quick questions	Anything modern	You will never come close to the limit
Editing long documents	100,000 plus	The document plus your conversation about it
Contract and policy review	200,000 plus, and verify	The document plus the reasoning about it, with the caveat above
Reasoning across a whole codebase	The largest available	Code is token-dense and cross-file reasoning needs breadth
Analysing months of transcripts	The largest available, or retrieval	Often better served by retrieval than by brute force
Building a product on an API	Smaller than you think, plus caching	Long prompts are the main driver of both cost and latency

For most people the honest answer is that context window is not the deciding factor between subscriptions. Writing quality, ecosystem, and price matter more. It becomes the deciding factor in exactly one situation: your work regularly involves feeding in more material than a normal window holds, and in that case the difference is not marginal, it is the difference between possible and impossible.

If you want to test that yourself, the practical approach is to run the same long document through two or three models and check the answers against something you already know. A workspace like Whizi makes that easy because GPT, Claude, Gemini, Grok, and DeepSeek sit behind one subscription, and Gemini's very large window is one click from Claude's careful synthesis. See comparing models side by side, or read how to choose an AI model for the wider decision.