Every AI model has a context window—the maximum amount of text it can process in a single request. Send too much and the model either truncates your input, returns an error, or silently drops the earliest parts of your conversation. For solo operators feeding customer emails, blog drafts, or research notes into an AI assistant, hitting that ceiling means lost context and useless output.
Understanding how context windows work, what actually counts against them, and when to split your input saves you from re-running failed requests and wondering why the model ignored half your instructions.
What the context window actually measures
Context windows are measured in tokens, not characters or words. A token is roughly four characters in English, so a 128,000-token window handles around 96,000 words—but that’s a rough average. Punctuation, code, and non-English text tokenize differently.
The window includes everything: your system prompt, the user message, any examples you provide, the model’s previous responses in the conversation, and the new response it’s generating. If you’re running a multi-turn chat, earlier exchanges eat into the budget before your latest question even arrives.
As of November 2026, Claude offers models with context windows up to 200,000 tokens (checked November 2026). OpenAI’s GPT-4 Turbo and GPT-4o support 128,000 tokens (checked November 2026). Smaller models and older versions cap out much lower—sometimes as low as 4,096 tokens.
When you hit the limit
Different platforms handle overflow differently. Some models truncate silently from the beginning of the conversation, keeping only the most recent exchanges. Others return an error and refuse to process anything. A few will let you submit the request but deliver incomplete or incoherent output because critical context was dropped mid-thought.
If you’re pasting a 15,000-word research document and a 500-word prompt into a model with an 8,000-token window, the model will never see your instructions—they’ll be cut first. The API won’t always warn you.
Check your provider’s documentation for token-counting tools. OpenAI provides a browser-based tokenizer (checked November 2026) that shows exactly how your text gets chunked. Anthropic’s API returns token counts in the response metadata.
Chunking strategies that preserve meaning
When your input exceeds the window, split it intelligently rather than chopping at arbitrary character limits. Break on section headings, paragraph boundaries, or logical topic shifts—not mid-sentence.
Process each chunk separately with the same prompt, then either concatenate the outputs or run a second pass that summarizes the individual results. For long documents, include a brief summary of earlier chunks in each subsequent request so the model maintains continuity.
Avoid sliding-window approaches where you overlap chunks by 10–20%. They burn tokens on repetition and rarely improve output quality unless you’re doing deep semantic analysis across boundaries.
If you’re building a retrieval-augmented generation (RAG) system, pre-filter your knowledge base so only relevant excerpts enter the context window. Embedding-based search and vector databases let you pull the five most relevant paragraphs instead of dumping entire articles into the prompt.
What doesn’t count (and what shouldn’t)
System prompts and fixed instructions count against the window every time, even if they never change. If you’re running the same 2,000-token system message on every request, you’ve permanently reduced your effective window by 2,000 tokens. Trim boilerplate, combine redundant examples, and move static reference material out of the prompt and into a pre-processed lookup table.
Formatting tokens add up faster than you expect. JSON scaffolding, XML tags, and markdown syntax all consume budget. If you’re wrapping every data point in verbose markup, you’re trading tokens for structure that the model may not need.
Model responses count toward the window in multi-turn conversations, so long assistant replies in early turns compress the space available for later exchanges. If you’re running a ten-message chat and the model wrote 1,500 tokens in message three, those tokens reduce what you can send in message eleven.
When to pay for a bigger window
Larger context windows cost more per token. Claude‘s extended-context models and OpenAI’s GPT-4 Turbo charge higher per-million-token rates than smaller-window alternatives. If you’re processing hundreds of requests daily, a bigger window might double your API bill without improving output.
Use a larger window when you genuinely need the model to synthesize across many pages—legal document review, multi-file code analysis, or research summarization. Don’t default to the largest available model for single-question tasks that fit comfortably in 4,000 tokens.
For high-volume workflows, test whether breaking a 20,000-token job into four 5,000-token chunks and using a cheaper model produces acceptable results. The cost difference often outweighs the minor quality trade-off.
One thing to try this week: Paste your most common AI prompt into your provider’s token counter and check how much headroom you actually have. If you’re regularly within 10% of the limit, either shrink your input or switch to a model with a bigger window before you start losing context in production.
Got a question about working with AI tools in your online business? Reply to this email—we answer reader questions every Sunday.
Heads up — some links in this article are affiliate links. If you sign up through them, we may earn a small commission at no extra cost to you. We only recommend tools we use ourselves.