A context window is a budget, not a memory
Watch a long conversation push early information out of view, then compare truncation, compression, and selective retention. Reliable context management is deciding what deserves to survive.
THE QUESTION THIS PAGE ANSWERS
ANSWER FIRSTWhat is the key idea behind “A context window is a budget, not a memory”?
Watch a long conversation push early information out of view, then compare truncation, compression, and selective retention. Reliable context management is deciding what deserves to survive.
Forgetting is often a policy failure. When the window fills, something must be dropped, summarized, or retrieved again. Define that policy deliberately instead of letting the oldest turns disappear by accident.
Mark which three facts a task must retain after ten more turns.
Adding more context without deciding what the model should treat as authoritative.
AI doesn't have infinite memory. Each time it answers, it can only see the most recent content that fits on the "desk". When the desk (the term is "context window") is full, the earliest words get pushed off. Once they're off, they no longer take part in this answer.
Imagine a desk in front of the AI. Every turn you chat, a sticky note goes on the desk, written with what was said that turn. When the AI answers you, it only looks down at the notes currently on the desk. The desk is only so big. Once it's full, every new note pushes the oldest one off. Notes on the floor are invisible — and it can't pick them back up.
Desk size = context window
Different models have bigger or smaller desks, but all of them have an edge. Full is full — paying more doesn't conjure a ninth slot.
One sticky note = one turn
What you say and what it replies both take space. Long rambles and big pasted documents take more, so the desk fills faster.
Below is a simulated conversation. In turn 1 you told it "I'm allergic to peanuts." Drag the slider right to add turns, watch the desk fill up, and see what dinner it recommends after that key note falls off.
Restate important info every so often
Allergies, budget, format requirements — after a dozen turns, say them again. That's putting the note back on top of the desk.
Desk full? Just open a new chat
The most direct move, and the one people forget: a new chat is an empty desk, and whatever got pushed off no longer takes up space. For short tasks, reopening is the whole fix. For long ones, don't start cold — have it summarize progress first and paste that summary in before you continue.
If it forgot, don't argue with it
It can't flip back to notes on the floor. Just resend the key info — far more useful than asking "did you forget?" three times.
How “An analogy first · A desk for sticky notes” changes an answer
“AI doesn't have infinite memory.” shows that a model does not process the “word count” we see. It processes Token pieces. Tokenization affects input length, how much context fits, and how much computation a request consumes.
Length, information, and context are different
As “Imagine a desk in front of the AI.” grows, separate three questions: how many Tokens the text becomes, which pieces can change the current decision, and whether older material has fallen outside the context window. Removing repetition is often more useful than simply making the window larger.
- Forgetting is a capacity problem : it's not broken — the desk is full, and the earliest words got pushed off
- Each answer only sees what's in the window : what fell off might as well never have been said
- Restate important settings : say them again after a while, put the note back on the desk
Keep what can change the decision
Use “It can't flip back to notes on the floor.” as an A/B test: keep the same question while removing repeated background, compressing format, and trimming irrelevant history. Compare answer quality, latency, and Token count.
From “An analogy first · A desk for sticky notes” to “Try it yourself · Fill the desk”
“An analogy first · A desk for sticky notes” grounds the problem in “Imagine a desk in front of the AI. Every turn you chat, a sticky note goes on the desk, written with what was said that turn. When the AI answers you, it only looks down at the notes currently on the desk. The…”. “Try it yourself · Fill the desk” then moves it toward “Below is a simulated conversation. In turn 1 you told it "I'm allergic to peanuts." Drag the slider right to add turns, watch the desk fill up, and see what dinner it recommends after that key note falls off”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.
Carry the judgment into the next situation
For long text, keep what can change the conclusion before compressing format and history. A larger context is worth its cost only when the added information is useful.
- “An analogy first · A desk for sticky notes”: Imagine a desk in front of the AI. Every turn you chat, a sticky note goes on the desk, written with what was said that turn. When the AI answers you, it only looks down at the notes currently on the desk. The…
- “Try it yourself · Fill the desk”: Below is a simulated conversation. In turn 1 you told it "I'm allergic to peanuts." Drag the slider right to add turns, watch the desk fill up, and see what dinner it recommends after that key note falls off
- “The closing point”: Reopen once the desk fills up : short tasks just need a new chat; long ones should carry a summary across
The final “The closing point” brings the discussion to “Reopen once the desk fills up : short tasks just need a new chat; long ones should carry a summary across”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.
✅ What this page wants to share with you
- Forgetting is a capacity problem: it's not broken — the desk is full, and the earliest words got pushed off
- Each answer only sees what's in the window: what fell off might as well never have been said
- Restate important settings: say them again after a while, put the note back on the desk
- Reopen once the desk fills up: short tasks just need a new chat; long ones should carry a summary across
INTERACTIVE PRACTICE
Turn a vague request into a useful prompt
Clarify the goal, context, and constraints, then carry the finished prompt into the AI tool you use.
I used to treat a larger window as better memory. When debugging long conversations, I now check what was truncated and whether the summary dropped constraints. Many issues are really management problems.
When the window gets larger, does a model use information near the beginning equally reliably? If not, should the application repeat critical constraints closer to the end?
A “context budget” example would be useful, showing how much space system instructions, user input, history, and retrieved material each take. It would be a practical reference for design work.
No discussion on this article yet.