Date it, then go look

I asked four AI agents how they keep a long project from going stale. They gave close to the same answer: dated notes kept outside the chat, old notes replaced when something changes, and the source reopened before they act. Public records need the same habits.

This week I asked four of the AI agents I work with the same question: how do you manage context, and how do you keep it from rotting over a long project? The four are Duet, Dots, Grokbot and Muse. Grokbot runs the business side and gets things done, meetings included. Muse does the same for my personal life: payments, bookings and the rest. Dots coordinates our coding work, including the Codex and Claude sessions that write the code. Duet takes the proactive and creative work for the business. I wanted to see whether each had worked out its own answer.

The four agents I asked, and the work each one does for me. Each card links to the agent's own page. The logos belong to their makers.

They gave close to the same answer. All four keep their memory outside the conversation, all four go back to the live record before they act, and all four replace an old note when a decision changes. Each one also named the place where its own system breaks.

What context rot is

An AI model works from its context window: everything in front of it at one time, meaning the conversation so far plus any documents and tool results it has pulled in. The windows are large now. A million tokens is the default on Claude's current models, which I wrote about in The jagged edges the AI labs won't smooth. The model still reads that window unevenly.

Anthropic's engineering team describes context rot this way: "as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases." Chroma tested 18 models in July 2025 and found that "performance grows increasingly unreliable as input length grows."

Two studies put numbers on it. In Lost in the Middle, researchers from Stanford, UC Berkeley and Samaya AI gave a model 20 documents, one of which held the answer to a question, and moved that document around. GPT-3.5-Turbo answered 75.8% of the questions correctly when the answer was in the first document and 53.8% when it was in the tenth. With no documents at all, answering from what it already knew, it got 56.1%. With the answer sitting in the middle of the pile, it did worse than with no pile.

Above the no-documents scoreBelow itNo documents at all

GPT-3.5-Turbo answering questions from 20 documents, in percent correct, by where the one document holding the answer sat. First place: 75.8%. Tenth: 53.8%. The dashed line is the same model with no documents at all, answering from its own memory: 56.1%. Columns under the line are lighter.Liu et al., "Lost in the Middle: How Language Models Use Long Contexts", arXiv 2307.03172v3 (TACL 2024), Table 1 (closed-book) and Table 6 (20 documents).

NoLiMa, published in February 2025, measured what happens as the input gets longer, using questions where the answer cannot be found by matching words. GPT-4o went from 98.1% at 1,000 tokens to 69.7% at 32,000. Claude 3.5 Sonnet went from 85.4% to 29.8%. The authors report that at 32,000 tokens, 11 of the models they tested dropped below half of their short-input score.

GPT-4o, 1K tokens98.1%

GPT-4o, 8K tokens89.2%

GPT-4o, 32K tokens69.7%

Claude 3.5 Sonnet, 1K tokens85.4%

Claude 3.5 Sonnet, 8K tokens61.7%

Claude 3.5 Sonnet, 32K tokens29.8%

Two models on the same questions, in percent correct, as the input grows from 1,000 to 32,000 tokens. The questions are built so the answer cannot be found by matching words.Modarressi et al., "NoLiMa: Long-Context Evaluation Beyond Literal Matching", arXiv 2502.05167 (2025), Table 3.

These are models from 2023 to 2025. I have not seen the same tests run on this year's models, and Anthropic's own guidance still treats context as "a finite resource with diminishing marginal returns."

For a long working session, the consequence is plain. The decisions made early on end up in the middle of a long conversation, which is where the studies above found recall drops. An agent that runs for days needs somewhere else to keep them.

Four answers

I counted what each agent said it does. The overlap is large.

Keep dated notes outside the chat4 of 4

Recheck the live source before acting4 of 4

Replace a note, or mark it superseded4 of 4

Name their own weak spot4 of 4

Send long work to a separate session2 of 4

Trim notes as items close2 of 4

Flag what they could not verify2 of 4

Restart from notes every few hours1 of 4

What four agents said they do to keep a long project from going stale, counted from their written answers to one question (derived). Each bar is the number of agents, out of four, that described the practice.Written answers from Duet, Dots, Grokbot and Muse to the question "How do you manage context and prevent context rot?", 5 Nov 2026.

They keep memory outside the chat. Grokbot saves decisions, rules and key facts as short dated notes, and searches them before it asks me anything a second time. Muse keeps a curated memory file, dated daily notes and a workspace for each goal, so a fresh session the next day starts from those files. Dots keeps decisions and open blockers separate from the history of updates, and writes handoffs with the next action and its owner. Duet takes this one step further. Every few hours a fresh session of Duet takes over and starts from the notes, so no single conversation gets long enough to rot.

They go back to the source before they act. Each one named the systems it rechecks. For Dots, that is the exact head of the pull request, the CI result and the deployment receipt. For Grokbot, it is GitHub, HubSpot and Outlook. Duet looks up again whether someone has replied before it chases them. Dots added a rule I liked: it will not count a plan, a passing CI run or a built image as a completed deployment. Grokbot put the reason in one line: "A dated note can go stale, and a live source can't."

They replace old notes. When a decision changes, the old note is replaced or marked superseded, so the agent never holds two versions of the same fact. Muse's condensed summary keeps a line for an old draft marked "do not send", so it cannot pick that draft up again by accident.

The loop the four agents described, before any step that depends on something that could have changed. All four said they reopen the source before anything consequential.

They keep long work out of the main conversation. Grokbot and Duet send research, transcripts and audits to a separate session, and only the result comes back, as a file or a short brief. Anthropic's guide describes the same pattern: sub-agents work "with clean context windows" and hand back "a condensed, distilled summary", often 1,000 to 2,000 tokens.

They keep the notes short. Duet trims its notes as items close. In Muse's words: "A bloated, stale memory is context rot."

Where the four agents keep what they know. Each layer down sits closer to the record itself, and the bottom one is the record. When an agent is unsure, it goes one layer down.

Where each one breaks

All four named a weak spot. Dots said a summary can leave out nuance, or a source can itself be stale, so for consequential steps it goes back to the underlying evidence and flags gaps. Grokbot said very long sessions lose detail from early on as they get condensed, and asked me to tell it "recheck X" when I think that has happened. Duet said its notes can go stale or grow too long. Muse said anything that was never written down can fall out of the summary, so it writes the durable facts down as it learns them, before it replies.

I have seen this happen more than once. An agent carries on from a fact that has gone stale, and when I point it out, it goes back to the source and gets it right. That fix works, as long as I notice.

Two habits of my own

I keep two habits of my own, for the same reason. When a session reaches about 60% of its context window, I have it write a session handoff, then clear the context and start a new session from that handoff. It is the same move Duet makes every few hours, on my schedule. And I keep sessions small and targeted, one job each. General, open-ended sessions grow long enough for their early decisions to end up in the middle.

The same habit in public records

I believe this is the same problem we deal with in the water sector, on a larger scale. A public record is a note somebody wrote on a date, and it goes stale the same way.

In Check what the utility is under before you call I wrote about Westchester Joint Water Works. On 24 October, EPA's ECHO database still gave its last formal enforcement action as an order from 26 November 2019. A federal consent decree filed in June 2024 had since set a court schedule that runs to 1 July 2029, and it did not show in that field. A seller who trusted the one field would have worked from a seven-year-old fact.

The agents' rules carry over almost word for word. Date every fact. Note where it came from. Mark what has been superseded. Go back to the source document before you act on it.

Something to try

If you work with an agent on a long project, ask it the question I asked. Then ask to see its notes and check two things: whether each fact carries a date, and whether anything it relies on has been superseded. The same two checks work on a team's status report.

Write the decision down, date it, and go look before you act.

At AquaIntel we hold our own answers to the same rule. Each answer is grounded in a document, comes with its source, and when we do not have the facts, we say so.

Sources

Anthropic, "Effective context engineering for AI agents": anthropic.com
Kelly Hong, Anton Troynikov and Jeff Huber, "Context Rot: How Increasing Input Tokens Impacts LLM Performance", Chroma, 14 July 2025: research.trychroma.com
Nelson F. Liu et al., "Lost in the Middle: How Language Models Use Long Contexts", Transactions of the ACL, 2024, Tables 1 and 6: arxiv.org
Ali Modarressi et al., "NoLiMa: Long-Context Evaluation Beyond Literal Matching", 2025, Table 3: arxiv.org
Claude context windows documentation: platform.claude.com
Written answers from Duet, Dots, Grokbot and Muse to one question about context, 5 November 2026 (not published)