Why AI Forgets Your Rules In Long Chats
From the course: Agentic AI For Founders & Senior Operators
Context rot is when a long session starts forgetting what you told it at the start, including instructions like “don't send anything without checking with me.” It comes from how the attention mechanism works, and it sets in if you never compact or clear the context. There's no clear warning, so watch for the AI forgetting what you said hours ago or fixating on small things, like repeating one opinion in every reply.
Transcript
That is actually called context rot. And what context rot means in practical terms is, let's say we took this example, and this is what we said to it first. Let's say... a good example of this actually is... Over time, Claude will continue to forget the things that you said at the outset. So if at the outset you gave it project instructions and user instructions, hypothetically, you could try this yourself. Let's say you said, "Don't send anything without checking with me." While that's explicitly there and you still need to put it in, if the session runs for really, really, really long and you never compact the context or clear it away, Claude will start to forget that. And that's just how the attention algorithm or the attention mechanism works under the hood for agents. Practically for us what this means is that we need to be able to... it's kind of like having a BS detector. So you kind of need to have that context rot detector in your mind, all in service of making sure that you're getting the right output from your agent. There isn't really like a signal for this, but you will sort of know that sometimes it will either forget the things that you told it like two hours back or it will start to bring in very irrelevant things and it tends to latch on to silly little things that we don't know about. Like sometimes I'll give it an opinion on something and then it will just keep reinforcing that same opinion for every single subsequent message that I send it. Okay. And that's context rot.