Why LLMs begin to forget ❌️ · Digital Roadmap AI Academy

Why LLMs begin to forget ❌️

I knew this was the case when I said. Let me do some more research on this. Because I have a lot of chats. Windows where i've had the same chat open for weeks and I keep adding more and more to the chat, and then the llm tells me that the files I have uploaded are expired. But I kept pushing it lol

Lesson learned: don't keep long chats open.. start a new one because the LLM memory will not keep going back up to the top of the chat.

🤖 See below from gemini:

When you "hit" or exceed the context window, the model doesn't delete files or conversations; it simply starts "forgetting" the earliest parts of the current chat to make room for new information.

Here is how that works in practice:

1. The "Sliding Window" Effect

Think of the context window like a rolling spotlight.

* As the conversation gets longer and longer, the spotlight moves forward.

* The messages at the very beginning of the chat might fall into the "shadows."

* The Result: The AI might lose track of a specific detail you mentioned 50 messages ago, but the text of those messages is still visible to you in the chat UI.

2. Files and the Context Window

If you upload a large file (like a 200-page PDF):

* Storage: The file is saved to your chat history permanently.

* Context: The AI "loads" that file into its context window so it can answer questions about it. If the conversation goes on for a very long time after that, the AI might "lose focus" on the file's details. However, most modern LLMs (like Gemini) have systems to "re-read" the file if you ask a question about it again.

3. "Forgetting" vs. "Deleting"

It is important to distinguish between the two:

* Forgetting (Context): The AI can't "see" the old data in its active memory anymore. This is temporary and only affects the AI's logic for that specific moment.

* Deleting (Storage): This is the permanent removal of data. Exceeding the context window never triggers a deletion of your chat history or your uploaded files.

Comparison at a Glance:

| Feature | When Context is Full... | Is it Deleted? |

|---|---|---|

| Original Local Files | Nothing happens. | No |

| Chat History (Sidebar) | Everything remains saved. | No |

| AI's Immediate Memory | Oldest messages "slide" out of view. | Yes (Temporarily) |

> Peer-to-peer tip: If you feel like the AI is starting to get "confused" because the conversation has become too long, the best move is to start a fresh chat. This clears the "clutter" and lets the AI focus on your current task with a full, empty context window.

🤖 you said research the specific context window limit for the version of the AI model and and version that you are using.

Let me know if you've ever experienced this.And how do you fix it👇

19 comments