Why LLMs begin to forget ❌️
I knew this was the case when I said. Let me do some more research on this. Because I have a lot of chats. Windows where i've had the same chat open for weeks and I keep adding more and more to the chat, and then the llm tells me that the files I have uploaded are expired. But I kept pushing it lol Lesson learned: don't keep long chats open.. start a new one because the LLM memory will not keep going back up to the top of the chat. 🤖 See below from gemini: When you "hit" or exceed the context window, the model doesn't delete files or conversations; it simply starts "forgetting" the earliest parts of the current chat to make room for new information. Here is how that works in practice: 1. The "Sliding Window" Effect Think of the context window like a rolling spotlight. * As the conversation gets longer and longer, the spotlight moves forward. * The messages at the very beginning of the chat might fall into the "shadows." * The Result: The AI might lose track of a specific detail you mentioned 50 messages ago, but the text of those messages is still visible to you in the chat UI. 2. Files and the Context Window If you upload a large file (like a 200-page PDF): * Storage: The file is saved to your chat history permanently. * Context: The AI "loads" that file into its context window so it can answer questions about it. If the conversation goes on for a very long time after that, the AI might "lose focus" on the file's details. However, most modern LLMs (like Gemini) have systems to "re-read" the file if you ask a question about it again. 3. "Forgetting" vs. "Deleting" It is important to distinguish between the two: * Forgetting (Context): The AI can't "see" the old data in its active memory anymore. This is temporary and only affects the AI's logic for that specific moment. * Deleting (Storage): This is the permanent removal of data. Exceeding the context window never triggers a deletion of your chat history or your uploaded files. Comparison at a Glance: | Feature | When Context is Full... | Is it Deleted? |