When Four Memory Systems Hit the Same Wall
I built a knowledge graph out of my own work sessions. Hundreds of them — transcripts of me building a system with LLMs, extracted into concepts, decisions, findings, and the edges between them. For a while it felt like the thing was working. I'd query it, get back a clean structured answer, and move on. Then I ran a foreign model against it. I gave a different model my concept definitions and asked it to reconstruct the system, both the vocabulary and the relationships. It recovered 97.7% of the words. It recovered 61.1% of the structure. That 36-point gap was the first time I could see the problem instead of just living inside it. The vocabulary transferred because the definitions were written carefully. The edges didn't, because the edges were the part I'd let the extraction handle. And the whole time, querying the graph had felt complete. The structure came back typed, connected, confident-looking — so I stopped looking. I started calling it premature retrieval closure: the retrieval returns something shaped like a whole answer, which is exactly why I didn't notice the parts that were missing. Part 10 of Building at the Edges of LLM Tooling . If you're running a long-term project through an LLM-backed memory system (anything that turns raw sessions into structured, persistent memory), this is about the step where the structure starts lying about how complete it is. Start here . Why It Breaks Every memory system of this kind does the same move. An LLM reads raw interaction (a conversation, a document, a session log) and lifts structured memory out of it: entities, facts, rules, summaries. That structured memory becomes the thing the agent reads later, instead of the raw record. The lift is where fidelity goes. Pulling clean structure out of messy text means making decisions the text didn't make explicit: which entity this pronoun refers to, whether a relationship is real or inferred, what to keep and what to drop. Those decisions can be wrong, and when they are,