Agent Memory & Context Engineering

How agents remember - and why deciding what to forget is the real skill An agent that starts every step with a blank mind cannot really pursue a goal. It would reintroduce itself to you on every message, forget what it just tried, and repeat the same mistake forever. Memory is what turns a stateless model into something that accumulates - that knows who you are, what it has already done, and what it learned last Tuesday. This post is about how that works and, more importantly, about the discipline of deciding what an agent should remember at all. The context window is not memory. The first thing to unlearn: a model’s context window is not its memory. The context window is working memory - RAM, not a hard drive. It is finite, it is reset on every request, and every token in it costs money and dilutes the model’s attention. Stuffing an entire conversation history and knowledge base into the prompt does not scale, and past a point it actively hurts - the model loses the important signal in a sea of stale detail. Real memory lives outside the window and is selectively loaded into it when needed. Four kinds of memory Borrowing loosely from cognitive science, agent memory is usually split into four types, and good systems use all of them: Short-term/working memory - the current conversation and the agent’s recent thoughts and observations. Lives in the context window. Long-term episodic memory - a record of what happened : past conversations, decisions, and the outcomes of previous tasks. Long-term semantic memory - facts and knowledge: who the user is, domain information, documents. This is what retrieval-augmented generation pulls from. Procedural memory - how to do things : learned skills, tool-use patterns, and reusable strategies. Short-term memory: the rolling buffer The simplest memory is just keeping recent turns in the prompt. The problem is that conversations outgrow the window, so the standard move is to keep the last few turns verbatim and summarise the older