Token Drift Explained: Why Your Agent Gets Slower and More Expensive

Your agent feels fast during a demo. Then a real session reaches twenty turns, several tools have returned large payloads, and every response starts taking longer and costing more. This pattern is often called token drift : the effective input context grows as an agent carries more conversation history, tool output, retrieved documents, and state into each model call. The model itself is not gradually becoming less efficient. The application is asking it to process more material on every turn. Token drift is manageable, but only if context is treated as a budgeted system resource rather than an unlimited transcript. What Token Drift Actually Means Most conversational agents build each request from several sources: a system prompt, tool definitions, recent messages, retrieved context, durable memory, and sometimes a summary of older work. Whether the application sends that state on every request or a provider manages part of it, the model still has an effective context to process. Consider an illustrative session: Turn Effective input What changed 1 1,200 tokens System prompt, tools, and one user message 8 6,900 tokens Conversation history and two tool results 20 18,400 tokens More history, retrieved documents, and accumulated state The exact price and latency depend on the model, provider, cache behavior, and workload. The important signal is the trend: later calls repeatedly process a larger context. Why Agents Accumulate Tokens So Quickly Conversation history is only one source of growth. Production agents often accumulate tokens in several places at once: Repeated transcripts: Every prior user and assistant message remains in context. Tool schemas: Large tool descriptions and JSON schemas may be attached to every model call. Tool results: Search results, stack traces, database rows, and API responses can be much larger than the user's request. Retrieved documents: RAG pipelines sometimes add too many chunks or keep stale retrieval results across turns. Retries an