Stale RAG vs. expensive RAG: how to cache RAG context without serving outdated answers

If you run a RAG system in production, you eventually hit a dilemma that has nothing to do with your model and everything to do with your cache. Cache the answers to save tokens and latency, and one day a source document changes — but your cache keeps cheerfully serving the answer it built from the old document. Nobody gets an error. The number is just quietly wrong. Cache nothing , and every single call re-retrieves the same chunks, re-reads them, and re-pays the full context bill to rebuild an understanding you already built five minutes ago for a nearly identical question. Stale or expensive. Most teams pick "expensive" because at least it's correct, then bolt on a TTL and hope. This post is about why the TTL doesn't save you, and about two specific, mechanical fixes that let you cache RAG context and stay fresh. I maintain an open-source library called Coalent that implements both, so I'll use it for the runnable examples — but the two ideas are portable and worth stealing even if you never pip install anything. Failure mode 1: the stale RAG cache (and why a TTL won't save you) Here's the standard "answer cache" sitting in front of retrieval: answer = cache . get ( query ) if answer is None : chunks = retriever . retrieve ( query ) answer = llm . synthesize ( query , chunks ) cache . set ( query , answer , ttl = 3600 ) return answer This works until billing.md changes. The refund window goes from 30 days to 14. Your cache has an answer keyed on "what is our refund policy?" that says 30, and it will keep saying 30 for up to an hour — or forever, if the same question keeps refreshing a TTL that never expires under load. The reason this is hard is that the cache key (the query) has no relationship to the thing that changed (the source). You cached an answer; you threw away the fact that this particular answer was derived from billing.md . So when billing.md changes, you have no way to find the answers that depended on it. The TTL is a confession that you can't answ