今日已更新 32 条资讯 | 累计 27207 条内容
关于我们

The cache key that ignored the question

EazyHood 2026年08月02日 23:47 0 次阅读 来源:Dev.to

Two people asked a context compressor two completely different questions. It gave them the same answer. Not a similar answer — byte for byte the same 544 characters. Here's what that looked like: query="Fix the IntegrityError on commit" level=L0 -> 159 tok cache_hit=False query="Explain the tax rounding TODO in compute_tax" level=L3 -> 159 tok cache_hit=True identical output: yes (544 chars both) Different question. Different compression level. Same 544 characters, served from cache. Finding it I wasn't looking for this. I was auditing something else entirely — measuring how much meaning a context compressor loses, not how fast it runs. My harness feeds the same corpus through the compressor with different queries and checks which critical substrings survive: file paths, error types, line numbers, identifiers. I noticed two rows in my results table were identical. Same token count, same output. My first assumption was that my own harness had a bug — that I was passing the same query twice and hadn't noticed. So I changed the second query to something with no words in common with the first, and bumped the compression level from L0 to L3, which should change the output dramatically on its own. Same 544 characters. That was the moment it stopped being my bug. The cause One line: sid = content_hash(content) That sid was doing two jobs. It was the shadow ID — the handle used to refer to a stored document. And it was also the cache key. As a shadow ID it's correct: the same content should get the same handle. As a cache key it's wrong, because the output of compress() doesn't depend only on the content. It depends on the content and the query and the compression level. Two of those three inputs were simply not part of the key. So the first caller warmed the cache for a piece of content, and everyone who touched that same content afterwards got the first caller's answer — regardless of what they actually asked for. Why this is worse than a stale cache A stale cache gives y

本文内容来源于互联网,版权归原作者所有
查看原文