why a simple string match beat apple's nlembedding for local rag
Why a simple string match beat Apple's NLEmbedding for local RAG how apple's nlembedding drove me crazy and how i built my own hybrid search engine recently, while working on my personal ai agent (pheronagent), i was focused on perfecting its memory and retrieval system. everyone is talking about that famous acronym: rag (retrieval-augmented generation). the system is simple: i feed the agent my documents, it converts them into vectors (embeddings), and when i ask a question, it finds the most similar vectors and answers me. sounds perfect on paper, right? so, like any loyal apple ecosystem developer, instead of downloading massive models from external sources (or burning money on apis), i decided to use nlembedding—the native capability of the operating system that runs directly on-device. after all, apple had embedded this into the os; it was both fast and privacy-focused. but real life, as it turns out, doesn't progress as smoothly as wwdc presentations... where have i worked? - the first explosion it all started with a very innocent question. i had uploaded my cv to the system. while chatting with my agent, i casually asked: "where have i worked?" i expected the agent to fire up the metal cores in the background within seconds, find my cv, and list the companies for me. instead, the agent stared blankly. i opened the logs to see what the hell the search engine was doing behind the scenes. the shocking scenario was exactly this: cosine similarity between the query and my actual cv text: 0.587 the threshold i set for relevance: 0.60 it missed it by a hair! "no worries," i thought. "we can just lower the threshold a bit, make it 0.55, and call it a day." but then i saw the truly terrifying thing just one line below. for the exact same query, guess what score a completely irrelevant, junk record in the system—a list of files containing .ds_store—got? 0.59 - 0.60! wait a minute... my detailed, multi-page resume gets a score of 0.587 just because it doesn't contain th