Building a Hybrid RAG System with FAISS, BM25, and Agentic AI

As part of my AI Engineering journey, I recently worked on a project that helped me understand how Retrieval-Augmented Generation (RAG) works in practice. I built a Hybrid RAG system that combines FAISS vector search and BM25 keyword search to retrieve relevant information from a knowledge base and use it to generate grounded answers. In this post, I’ll briefly share what I built, how the system works, and some of the things I learned along the way. Why RAG? Large Language Models are great at generating natural-language responses, but they may not have access to information contained in a specific document or knowledge base. RAG addresses this by first retrieving relevant information from an external knowledge base and then providing that information to the LLM as context. The basic workflow is: User Query ↓ Retrieve Relevant Information ↓ Provide Context to LLM ↓ Generate Answer For my project, I wanted to take this a step further by combining semantic search and keyword search. 🔍 Hybrid Retrieval The system uses two retrieval methods: Vector Search with FAISS Document content is divided into smaller chunks and converted into vector embeddings. These embeddings are stored in a FAISS index, which is used to find documents that are semantically similar to the user’s query. This is useful even when the query and the document use different wording. Keyword Search with BM25 The second retrieval method is BM25. BM25 focuses on the occurrence and importance of terms in the query and documents. This makes it useful for exact terminology, technical terms, names, and identifiers. Instead of depending on only one retrieval method, both approaches are combined. User Query │ ┌──────────┴──────────┐ ↓ ↓ FAISS Search BM25 Search Semantic Search Keyword Search │ │ └──────────┬──────────┘ ↓ Hybrid Ranking ↓ Relevant Context ↓ LLM ↓ Final Answer The FAISS and BM25 scores are normalized and combined using weighted scoring. The results are then ranked, and the highest-ranked chunks ar