Chapter 2 (Part 2) Knowledge Retrieval Engine

2.6 Why Retrieval Is Necessary A Large Language Model only knows what was available during its training. If the user asks about something that changed after the model was trained, the model may not know the latest information. Instead of forcing the language model to guess, ACAI first determines whether external information is required. User Prompt │ ▼ Need External Knowledge? │ ┌────┴────┐ │ │ No Yes │ │ ▼ ▼ Continue Search Engine │ ▼ Document Ranking │ ▼ Source Selection │ ▼ Context Builder Internal Workflow The Retrieval Engine performs several sequential operations: Stage 1 — Query Generation The original user prompt is transformed into one or more optimized search queries. Example User Prompt Explain quantum computing. Generated Queries Quantum computing basics Quantum computing architecture Quantum algorithms Quantum hardware Instead of searching only once, multiple optimized queries increase the chance of retrieving relevant information. Stage 2 — Source Collection The retrieval system collects candidate documents. Possible sources include: • Internal Knowledge Base • Company Documentation • Scientific Papers • Technical Documentation • API Documentation • User Files • Vector Database Stage 3 — Ranking Not every document is equally useful. The Ranking Engine assigns a relevance score. Document A Score 96% Document B Score 91% Document C Score 72% Document D Score 44% Only the highest-quality documents move to the next stage. Stage 4 — Filtering The system removes: • Duplicate documents • Low-quality sources • Irrelevant information • Outdated documents The objective is to reduce noise before reasoning begins. 2.7 Context Optimization Layer One of the largest limitations of LLMs is the context window. Suppose the retrieval engine returns 500 pages. The model cannot efficiently process every page. Therefore ACAI introduces a Context Optimizer. Workflow 500 Pages ↓ Ranking ↓ Filtering ↓ Compression ↓ Important Facts ↓ LLM Instead of sending every token, only the