AI Fundamentals - Part 3: Giving AI Knowledge Beyond Its Training

In Part 2 , we learned why AI sometimes hallucinates. One of the biggest reasons is that an LLM can only answer based on what it learned during training and the information available in its context window. We also introduced grounding -providing the model with reliable information at runtime instead of expecting it to know everything. But that raises an important question: Where does that information come from? Modern AI applications don't simply dump an entire database or a thousand-page PDF into the prompt. Instead, they first identify the most relevant pieces of information and only send those to the model. In this article, we'll learn how that works. Running Example Let's continue building our AI-powered Travel Planner . So far, it can answer general travel questions using the knowledge it learned during training. Now we want to make it much smarter by uploading several documents into our application: Lonely Planet's Japan travel guide A PDF containing train schedules A document listing recommended local restaurants Hotel information Internal travel policies for our company Together, these documents contain hundreds of pages. Now the user asks: I'm staying near Tokyo Station. Which ramen restaurant from our travel guide is within walking distance and is known for vegetarian options? Somewhere in those hundreds of pages is the answer. The challenge is no longer generating text-it's finding the right information first. The Problem: An LLM Can't Read Your Entire Knowledge Base Every Time A common misconception is that AI applications simply send all their documents to the model. Imagine our travel guide contains 450 pages, thousands of restaurant listings, hotel descriptions, transportation details, and sightseeing recommendations. Sending all of that to the LLM every time someone asks "Where should I eat tonight?" creates several problems. First, many documents are simply too large to fit inside the model's context window. Second, even if they did fit, making the