Build a RAG Pipeline for Internal Runbooks with FastAPI and Chroma

Pipeline & Prompts | Byte size guides on DevOps, Cloud and AI AI in the Stack #2 ⚡ Byte Size Summary RAG inserts a retrieval layer between your existing runbooks and an LLM — answers come from your documentation, not generic training data, with source citations included. This article builds a complete FastAPI service with /ingest , /query , and /health endpoints, using OpenAI embeddings and Chroma as the vector store. Everything is cloneable from GitHub. The goal is not to replace your runbooks. It is to make them queryable at the moment an incident is happening. I have never met a platform team with bad runbooks. I have met plenty of platform teams where the runbooks exist, are reasonably well written, are stored somewhere sensible — and are still completely useless at 2am when something is on fire. Not because the content is wrong. Because nobody can find the right one fast enough. The search in Confluence returns fourteen results and none of them are titled the way the engineer is thinking about the problem. The person on call is junior and doesn't know the runbook exists. The runbook was written for a slightly different version of the service and nobody updated it. The runbook problem is not a writing problem. It is a retrieval problem. That is exactly the problem RAG was built to solve — and it is one of the highest-ROI first applications of AI in a platform engineering context. Not because it is technically impressive. Because it closes a gap that costs your team hours every month. This article builds a working pipeline. By the end you will have a FastAPI service that takes a natural language question — "why is my pod stuck in CrashLoopBackOff after a config change?" — and returns an answer grounded in your actual runbooks, with the source document cited. Everything is in the GitHub repo agentic-devops What RAG Is — Without the Hype RAG stands for Retrieval-Augmented Generation. Instead of asking an LLM a question and hoping its training data contains the answ