Migrating an Agentic RAG App to AWS Serverless

A previous post covered building an agentic RAG router over **LLM Zoomcamp * capstone submissions. This series is about moving it off a single EC2 box and onto AWS serverless, where nearly every obvious choice turned out to be wrong, and the most valuable decision was deleting a database rather than migrating it. This part covers the starting point and the resulting architecture; The course on designing and creating a RAG project has concluded. The final project is working as expected. Now it's time to optimize and use the developed concept as a production system. In the previous article, I described creating an agent-based RAG router for processing LLM theses submitted at Zoomcamp. I decided to write a series of articles dedicated to migrating the project from a single EC2 server to a serverless AWS environment. This section examines the starting point and the resulting architecture. Starting point and its drawbacks The initial assistant is working. It answers fundamentally different types of questions about these repositories. For example, "What does this project do?" using a hybrid search of README files, and "various questions about performance statistics and libraries used in projects?" using a precise parameterized SQL query. The agent router chooses between them. In my capstone, it runs as six Docker Compose containers on a single t4g.small instance, deployed via SSH, with a 2GB swap file as a cheap safety net so that Postgres, Streamlit (which contains the Torch embedder and Torch cross-encoder in RAM), and Grafana can coexist on a 2GB RAM instance. This isn't bad for a capstone project. But it has properties I'd like to eliminate for a real system. The corpus - 24,775 fragments, 115MB of JSON-is loaded and rebuilt into the process's RAM on each run. Two local machine learning models require a 1.7GB virtual environment and an image. Several gigabytes in size. There's no infrastructure as code at all. This approach is completely unsuitable for a real system.