Small Models, Great Tools: The Engineering Behind a Local AI Agent in Production

There is a persistent myth that to build a worthy code assistant, you absolutely must use GPT or Claude. This is false. You don't need a 1-trillion parameter model. You need a small local model and extremely rigorous engineering around it. This is the direction history is taking for companies. As Mark Zuckerberg mentioned, the future isn't a single omniscient model, but "every company having its own specialized AI" . And this specialization necessarily involves fine-tuning and local deployment (or on sovereign servers) to guarantee data security. The thesis behind the construction of Vibrisse Agent can be summed up in one sentence: Small models, Great tools. In this article, I will detail the technical stack and concrete engineering solutions I implemented to tame a local model and make it reliable in production: LangGraph, Ollama, FastAPI, React (no build step, with embedded custom CSS) , all running on a machine with 32 GB of RAM. For the curious who want to run the agent on their machine right now: // MacOs / Linux curl -sSL https://agent.vibrisse-studio.dev/install.sh | bash // Windows irm https://agent.vibrisse-studio.dev/install.ps1 | iex Architecture: Why a State Machine (LangGraph)? At first, when building an LLM application, we tend to think in sequential chains: Input -> Prompt -> Tool -> Output . The problem is that if one node fails, the whole chain stops without us being able to catch the error or understand the context of the crash. That's where LangGraph comes in. Vibrisse's architecture isn't a chain, it's a state machine . Every node in the graph has a very precise responsibility, shares a global conversation state, and uses conditional transitions to move to the next node. I implemented the Supervisor / Worker pattern: The Supervisor analyzes the user's intent. It does nothing else but route. It dispatches the task to specialized Workers (the RAG Worker, the Search Worker, the Ghost Worker...). If a Worker fails or needs more information, it can se