Multi-Provider LLM Router, or How I Got Tired of Forgetting Which API Format I Had To Use

If you've ever built an application that integrates with multiple LLM providers (Anthropic, Google, OpenAI, DeepSeek), you already know the pain: Each provider has its own distinct Python SDK. Streaming responses using Server-Sent Events (SSE) requires divergent parser logic. Thinking / Reasoning blocks are formatted completely differently. I recently extracted the core streaming router from my platform into an open-source FastAPI template. Here is how it works. Objective A single asynchronous endpoint: POST /v1/chat/stream It accepts a unified request payload and returns a standardized SSE stream emitting four clean events: event: thinking — Internal model reasoning tokens (streamed in real-time). event: content — User-facing response text. event: tool_call — Function calling requests. event: done — Stream completion ( [DONE] ). Architecture Instead of pulling heavy wrapper frameworks, use direct asynchronous HTTP via httpx.AsyncClient and the official Google GenAI SDK: fastapi-multi-llm-starter/ ├── app/ │ ├── config.py # Pydantic Settings loading environment variables │ ├── main.py # FastAPI app with CORS, health check & test playground │ ├── models.json # Dynamic model catalog (Claude, Gemini, GPT) │ ├── router.py # Unified multi-provider async stream dispatcher │ └── schemas.py # Strict Pydantic v2 validation models ├── tests/ # Automated unit tests (pytest) ├── requirements.txt └── README.md Dynamic Model Catalog I disliked the idea of hardcoded models, so I decoupled them into a models.json file: { "models" : [ { "id" : "claude-sonnet-5" , "name" : "Claude Sonnet 5" , "provider" : "Anthropic" , "thinking" : true }, { "id" : "gemini-3.8-flash" , "name" : "Gemini 3.8 Flash" , "provider" : "Google" , "thinking" : true }, { "id" : "gpt-5.6-terra" , "name" : "GPT 5.6 Terra" , "provider" : "OpenAI" , "thinking" : true } ] } Now, if you want to add another model, you just edit the JSON. The backend and the embedded UI dynamically populate available models via GET /v