How I Cut My LLM API Bill by 80% With a Simple Router

No fancy infrastructure. Just a 50-line Python function that picks the right model for the right query. Last month my LLM API bill hit $340. This month: $67. Same traffic. Same product. The only change was adding a simple router that stops sending every request to Claude Sonnet when GPT-4o mini can handle it just as well. Here's exactly how it works. The Problem When you prototype, you pick one model and hardcode it everywhere. Usually something capable like GPT-4o or Claude Sonnet, because you want good results fast. Then you ship, traffic grows, and you get a bill that makes you question your life choices. The thing is — not all queries need a flagship model. In a typical RAG app: "What is the return policy?" → GPT-4o mini handles this fine "Summarize these 5 conflicting documents and identify the key disagreement" → needs Sonnet You're paying Sonnet prices for return policy questions. That's the bug. The Fix: A Complexity Router import anthropic from openai import OpenAI openai_client = OpenAI() anthropic_client = anthropic.Anthropic() def classify_complexity(query: str) -> str: """Returns 'simple' or 'complex'.""" simple_indicators = [ len(query.split()) < 15, query.endswith("?") and query.count("?") == 1, not any(w in query.lower() for w in [ "compare", "analyze", "summarize", "explain why", "difference between", "pros and cons", "evaluate" ]) ] return "simple" if sum(simple_indicators) >= 2 else "complex" def route(query: str, context: str = "") -> str: complexity = classify_complexity(query) if complexity == "simple": # $0.15/M input — GPT-4o mini response = openai_client.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "system", "content": context}, {"role": "user", "content": query} ] ) return response.choices[0].message.content else: # $3.00/M input — Claude Sonnet (only when needed) response = anthropic_client.messages.create( model="claude-sonnet-4-6", max_tokens=1024, system=context, messages=[{"role": "user", "content": query}] ) retur