How I Cut Our AI API Bill by 95% — The Engineering Playbook
How I Cut Our AI API Bill by 95% — The Engineering Playbook When our finance lead forwarded me the AWS bill for March, I almost choked on my coffee. We were a team of nine engineers shipping AI features, and somehow we'd burned through enough on inference to cover two salaries. The worst part? I hadn't even noticed because the charges were scattered across OpenAI, Anthropic, and a couple of side experiments. That's the moment I decided to actually treat LLM spending like a real infrastructure problem instead of a credit card swipe. What follows is the playbook I wish I'd had on day one. These aren't theoretical tips — they're the exact moves I made across three products to get our run-rate down to roughly 5% of where it started, without shipping worse software. The Harsh Truth About Model Defaults Here's the dirty secret nobody tells you in the LLM hype cycle: most teams default to the most famous model for every single call. GPT-4o for everything. Claude Sonnet for everything. Then they wonder why their "simple AI feature" costs them a kidney. The model selection decision is where I recovered the majority of my budget. When you look at it rationally, the gap between the flagship tier and the cheap-tier models is absurd for tasks that don't require frontier reasoning. This is the matrix I landed on, and it still governs our routing today: Task What I Used To Use What I Use Now Cost Cut Simple chat GPT-4o ($10/M out) DeepSeek V4 Flash ($0.25/M) 97.5% Classification GPT-4o-mini ($0.60/M) Qwen3-8B ($0.01/M) 98.3% Code generation GPT-4o ($10/M) DeepSeek Coder ($0.25/M) 97.5% Summarization GPT-4o ($10/M) Qwen3-32B ($0.28/M) 97.2% Translation GPT-4o ($10/M) Qwen-MT-Turbo ($0.30/M) 97% I want you to really sit with the classification row. Qwen3-8B at $0.01 per million output tokens. That's sixty times cheaper than GPT-4o-mini. For a binary sentiment classifier, the accuracy difference in my benchmarks was under 1.5 percentage points. The ROI math isn't even close. The code