今日已更新 133 条资讯 | 累计 29586 条内容
关于我们

Default-to-Flagship Is Now a Cost Bug: Tiered Model Routing for Agentic Workloads

ai maya 2026年08月09日 08:17 1 次阅读 来源:Dev.to

For two years the reflex was simple: reach for the biggest model you can afford and call it a day. In 2026 that reflex quietly became a bug in your cost model. The clearest signal came this summer, when a smaller, cheaper "flash"-tier model started edging out its own flagship sibling on the workload developers care about most — multi-step agentic coding — at a fraction of the price. When the fast tier wins the hard benchmark, "always use the flagship" stops being a safe default and starts being waste. Here's how to fix it without turning your stack into a science project. Why the reflex is expensive Agent workloads are not one big call. A single task fans out into dozens of small ones: planning, tool selection, argument formatting, summarizing a file, deciding whether to continue. Most of those steps are easy . Routing every one of them through a frontier model is like taking a helicopter to the corner store — it works, but you are paying helicopter prices for a walk. The trap is that the cost is invisible per call and enormous in aggregate. You never see the moment you overpaid; you just see the invoice. The three-tier ladder Think in tiers, not models: Cheap/fast tier — classification, extraction, short rewrites, routing decisions, "is this done?" checks. Most steps live here. Mid tier — normal reasoning, code edits, tool use with moderate context. Flagship tier — genuinely hard reasoning, long-context synthesis, the step where a wrong answer poisons everything downstream. The goal is to keep the flagship tier for the 5–15% of steps that actually need it, and let the cheap tier carry the volume. How to decide the tier per request Two mechanisms, used together: Static heuristics for the obvious cases. Short prompt + structured output + low stakes → cheap tier. Anything touching a large context window or a irreversible action → escalate. Eval-gated escalation for everything else. Start at the cheap tier, and only promote to a bigger model when your evals prove the c

本文内容来源于互联网,版权归原作者所有
查看原文