今日已更新 339 条资讯 | 累计 19899 条内容
关于我们

The LLM Should Never Do the Math

Jeremy Longshore 2026年06月29日 21:00 2 次阅读 来源:Dev.to

A CFO will not act on a number an LLM eyeballed. They will not act on a number the model "estimated" by reasoning over a usage dump. And they should not — because the moment a language model emits a dollar figure it computed itself, that figure is a guess wearing the costume of a fact. This is the design constraint behind databricks-cost-leak-hunter , the pilot skill of the databricks-pack v2 rebuild shipped in the claude-code-plugins marketplace ( PR #906 ). Given a live, authenticated Databricks workspace, it surfaces real cost leaks across four named categories, ranks them by monthly dollar impact, and emits a report a finance reader can act on. The marketplace validator graded it B (88/100, zero errors). The SKILL.md is 329 lines. The single most important thing in it is a rule the model is structurally prevented from breaking: the LLM never does the dollar arithmetic. Why not just let the agent read the bill and summarize it? Because that is exactly how you ship a confidently wrong cost report. Hand a model a few thousand rows of system.billing.usage and ask it for the top cost leaks, and it will give you a fluent answer. It will add DBUs. It will multiply by a price it half-remembers. It will round. Every one of those steps is a place the model can be plausibly, invisibly wrong — and the output reads identically whether the math is right or hallucinated. The failure mode of an LLM doing FinOps is not a crash. It is a clean, well-formatted, wrong number. The fix is architectural, not prompt-engineering. The model is allowed to decide what to look for and how to explain it . It is never allowed to be the calculator. The dollar primitive: confirmed, never estimated Every confirmed figure comes from the customer's own billing tables — system.billing.usage joined to system.billing.list_prices . Not a model estimate. Not a public price list. The number Databricks actually billed. That join is defined once, as a priced CTE, and reused by every category query. Usage i

本文内容来源于互联网,版权归原作者所有
查看原文