Testing an LLM Input Layer for Poker Calculators: Verified Math, Unverified Interpretation

This article is about a poker-analysis framework, but the engineering problem is common to LLM tool use. The framework uses an LLM as an input and control layer. It reads a natural-language poker question, chooses a local calculator, and proposes typed fields. A Python program, not the LLM, performs the numerical calculation and returns a structured result with verification data. In this evaluation, a coordinator manually passed each calculator-eligible saved proposal to the command-line calculator; no automatic runtime bridge connected them. The design intent was to reduce manual arithmetic checking by sending numerical claims to deterministic, internally verified local software. The evaluation below tests whether those claims were checked and whether the handoff remained auditable. It did not measure time saved or the overall quality of the resulting poker analysis. The calculator catalog is poker-specific. This is not a poker strategy guide, and you do not need to know poker strategy to follow the failure. The question is whether a correct calculator can produce a verified result after the LLM chooses an interpretation without asking the user to confirm it. The workflow was tested with 25 hand-authored cases that were fixed before execution. They are labeled C01 through C25: C01–C23 tested the LLM's routing, proposed input, and boundary decisions; C24 and C25 repeated two accepted inputs to check non-volatile result semantics. The labels are test numbers, not poker terminology. The main example uses one small pot-odds model. In this model, the pot is the shared pool of chips the players are competing for. The calculator's inputs are: pot_before_bet : the amount already in the pot before the opponent's new bet; opponent_bet : the amount the opponent adds; call_cost : the amount the player must add to continue; expected_rake : an optional amount removed from the final pot. The calculation is: net final pot = pot_before_bet + opponent_bet + call_cost - expected_rake