How to Test an AI Agent's Tool Selection Without Trusting Its Own Logs

You have built an AI agent harness. It calls tools, routes requests, and returns results. Your team trusts its telemetry to tell you which tool was selected and why. That trust is a liability. An agent's own logs are self-reported. They tell you what the agent thinks it did, not what actually happened. A hallucinated tool name, a misrouted parameter, a silent fallback to a different function — none of these surface in the agent's own trace. You need an external witness. Here is how to build one. The Problem: Self-Reported Truth Is Not Truth Most teams validate agent behavior by reading the agent's own output. They check the tool_calls field in the response, match it against an expected schema, and call it done. This works until it doesn't. Consider a common failure mode: the agent decides to call search_knowledge_base but the LLM formats the tool name as searchKnowledgeBase . The routing layer silently normalizes it, the call succeeds, and the agent logs search_knowledge_base . Your test passes. The actual execution path was different from what you verified. Another pattern: the agent selects the correct tool but passes a parameter that the tool silently coerces. A date string gets parsed into a different timezone. A user ID gets truncated. The tool returns a result, the agent logs success, and your test never catches the drift. The root cause is the same. You are testing the agent's intent , not its execution . Intent is cheap to fake. Execution leaves fingerprints. The Solution: An External Observer You need a layer that sits between the agent and the tools it calls. This observer records every invocation — tool name, parameters, response, latency — without the agent knowing it is being watched. The observer does not trust the agent's logs. It trusts what it sees on the wire. Here is the architecture at a high level: Intercept every outbound call from the agent to a tool. Record the raw request before any normalization or routing. Compare the recorded call against