Building AI Agents That Don't Hallucinate: Structured Workflows, Guardrails, and Per-Step Evaluation
Building AI Agents That Don't Hallucinate: Structured Workflows, Guardrails, and Per-Step Evaluation How we replaced fragile prompt chains with typed schemas, validation gates, and evaluation at every step — 94% task success vs 60% baseline The Prompt Chain Trap January 2024. We built a "research agent" — 12 prompts chained together: Decompose question → 2. Search planning → 3. Execute searches → 4. Extract facts → 5. Synthesize → 6. Fact-check → 7. Format → ... It worked 60% of the time. The other 40%: Step 3 returned malformed JSON → Step 4 crashed Step 5 hallucinated citations → Step 6 missed it Step 7 output wrong format → Downstream consumer failed No visibility into which step failed Debugging meant reading 12 LLM calls' worth of logs. Adding a step broke three others. The Shift: Agents as Typed Workflows We moved from prompt chains to structured workflows with: Pydantic schemas for every step input/output Guardrails that validate and auto-retry Explicit state machine (not implicit chaining) Evaluation harness per step (not just end-to-end) ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Decompose │──▶│ Search │──▶│ Extract │──▶│ Synthesize │ │ Question │ │ Planning │ │ Facts │ │ Answer │ │ │ │ │ │ │ │ │ │ In: Query │ │ In: Plan │ │ In: Results │ │ In: Facts │ │ Out: SubQ[] │ │ Out: Steps │ │ Out: Fact[] │ │ Out: Answer │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ │ ▼ ▼ ▼ ▼ [Schema] [Schema] [Schema] [Schema] [Guardrail] [Guardrail] [Guardrail] [Guardrail] [Eval: 0.9] [Eval: 0.85] [Eval: 0.9] [Eval: 0.95] Core Abstractions # agent_eval/schemas.py from pydantic import BaseModel , Field from typing import Literal , Any class DecomposeInput ( BaseModel ): user_query : str context : dict = Field ( default_factory = dict ) class DecomposeOutput ( BaseModel ): sub_questions : list [ str ] = Field ( min_length = 1 , max_length = 5 ) requires_tools : bool reasoning : str class PlanInput ( BaseModel ): sub_questions : list [