Building AarogyaMitra: My 10-Day Journey Building a Voice AI Agent for Healthcare Access

From a Simple Voice Conversation to a Multi-Capability Healthcare Voice Agent Over the past 10 days, I had the opportunity to participate in 10 Days of Voice Agents — VoiceForBharat Edition , a challenge focused on learning how to build practical, real-world voice AI agents. Instead of treating the challenge as just a series of coding tasks, I wanted to build something around a problem that genuinely matters: making healthcare access more conversational and accessible through voice. That idea became AarogyaMitra — a voice-first healthcare access assistant designed to interact with users naturally, provide useful assistance, use tools when required, remember relevant user context, and involve humans or specialist agents when the situation requires it. This article documents my journey, the architecture behind the project, the important features I built, the challenges I faced, and what I learned while developing a real-time voice AI system. What is AarogyaMitra? AarogyaMitra is a voice AI assistant focused on the Health Access track of the VoiceForBharat challenge. The goal is simple: Make healthcare assistance more accessible through natural voice conversations. Many digital healthcare experiences assume that users are comfortable reading, typing, navigating menus, and interacting with conventional applications. Voice can provide a more natural alternative. Instead of searching through menus or typing a question, a user can simply speak to the assistant and have a conversation. AarogyaMitra is designed around this idea. The core objectives are: Make healthcare-related interactions more conversational Provide a simple voice-first interface Use AI tools when additional information or actions are required Maintain useful context during conversations Follow safety-oriented guardrails Escalate situations that require human assistance Route specialized requests to a specialist agent AarogyaMitra is intended to assist users, not replace qualified healthcare professionals .