I Built an AI That Turns 2 Hours of Compliance Paperwork Into 3 Minutes — Full Architecture Teardown

Financial advisors have a dirty secret: they spend almost half their working hours not advising anyone. The culprit? Compliance documentation. After every client meeting, advisors must document what was discussed, what was recommended, whether those recommendations were suitable, and whether they followed FINRA and SEC rules — all in a format their CRM can ingest. A 45-minute meeting routinely generates 2 hours of paperwork. I built an open-source tool that does it in about 3 minutes. Here's exactly how — every architectural decision, every trade-off, and every line of code that matters. The Problem Is More Specific Than You Think When I started talking to advisory firms, I expected "meetings take too long" or "we need better CRM software." Instead, every compliance officer said the same thing: "We're not worried about the notes. We're worried about what's NOT in the notes." The real pain isn't documentation speed — it's the compliance gap. If a client says "I can't afford to lose this money" and the advisor recommends an aggressive growth fund, that's a FINRA 2111 suitability violation. But if the note-taker (usually the advisor, writing from memory hours later) forgets that quote? No record of the red flag. This changed my entire system design. It's not a transcription tool with formatting. It's a compliance engine that listens for mismatches. Architecture Four-stage pipeline: Audio → Transcription → Structured Extraction → Compliance Check → CRM Note (Whisper) (Claude via (Rule engine) (Formatter) OpenRouter) Stack: Python/FastAPI + React frontend + Whisper (local) + Claude via OpenRouter Two key design choices: Whisper runs locally. Advisory meetings contain PII and legally privileged information. Sending audio to third-party APIs isn't optional for most firms — it's a regulatory non-starter. Compliance engine is NOT an LLM. You can't have a probabilistic system making deterministic compliance judgments. The compliance check uses hardcoded rules against structur