We published how we measure our AI scribe's faithfulness, and built a checker anyone can run on any scribe's note

I founded Krasyn, an outpatient EMR with an AI scribe inside it. Krasyn has run a working outpatient clinic's real patient records since March 2026, so what our scribe drafts ends up in charts that real clinicians sign. This post covers two things we shipped in August: a published benchmark of how faithful those drafts are to the transcript, and Note Check, a tool that reads any scribe's note against its transcript and lists what the transcript does not support. Why a fluent note is the problem A faithful note and a note with one invented blood pressure look the same on the screen, and the clinician who signs it owns every sentence. Published evaluations put ambient-scribe hallucination at about 1 to 3 percent of notes. A March 2026 analysis of 71,173 AI-drafted and finalized note sections found a confirmed edit in 5.8 percent of them. The drafting got automated. The checking did not. I wanted a number for our own scribe that I could defend, with the definitions printed next to it. A benchmark without definitions is marketing. The unit: a clinical assertion We measure at the level of a clinical assertion, one atomic statement about the patient that could be true or false on its own. "Denies fever, chills, and nausea" is three assertions. A measurement and its value are one. Hedging is kept verbatim. Every assertion gets exactly one label against the transcript: Supported: the transcript says it, or it is a faithful paraphrase or clinical translation. Inferred: not stated, but a reasonable clinical inference with a basis in the transcript. Tracked separately because it is the contested category. Unsupported: no basis in the transcript at all. Contradicted: the transcript says the opposite, including a symptom the patient denied, a treatment the clinician declined, or another person's symptom attributed to the patient. Hallucination rate is unsupported plus contradicted over all assertions. Coverage is measured separately against key facts per case, because a note tha