Why WhatsApp voice notes break general-purpose transcription
Most speech-to-text is benchmarked on audio that looks nothing like a WhatsApp voice note. The standard evaluation sets are read speech, broadcast news, or recorded interviews: single speaker, decent microphone, one language, quiet room, speaker aware they are being recorded. A WhatsApp voice note is close to the opposite on every axis. I have spent a while building around this, and the gap turned out to be wider than I expected. Acoustics Phone held at arm's length while walking, in a car, in a kitchen, on a street. Distance-to-mic varies wildly within a single recording , which breaks a lot of assumptions about consistent gain. Then there is the codec. Voice notes are Opus at low bitrate — efficient, but it discards exactly the high-frequency detail that helps disambiguate fricatives. /s/ versus /f/ versus /th/ get genuinely harder, and those distinctions carry real meaning. Register Conversational, not read. False starts, self-corrections, filler, trailing off mid-sentence, and long pauses that are not sentence boundaries — someone thinking, or getting distracted. Punctuation inference is much harder here than on read speech. And punctuation is most of what makes a transcript skimmable rather than a wall of text. A perfectly accurate word sequence with no paragraph breaks is close to useless if the point was to let someone read it faster than listening. Language This is the one that surprised me most. Voice notes are heavily code-switched. People drop English technical terms into Urdu, Hindi, Arabic, Spanish sentences constantly — not as an edge case, as the default register for a huge number of speakers. If you force a single language selection up front, you mangle every mixed utterance. Auto-detection is not a convenience feature in this domain. It is a correctness requirement. Length distribution Most notes are 5–45 seconds. Very little context to work with, and per-request overhead dominates if you architected for long files. Batching strategies that make sen