The stricter the quality gate, the more monotone readings survive — Selection bias caused by verification

📝 Originally published (in Japanese) at forge.workstyle.tech . Building an Emotion-Expressive TTS Model: How Quality Gates Can Backfire I was working on an automated pipeline to generate training data for an emotion-expressive TTS model. For each of 12 emotions (joy, sadness, anger, fear, etc.), I prepared several audio clips with the target emotion applied. Naturally, this required quality control. I used Whisper to transcribe the generated audio and only kept clips that matched the script exactly. The resulting model ended up sounding flat and monotone. The issue turned out to be the quality control process itself. Switching Emotion Styles Still Sounds Like the Same Voice This model has distinct "styles" for each emotion—joy style, sadness style, etc.—and can synthesize speech while switching between them. However, when switching styles, the perceived difference in the audio is minimal. The numbers made this painfully clear. I measured the cosine similarity between each emotion-style synthesis and a neutral style. If the emotion was properly applied, the similarity should decrease (i.e., the value should be smaller). Bulk-generated corpus: cos 0.77–0.94 A well-made individual: cos 0.164 A cosine similarity near 0.9 means that even when using the "joy" style, the output sounds almost identical to the neutral voice. The emotion styles were effectively non-functional. Honestly, when I first listened to the samples, I thought, "Eh, it's fine." It wasn't until the numbers showed 0.9 that I realized something was seriously wrong. Three Root Causes Two of these were configuration issues, but the third is the real culprit. 1. Speaker CFG Settings Cause Interjections to Sound Like a Different Person When scripts started with interjections like "Waa!" or "Eh!" , the beginning of the clip would sound like a completely different voice. The parameter controlling fidelity to the reference audio was breaking speaker consistency when emotion was applied. 2. Emojis Get Read as Aud