18 Insights from Mass-Producing Voice Models — From Diffusion TTS Voice Design to Training Corpus Creation and Quality Gate Pitfalls

📝 Originally published (in Japanese) at forge.workstyle.tech . This is a record of designing voices from single-line captions, automatically creating a learning corpus, and passing all 12 role-specific voices (narrator/counselor/sales/presenter/operator/MC for both men and women) through full inspection. I wrote about the failures I encountered during approximately one month of actual work, divided into 18 articles. This article is the table of contents. The Conclusion Upfront Voice design, voice manufacturing, and voice operation are different technologies with different failures. Design uses diffusion TTS. The voice is determined by the caption and random seed, making it fully reproducible. Manufacturing is primarily about corpus generation. The design of the quality gate directly determines the voice quality. Operation relies on lightweight pre-trained models. Diffusion TTS is too slow for conversation (2.5 times slower on the same GPU). The biggest lesson boils down to one point: Having a quality gate and it being effective are two different things. Six of these 18 articles are about gates that existed but weren’t effective. Reading Order The articles are arranged in the order of design → manufacturing → inspection → operation. Reading from the top will take you through the journey of a single voice being created and deployed into production. Chapter 1: Design — How to Determine the Voice The TTS Chosen for Sound Quality Was Too Slow for Conversation A 2.5x real-time factor (RTF) difference. How we settled on a two-stage approach: designing voices with diffusion TTS and using pre-trained models for speech. Drawing Voices Like a Gacha Voices are determined by captions and random seeds. By keeping a ledger of design values, voices can be recreated even if the model is lost. Letting a Machine Choose "Narrator-like Voices" from 24 Candidates Listening to all candidates is unsustainable. Automatically measure speech rate, intonation, and stability to only listen to t