完全端到端的藏语语音合成方法

An End-to-End Tibetan Speech Synthesis Method

在迈向多语言多模态大模型的时代下,藏语语音合成技术的研究意义更加凸显,其目的是将文字信息转化为可听的声音信息,使得人机交互更加便捷和人性化。该文针对目前主流的两段式端到端藏语语音合成方法出现重复吐词、跳词、漏词等鲁棒性差且推理速度慢的问题,研究了基于深度生成模型的完全端到端VITS模型及其在藏语语音合成中的应用。首先,通过自然语音采集、自动标注和声学分析等构建一个7000条中等规模的藏语卫藏方言语音数据库;其次,由于现有开源模型不能很好地表征藏文音节结构特征,且现有相关描述无法全面地刻画藏语语音结构,为此提出了现代藏文存在7种字形结构的主张,并将其转写成对应的音素序列作为模型的输入;最后,经典的开源模型VITS应用在上述语音数据上进行了藏语语音合成试验。同时,为了提高合成系统的鲁棒性,在模型中引入了预训练的音素强制对齐信息。实验结果表明,相比两阶段的方法,通过端到端建模不仅有效减少了模型的推理时间,进一步提高合成语音的质量,而且基于7种藏文字形结构的音素序列作为建模单元,显著提升藏语音段特征的覆盖率,从而缓解低资源且黏着语常见的数据稀疏带来发音错误,以及音素分散导致模型训练困难问题。

The Tibetan speech synthesis is to convert text information into audible sound information.To further improve current Tibetan speech synthesis with such issues as repeated words,skipped words and missed words,this paper studies the application of a complete end-to-end VITS model in Tibetan speech synthesis.First,a database of 7000 medium-scale Tibetan dialect speech is constructed through natural speech acquisition,automatic annotation and acoustic analysis.Second,we propose seven font structures to better represent the Tibetan phonetic and syllabic structure,and translate them into corresponding phoneme sequences as the input of the model.Finally,a typical open source model VITS is applied with pre-trained phoneme to force alignment information.The experimental results show that compared with the mainstream method,the end-to-end model effectively improves the quality of synthesized speech with a reduced reasoning time.