基于深度学习的蒙古语AI合成主播

Mongolian AI Composite Anchor Based on Deep Learning

目前,汉文AI合成主播已被运用于新闻内容生产和传播,为传统新闻媒体行业开辟了新的发展路径,但蒙古语AI合成主播技术的研究仍处于起步阶段。为了研发蒙古语AI合成主播系统,本文采用深度学习技术提出了基于嘴型分类的蒙古语AI合成主播模型。首先采用ObamaNet模型构建了蒙古语AI合成主播基线系统,因为基线系统的时间开销大,提出了基于嘴型分类的蒙古语AI合成主播模型,此方法使用9种嘴型标签代表所有的嘴型状态,将得到的语音特征同步到不同的嘴型,根据得到的嘴型选择候选帧,得到AI主播合成视频。文章构建了蒙古语AI合成主播视频语料库,并以此为基础进行了实验比较。结果表明,文中提出的模型可以生成自然度较好的蒙古语AI合成主播视频。

At present,Chinese AI composite anchor has been used in the production and dissemination of news content,opens up a new development path for the traditional news media industry.However,the research on Mongolian AI composite anchor was still in its infancy.In order to develop the Mongolian AI composite anchor system,we use deep learning technology to propose a Mongolian AI composite anchor model based on mouth shape classification.First,the Mongolian AI composite anchor baseline system is constructed using the ObamaNet model.Because of the baseline system's large time cost,we proposed a Mongolian AI composite anchor model based on mouth shape classification.This method uses 9 mouth shapes to represent all mouth states,synchronizes the speech features to different mouth shapes,and selects candidate frames according to the mouth shapes to obtain AI anchor composite video.We also construct a Mongolian AI composite anchor video corpus,and make experimental comparison based on it.The experimental results show that the proposed model can generate Mongolian AI composite video with good naturalness.