基于多尺度特征融合的语音频带扩展

Speech bandwidth extension based on multi-scale feature fusion

针对现有的深度学习模型在语音频带扩展领域数据特征利用不充分、训练周期长以及生成语音质量不高等问题,提出了一种新型的端到端神经网络模型,该模型通过融合不同数据维度特征促使网络模型利用更少的数据特征量,获取更多的低高频映射关系解,从而减少模型的整体训练周期.为了提高长时序数据中关键特征的权重占比,设计了一种残差多头自注意力机制,从而达到数据特征利用率的最大化.此外,提出了一种基于时频域和Mel频谱的混合损失函数对模型进行优化.实验结果表明:该方法重构的宽带语音在主客观的评价中均优于传统方法和近年来的一些基于神经网络的语音频带扩展方法.

A new type of end-to-end neural network model was proposed by us,with the aim of addressing the issues of insufficient utilization of data features,long training periods,and low quality of generated speech by existing deep learning models.The integration of features from different data dimensions within the network model was encouraged,thereby reducing the utilization of data features and obtaining more low-and high-frequency mapping relationship solutions,resulting in a shortened overall training cycle of the model.To maximize the utilization of data features and enhance the weight ratio of key features in long time series data,a residual multi-head self-attention mechanism was designed.Additionally,a hybrid loss function based on the time-frequency domain and Mel spectrum was proposed to optimize the model.The wideband speech reconstructed by this method was evaluated subjectively and objectively,and the experimental results indicate that it is superior to traditional methods and some recent neural network-based speech bandwidth extension methods.