基于并行多注意力的语音增强网络

Speech Enhancement Network Based on Parallel Multi-Attention

针对受干扰语音的频域增强问题,提出一种基于并行多注意力机制和编解码结构的语音增强网络(PMAN)。网络输入经过短时傅里叶变换(STFT)的语音频域特征,包含振幅谱和复数谱,编码器使用密集卷积模块对输入数据信息进行整合,中间层的并行多注意力模块学习频域的局部和全局信息,并融合局部块注意力(LPA)机制捕捉语音频域二维(2D)结构,实现干净语音与干扰因素的2D层面分离。解码器将学习到的信息进行整合,分别生成振幅掩模和复数频谱,根据加权求和生成最终的语音复数频谱,使用时域与频域联合损失函数实现相位信息的融合。在VoicеBank+DEMAND语音数据集上的实验结果表明,与基于两阶段变换器的时域语音增强神经网络(TSTNN)相比,经过PMAN增强后语音的客观语音质量评价(PESQ)、短时客观可懂度(STOI)、分段信噪比(SSNR)指标值分别提升10.8%、1.1%、11.8%,具有较好的语音增强效果。

Regarding the issue of the frequency-domain enhancement of speech affected by interference,a speech enhancement network based on a parallel multi-attention mechanism and an encoding and decoding structure,known as PMAN,is proposed.The network uses speech frequency-domain features obtained through a Short-Time Fourier Transform(STFT),including amplitude and complex spectra.The encoder integrates input data using dense convolutional modules.The parallel multi-attention module of the intermediate layer learns both local and global information in the frequency-domain and incorporates a Local Patch Attention(LPA)mechanism to capture the Two-Dimensional(2D)structure of the speech frequency-domain,achieving separation between clean speech and interference factors in the 2D space.The decoder integrates the learned information and generates amplitude masks and complex spectra separately.The final speech complex spectrum is obtained via weighted summation,and a joint time-and frequency-domain loss function is used to fuse the phase information.Experimental results on the VoiceBank+DEMAND speech dataset demonstrate that PMAN achieves better speech enhancement performance than a time-domain speech enhancement Neural Network based on a Two-Stage Transformer(TSTNN),with improvements of 10.8%in Perceptual Evaluation of Speech Quality(PESQ),1.1%in Short-Time Objective Intelligibility(STOI),and 11.8%in Segmental Signal-to-Noise Ratio(SSNR).