YOLOX-S声光信息融合目标识别算法

YOLOX-S Based Acousto-optical Information Fusion Target Recognition Algorithm

针对现代战场单一探测手段的局限性和单模态目标识别存在信息不全面、易受噪声干扰等缺点,提出一种融合声光两种模态的目标识别方法。该方法利用深度卷积残差网络对声纹信息的对数梅尔频谱系数特征进行提取,使用YOLOX-S网络对目标进行光学特征提取,并计算目标的像空间位置与类别信息,然后在YOLOX-S模型预测部分的解耦头中引入用于处理声音特征的支路,将目标的光学特性与声学特性在YOLOX-S检测头分类支路上进行空间归一化,使视觉数据与声纹数据在同一可拼接域上进行映射与融合,对目标的声光融合特征进行识别推理。在自建数据集上进行验证,实验结果表明声纹信息和图像信息融合可以提供更全面的感知能力,使得目标的检测和识别更加准确和可靠。

In view of the limitations of single detection methods in modern battlefield and the shortcomings of single mode target recognition such as incomplete information and easy to be disturbed by noise,a new target recognition method combining two modes of sound and light was proposed.In this method,the log-mel spectral coefficient features of voiceprint information were extracted by deep convolutional residual network,the optical features of the target were extracted by YOLOX-S network,and the image space position and category information of the target were calculated.Then,a branch for processing sound features was introduced into the decoupling head of the prediction part of the YOLOX-S model.The optical and acoustic characteristics of the target were spatially normalized on the classification branch of the YOLOX-S detection head,so that the visual data and voicing data could be mapped and fused in the same concatenable domain,and the acousto-optical fusion features of the target could be identified and reasoned.The experimental results showed that the fusion of voiceprint information and image information could provide a more comprehensive perception capability and make the detection and recognition of objects more accurate and reliable.