多模态余弦相似孪生网络人脸跟踪算法

Multi-Modal Cosine Similarity Siamese Network Face Tracking Algorithm

为解决人脸跟踪算法光照敏感度高、复杂背景跟踪鲁棒性低等问题,本文基于SiamFC网络提出多模态余弦相似孪生网络人脸跟踪算法。首先,结合可见光图片与红外光图片信息互补的优势,设计多模态输入,借鉴PeleeNet网络、空洞卷积、可分离卷积思想设计轻量化特征提取模块进行特征提取。其次,使用余弦相似操作代替互相关操作,增大相似背景非相似域的距离,提升跟踪定位的准确度。最后,在公共数据集ChokePoin、RGBT234和自制数据集上进行该算法的性能评估。实验结果表明,该算法在跟踪效果上整体性能较高、跟踪速度流畅,推理速度为115.7 fps,能很好地应对遮挡、光线干扰、运动模糊等因素影响,具有较强的实时性和鲁棒性。

In order to solve the problems of high illumination sensitivity and low robustness of complex background tracking,a multi-modal cosine similar siamese network face tracking algorithm is proposed based on SiamFC network.Firstly,combined with the complementary advantages of visible spectrum and infrared spectrum information,multi-modal input was designed,and a lightweight feature extraction module was designed for feature extraction based on the ideas of PeleeNet network,Dilated Convolution and Depthwise Separable Convolution.Secondly,cosine similarity operation was used instead of cross-correlation operation to increase the distance between similar background and non similar domain and improve the accuracy of tracking and positioning.Finally,the performance of the algorithm was evaluated on the public dataset ChokePoin,RGBT234 and self-made databset.The experimental results show that in terms of tracking effect,the overall performance is high,the tracking speed is smooth,and the reasoning speed is 115.7 fps.It can well deal with the influence of occlusion,light interference,motion blur and other factors,and has strong real-time and robustness.