基于跨模态特征融合的RGB-D显著性目标检测

RGB-D Salient Object Detection Based on Cross-modal Feature Fusion

显著性目标检测(SOD)作为目前计算机视觉以及计算机图形学领域中研究的基本课题之一,是许多其他复杂任务的预处理阶段的任务,对例如图像理解与解释、视觉追踪、语义分割,视频分析等对象级应用的发展起到了极大的推动作用。随着深度传感器的普及,深度图像中蕴含的空间信息线索在显著性检测研究中提供了与RGB图像中蕴含的不同模态的辅助补充特征信息,这对于检测精度的提升来说愈发重要,因此如何有效地融合RGB与深度图像中的不同模态间的特征信息成为了RGB-D显著性目标检测课题中研究的重要问题。针对RGB与Depth模态间的特征融合问题,本文设计了一种基于跨模态特征信息融合的双流RGB-D显著目标检测网络模型,通过使用设计的跨模态特征融合模块去除某些低质量深度图带入的冗余与噪音,随后提取放大被优化改良过后的深度特征线索与RGB特征线索间的相似性与差异性,完成跨模态特征信息的有效融合。除此之外在网络编码结构的顶端增加了改良的非局部模块,通过自注意力机制更好地捕捉了的上下文信息以及像素间的长距离依赖。通过使用的两个数据集上的实验表明,这一模型在4个评价指标上取得了较好的表现。

As one of the basic topics currently in the field of computer vision and computer graphics, salient object detection(SOD) is the preprocessing stage of many other complex tasks. It has greatly promoted the development of object-level applications such as image understanding and interpretation, visual tracking, semantic segmentation, video analysis, etc. with the popularity of depth sensors, the spatial information clues contained in depth images provide auxiliary and supplementary feature information of different modalities from those contained in RGB images in saliency detection research, which is increasingly important for the improvement of detection accuracy. Therefore, how to effectively fuse different modal feature information from RGB and depth images has become an important issue in the research of RGB-D salient object detection. For this problem a dual-stream RGB-D salient object detection network model based on cross-modal feature information fusion is designed in this paper. By using the designed cross-modal feature fusion module to remove the redundancy and noise brought by some low-quality depth maps, the proposed model extracts the similarities and differences between the optimized and improved depth feature cues and RGB feature cues to complete the effective fusion of cross-modal feature information. In addition, an improved nonlocal module is added to the top of the network coding structure, and the context information and the long-distance dependency between pixels are better captured through the self-attention mechanism. Experiments on the two datasets used show that this model has achieved good performance on four evaluation indicators.