基于多层次特征融合的RGB-D显著性检测

RGB-D Saliency Detection Based on Multi-Level Feature Fusion

大多数RGB-D显著性检测方法在探索各层跨模态信息时,往往直接将深度图不加处理地与RGB图进行融合,并且在各个层次采用相同的融合策略。然而,这会产生两个问题:(1)低质量深度图会把大量的冗余信息带入网络中,给检测带来负面影响;(2)在各个层次上采用相同的融合策略,忽略了模型在不同层次对全局和局部特征具有不同的关注度。为了解决上述问题,提出了一种自顶向下的多层次特征融合结构,通过设计深度增强模块有效地过滤低质量深度图信息;设计高层交融模块有效地整合高层中的全局特征;设计低层交融模块有效提取与融合有用的局部特征。通过在5个公共数据集上与7种先进模型进行的综合实验表明,该模型在F(F-measure)指标、avgF(average F-measure)指标、S(S-measure)指标和MAE(mean absolute error)4个指标上表现出了良好的性能。

When most RGB-D saliency detection methods explore the cross-modal information of each layer,they often directly merge the depth map with the RGB map without processing,and use the same fusion strategy at each level.However,this will cause two problems:(1)Low-quality depth maps will bring a lot of redundant information into the network,which will have a negative impact on detection;(2)The same fusion strategy is adopted at all levels,which ignores that the model has different degrees of attention to global and local features at different levels.In order to solve the above problems,a top-down multi-level feature fusion structure is proposed.The low-quality depth map information is effectively filtered through the design of the depth enhancement module.The high-level fusion module is designed to effectively integrate the global features in the high-level.The low-level fusion module is designed to effectively extract and merge useful local features.Comprehensive experiments on five public datasets and seven advanced models show that the method in this paper has superiority in the four indicators of F value,avgF value,S value and MAE.