摘要
可见光成像在夜间或恶劣天气情况下易受光照影响,降低了语义分割系统的性能,而同时使用可见光/红外多波段成像传感器则可以缓解这个问题。提出了一种基于融合路径监督的多波段图像语义分割方法,在网络训练过程中直接对特征融合过程进行类别监督。首先,将分割网络中独立的特征融合模块组建为贯通的融合支路,利用高层特征指引低层特征融合;其次,对融合支路末端直接施加监督信号,以提升融合特征的鉴别性及网络的收敛速度;最后,为了改善对于小目标的分割效果,在融合支路上特别引入Dice损失,构成混合监督训练模式。在两个多波段图像数据集上的实验结果表明,与其他多波段图像语义分割方法相比,本文方法可以达到更优的分割效果,而且对小目标分割更有利。
Visible imaging is easily affected by illumination at night or in bad weather conditions,which reduces the performance of semantic segmentation system.The above problem can be alleviated by simultaneously using visible cameras and thermal Infrared Ray(IR)sensors.Although there are some jobs involved in the semantic segmentation of RGB-IR images,few of them attempted to improve the segmentation results by enhancing the discriminability of the fusion feature.Therefore,this paper proposes a novel framework of multi-spectral image semantic segmentation based on the process of supervised feature fusion.Firstly,the independent feature fusion modules constitute a whole fusion branch in the semantic segmentation network so that the fusion of high-level features can guide the fusion of lower-level features.Secondly,the segmentation supervision signal is directly applied to the end of the fusion branch to improve the discriminability of the fusion feature and the convergence speed of the network.Finally,in order to improve the segmentation of small objects,a dice loss instead of the cycle loss is specially introduced into the fusion branch to achieve a hybrid supervision of training process.It is shown via the experimental results on two multi-spectral datasets that,compared with other multi-spectral semantic segmentation methods,the proposed method can achieve better segmentation results and have more favorable for small objects segmentation.
作者
康萌萌
杨浩
谷小婧
顾幸生
KANG Mengmeng;YANG Hao;GU Xiaojing;GU Xingsheng(Key Laboratory of Advanced Control and Optimization for Chemical Processes,Ministry of Education,East China University of Science and Technology,Shanghai 200237,China;China Academy of Civil Aviation Science and Technology,Beijing 100028,China)
出处
《华东理工大学学报(自然科学版)》
CAS
CSCD
北大核心
2021年第2期233-240,共8页
Journal of East China University of Science and Technology
基金
国家自然科学基金(61973122,61973120,61573144)
上海市自然科学基金(18ZR1415000)。
关键词
多波段图像
语义分割
特征融合
混合监督
multi-spectral image
semantic segmentation
feature fusion
hybrid supervision