基于多模态图像信息的变电设备红外分割方法

Segmentation Method of Substation Equipment Infrared Image Based on Multimodal Image Information

无人机拍摄下的红外图像中变电设备的分割精度直接影响着热故障诊断的结果,针对复杂红外背景下变电设备分割精度低的问题,提出了一种融合可见光和红外图像的多模态路径聚合网络(Multimodal Path Aggregation Network,MPAN)。首先提取并融合两种模态图像的特征,考虑到两种模态图像的特征空间存在差异,提出了自适应特征融合模块(Adaptive Feature Fuse Module,AFFM),以充分融合两种模态特征;对具有多尺度特征的主干网络增加自底向上的金字塔网络,并对横向连接的路径增强模块引入自注意力机制;最后使用dice系数优化掩膜损失函数。实验结果表明,多模态图像的融合能够增强分割性能,且验证了提出各模块的有效性,该模型能够显著提高红外图像中变电设备实例分割的准确率。

The segmentation accuracy of substation equipment in infrared images captured by a UAV directly affects the results of thermal fault diagnosis.We proposed a multimodal path aggregation network(MPAN)that fuses visible and infrared images to address the problem of low segmentation accuracy of substation equipment in complex infrared backgrounds.First,we extracted and fused the features of two modal images,and considering the differences in the feature space of the two modal images,we proposed the adaptive feature fuse module(AFFM)to fuse the two modal features fully.We added a bottom-up pyramid network to the backbone with multi-scale features and a laterally connected path enhancement.Finally,we used dice coefficients to optimize the mask loss function.The experimental results showed that the fusion of multimodal images can enhance the segmentation performance and verify the effectiveness of the proposed modules,which can significantly improve the accuracy of the segmentation of substation equipment instances in infrared images.