跨层交互与多源特征融合的遥感图像语义分割

Cross-layer Interaction and Multi-source Feature Adaptive Fusion Networks for Semantic Segmentation of Remote Sensing Images

将数字表面模型提供的高程特征与光谱特征进行融合可以提高遥感图像的语义分割性能。然而,由于多模态数据特征之间存在显著差异,仅使用级联或者相加的方式对数据特征进行融合容易造成特征冗余和过量噪声。针对该问题,提出了一种跨层交互与多源特征自适应融合的语义分割网络CIMFANet。该网络在解码层实现对双模态特征的提取,采用密集的短跳跃连接进行跨层特征交互以防止空间细节信息的丢失,同时引入空洞空间金字塔池化模块提取跨层交互后的多尺度上下文特征。解码层使用多模态融合模块以自适应的方式融合来自双分支和前一层的特征。该方法在ISPRS提供的两个基准数据集上与其他方法进行对比实验。实验表明,该方法在某些地物要素方面具有更高的分割精度。

The fusion of elevation features provided by digital surface models with spectral features can improve the semantic segmentation performance of remote sensing images.However,due to the significant differences between multimodal data features,the fusion of data features using only cascading or summation is prone to feature redundancy and excessive noise.The network implements the extraction of bimodal features at the decoding layer,using dense short-hop connections for cross-layer feature interactions to prevent loss of spatial detail information,and introduces a void space pyramid pooling module to extract multi-scale contextual features after cross-layer interactions.The decoding layer uses a multimodal fusion module to fuse features from the two branches and the previous layer in an adaptive manner.The method is tested against other methods on two benchmark datasets provided by ISPRS.The experimental results show that the proposed method has higher segmentation accuracy in terms of some feature elements.