基于多任务学习的高光谱图像语义分割算法

Semantic segmentation algorithm for hyperspectral image based on multi-task learning

针对高光谱图像语义分割中空间信息利用不充分的问题,提出了一种基于多任务学习的语义分割算法,分别为语义分割及基于遥感指数的图像重建任务。网络主要由3D卷积和2D卷积组成,通过主成分分析(principal component analysis,PCA)来减少冗余信息,通过不同任务的重要程度优化两者的损失函数权重。2个特征在分割任务中融合送入由3D通道注意力及空间池化金字塔(spatial pyramid pooling,SPP)组成的光谱-空间特征,提取模块获取谱间相关性及空间上下文信息,从而实现更好的分割效果。通过在Indian Pines数据集上验证,总体分割精度(overall accuracy,OA)达到了99.55%,Kappa达到了99.49%,有效地提高了高光谱遥感图像分割精度,同时在Salinas数据上通过消融实验也验证了所提算法各模块及其参数设置的有效性。

Focused on insufficient consideration of the spatial information in semantic segmentation of hyperspectral data,an algorithm based on multi-task learning was proposed,which were semantic segmentation and image reconstruction tasks based on remote sensing index,respectively.The network was mainly composed of 3D convolution and 2D convolution.Principal component analysis(PCA)was used to reduce redundant information,and the weights of loss function in different tasks was optimized according to their importance.The features of the two branches were fused in the segmentation task and sent to the spectral-spatial feature extraction module composed of 3D channel attention and spatial pyramid pooling(SPP)to obtain complex correlation between spectral channels and spatial context information to achieve better segmentation results.It is verified on Indian Pines dataset that overall accuracy(OA)reaches 99.55%and Kappa reaches 99.49%,which effectively improves the segmentation accuracy of hyperspectral remote sensing images.The effectiveness of each module and parameter setting of the algorithm is verified by ablation experiments on the Salinas dataset.