融合ConvLSTM和多注意力机制网络的高光谱图像分类

Fusion of ConvLSTM and Multi-Attention Mechanism Network for Hyperspectral Image Classification

近年来,基于深度学习的模型在高光谱图像(HSI)分类方面效果显著。针对小样本数据基于深度学习的高光谱图像分类方法分类精度不高的问题,提出一种融合卷积长短期记忆(ConvLSTM)和多注意力机制网络的高光谱图像分类方法。该方法分三个分支:光谱分支、空间X分支和空间Y分支分别提取光谱特征、空间X特征和空间Y特征,并将三个方向的特征融合进行高光谱图像分类。由于ConvLSTM在学习有价值的特征和对光谱数据中的长期依赖关系建模方面表现出良好的性能,所以在光谱分支中用了3个隐藏层、卷积核大小为3×3、通道分别为150、100和60提取光谱信息。在空间X分支和空间Y分支,采用基于DenseNet和3D-CNN的Dense空间X块和Dense空间Y块分别提取空间X特征和空间Y特征。为了增强特征提取,在这三个分支中还分别引入了其特征方向的注意力机制,针对信息丰富的光谱波段设计了光谱注意块,信息丰富的像素点分别设计了空间X和空间Y注意块。在三个公开的高光谱数据集上进行了实验,即Indian Pines(IP)、Pavia University(UP)和Salinas Valley(SV)数据集;并对比了其他五种方法:基于RBF径向核的支持向量机模型(SVM)、更深更广的卷积神经网络模型(CDCNN)、快速密集光谱-空间卷积网络模型(FDSSC)、空谱残差网络模型(SSRN)、双分支双注意力机制网络模型(DBDA)。实验中,IP数据集上训练样本和验证样本的大小设为总样本的3%,UP和SV数据集上训练样本和验证样本的大小设为总样本的0.5%。该方法和所有基于深度学习的方法,批处理大小均设置为16,优化器设为Adam,学习率设置为0.0005,并动态调整学习率。由于SVM直接利用光谱信息进行分类,输入样本块像素大小为1×1,其他基于深度学习方法的输入样本块像素均设置为9×9。实验结果表明,该方法能充分利用高光谱图像的光谱和空间特征,在OA、AA、KAPPA等评价标准上均获得了更好的效果,其中,该方法的OA指标比次优的算法平均提高0.12%~2.04%。

In recent years,deep learning-based models have achieved remarkable results in the hyperspectral image(HSI)classification.Aiming at the low classification accuracy of deep learning-based HSI classification methods under limited sample data,this paper proposes an HSI classification method that combines ConvLSTM and a multi-attention mechanism network.The method is divided into three branches:spectral branch,spatial-X branch and spatial-Y branch to extract spectral features,spatial-X features and spatial-Y features respectively,and fuse the features in three directions for hyperspectral image classification.Since convolutional long short-term memory(ConvLSTM)shows good performance in learning valuable features and modeling long-term dependencies in spectral data,3 hidden layers are used in the spectral branch,and the convolution kernel size is 3×3,the channels are 150,100 and 60,respectively,to extract spectral information.On the spatial-X and spatial-Y branches,Dense spatial-X blocks and Dense spatial-Y blocks based on DenseNet and 3D-CNN are used to extract spatial-X and spatial-Y features,respectively.In order to enhance feature extraction,the attention mechanism of its feature direction is also introduced in these three branches,respectively.The spectral attention blocks are designed for the information-rich spectral bands,and a spatial-X attention block and a spatial-Y attention block are designed for the information-rich pixels,respectively.Experiments were conducted on three publicly available hyperspectral datasets,namely Indian Pines(IP),Pavia University(UP)and Salinas Valley(SV)datasets,and compared with five other methods:the SVM with RBF kernel(SVM),Going Deeper with Contextual CNN(CDCNN),Fast Dense Spectral-Spatial Convolution(FDSSC),Spectral-Spatial Residual Network(SSRN),Double-Branch Dual-Attention Mechanism Network(DBDA).In the experiments,the size of training and validation samples is set to 3%of the total samples on the IP dataset,and 0.5%of the total samples on the UP and SV datasets.For our method and all deep learning-based methods,the batch size is set to 16,the optimizer is set to Adam,the learning rate is set to 0.0005,and the learning rate is dynamically adjusted.Since SVM directly uses spectral information for classification,the pixel size of the input sample block is 1×1,and the pixels of other input sample blocks based on deep learning methods are all set to 9×9.The experimental results show that the method in this paper can fully use the spectral and spatial characteristics of HSI,and achieve better results in the evaluation criteria such as OA,AA,and KAPPA.Among them,the OA index of the method in this paper is improved by 0.12%~2.04%on average compared with the suboptimal algorithm.