模态间匹配学习的高光谱和激光雷达联合分类

Joint classification of hyperspectral and LiDAR data based on inter-modality match learning

如何有效地提取和融合不同模态的特征是高光谱图像和激光雷达数据联合分类的关键。近年来,得益于深度学习强大的特征学习能力,其在高光谱图像和激光雷达数据联合分类领域受到了越来越多的关注。然而,现有的深度学习模型大多基于监督学习的模式,分类性能依赖标注样本的数量和质量。为此,本文提出了一种基于模态间匹配学习的联合分类方法,充分利用未标注样本的信息,减少对标注信息的依赖性。具体而言,本文首先通过高光谱图像和激光雷达数据之间的匹配关系和KMeans聚类算法,构造模态匹配标签。然后,利用该标签训练含有多个卷积层的匹配学习网络。该网络由两个并行分支构成,每个分支负责提取单个模态的特征。最后,以该网络为基础,构造高光谱图像和激光雷达数据联合分类模型。该模型的参数由匹配学习网络进行初始化,因而只需要少量标注样本进行微调即可达到理想的分类效果。为了验证本文方法的有效性,在Houston和MUUFL两个常用的高光谱图像和激光雷达数据联合分类数据集上进行了大量的实验。实验结果表明,与已有的分类模型相比,本文方法能够获得更高的分类性能。

Several excellent models for joint classification of hyperspectral image and LiDAR data,which were designed on the basis of supervised learning methods such as convolutional neural networks,have been developed in recent years.Their classification performance depends largely on the quantity and quality of training samples.However,when the distribution of ground objects becomes increasingly complex and the resolutions of remote sensing images grow increasingly high,obtaining high-quality labels only with limited cost and manpower is difficult.Therefore,numerous scholars have made efforts to learn features directly from unlabeled samples.For instance,the theory of autoencoder was applied to multimodal joint classification,achieving satisfactory performance.Methods based on the reconstruction idea reduce the dependence on labeled information to a certain extent,but several problems that must be settled still exist.For example,these methods pay more attention to data reconstruction but fail to guarantee that the extracted features have sufficient discriminant capability,thus affecting the performance of joint classification.This paper proposes an effective model named Joint Classification of Hyperspectral and LiDAR Data Based on Inter-Modality Match Learning to address the aforementioned issue.Different from feature extraction models based on reconstruction idea,the proposed model tends to compare the matching relationship between samples from different modalities,thereby enhancing the discriminative capability of features.Specifically,this model comprises inter-modality matching learning and multimodal joint classification networks.The former is prone to identify matching of the input patch pairs of hyperspectral image and LiDAR data;therefore,reasonable construction of matching labels is essential.Thus,spatial positions of center pixels in cropped patches and KMeans clustering methods are employed.These constructed labels and patch pairs are combined to train the network.Notably,this process does not use manual labeled information and can directly extract features from abundant unlabeled samples.Furthermore,in the joint classification stage,the structure and trained parameters of matching learning network are transferred,and a small number of manually labeled training samples are then used to finetune the model parameters.Extensive experiments were conducted on two widely used datasets,namely Houston and MUUFL,to verify the effectiveness of the proposed model.These experiments include comparison experiments with several state-of-the-art models,hyperparameter analysis experiments,and ablation studies.Take the first experiment as an example.Compared with other models,such as CNN,EMFNet,AE_H,AE_HL,CAE_H,CAE_HL,IP-CNN,and PToP CNN,the proposed model can achieve higher performance on both datasets with OAs of 88.39%and 81.46%,respectively.Overall,the proposed model reduces the dependence on manually labeled data and improves the joint classification accuracy in the case of limited training samples.A superior model structure and additional testing datasets will be explored in the future to make further improvements.