In order to improve the spatial resolution of hyperspectral(HS)image and minimize the spectral distortion,an HS and multispectral(MS)image fusion approach based on convolutional neural network(CNN)is proposed.The prop...In order to improve the spatial resolution of hyperspectral(HS)image and minimize the spectral distortion,an HS and multispectral(MS)image fusion approach based on convolutional neural network(CNN)is proposed.The proposed approach incorporates the linear spectral mixture model and spatial-spectral spread transform model into the learning phase of network,aiming to fully exploit the spatial-spectral information of HS and MS images,and improve the spectral fidelity of fusion images.Experiments on two real remote sensing data under different resolutions demonstrate that compared with some state-of-the-art HS and MS image fusion methods,the proposed approach achieves superior spectral fidelities and lower fusion errors.展开更多
Disjoint sampling is critical for rigorous and unbiased evaluation of state-of-the-art(SOTA)models e.g.,Attention Graph and Vision Transformer.When training,validation,and test sets overlap or share data,it introduces...Disjoint sampling is critical for rigorous and unbiased evaluation of state-of-the-art(SOTA)models e.g.,Attention Graph and Vision Transformer.When training,validation,and test sets overlap or share data,it introduces a bias that inflates performance metrics and prevents accurate assessment of a model’s true ability to generalize to new examples.This paper presents an innovative disjoint sampling approach for training SOTA models for the Hyperspectral Image Classification(HSIC).By separating training,validation,and test data without overlap,the proposed method facilitates a fairer evaluation of how well a model can classify pixels it was not exposed to during training or validation.Experiments demonstrate the approach significantly improves a model’s generalization compared to alternatives that include training and validation data in test data(A trivial approach involves testing the model on the entire Hyperspectral dataset to generate the ground truth maps.This approach produces higher accuracy but ultimately results in low generalization performance).Disjoint sampling eliminates data leakage between sets and provides reliable metrics for benchmarking progress in HSIC.Disjoint sampling is critical for advancing SOTA models and their real-world application to large-scale land mapping with Hyperspectral sensors.Overall,with the disjoint test set,the performance of the deep models achieves 96.36%accuracy on Indian Pines data,99.73%on Pavia University data,98.29%on University of Houston data,99.43%on Botswana data,and 99.88%on Salinas data.展开更多
Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,w...Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.展开更多
Image fusion is performed between one band of multi-spectral image and two bands of hyperspectral image to produce fused image with the same spatial resolution as source multi-spectral image and the same spectral reso...Image fusion is performed between one band of multi-spectral image and two bands of hyperspectral image to produce fused image with the same spatial resolution as source multi-spectral image and the same spectral resolution as source hyperspeetral image. According to the characteristics and 3-Dimensional （3-D） feature analysis of multi-spectral and hyperspectral image data volume, the new fusion approach using 3-D wavelet based method is proposed. This approach is composed of four major procedures： Spatial and spectral resampling, 3-D wavelet transform, wavelet coefficient integration and 3-D inverse wavelet transform. Especially, a novel method, Ratio Image Based Spectral Resampling （RIBSR）method, is proposed to accomplish data resampling in spectral domain by utilizing the property of ratio image. And a new fusion rule, Average and Substitution （A＆S） rule, is employed as the fusion rule to accomplish wavelet coefficient integration. Experimental results illustrate that the fusion approach using 3-D wavelet transform can utilize both spatial and spectral characteristics of source images more adequately and produce fused image with higher quality and fewer artifacts than fusion approach using 2-D wavelet transform. It is also revealed that RIBSR method is capable of interpolating the missing data more effectively and correctly, and A＆S rule can integrate coefficients of source images in 3-D wavelet domain to preserve both spatial and spectral features of source images more properly.展开更多
Hyperspectral image(HSI)classification has been one of themost important tasks in the remote sensing community over the last few decades.Due to the presence of highly correlated bands and limited training samples in H...Hyperspectral image(HSI)classification has been one of themost important tasks in the remote sensing community over the last few decades.Due to the presence of highly correlated bands and limited training samples in HSI,discriminative feature extraction was challenging for traditional machine learning methods.Recently,deep learning based methods have been recognized as powerful feature extraction tool and have drawn a significant amount of attention in HSI classification.Among various deep learning models,convolutional neural networks(CNNs)have shown huge success and offered great potential to yield high performance in HSI classification.Motivated by this successful performance,this paper presents a systematic review of different CNN architectures for HSI classification and provides some future guidelines.To accomplish this,our study has taken a few important steps.First,we have focused on different CNN architectures,which are able to extract spectral,spatial,and joint spectral-spatial features.Then,many publications related to CNN based HSI classifications have been reviewed systematically.Further,a detailed comparative performance analysis has been presented between four CNN models namely 1D CNN,2D CNN,3D CNN,and feature fusion based CNN(FFCNN).Four benchmark HSI datasets have been used in our experiment for evaluating the performance.Finally,we concluded the paper with challenges on CNN based HSI classification and future guidelines that may help the researchers to work on HSI classification using CNN.展开更多
The breeding and selection of resistant varieties is an effective way to minimize wheat Fusarium head blight(FHB)hazards,so it is important to identify and evaluate resistant varieties.The traditional resistance pheno...The breeding and selection of resistant varieties is an effective way to minimize wheat Fusarium head blight(FHB)hazards,so it is important to identify and evaluate resistant varieties.The traditional resistance phenotype identification is still largely dependent on time-consuming manual methods.In this paper,the method for evaluating FHB resistance in wheat ears was optimized based on the fusion feature wavelength images of the hyperspectral imaging system and the Faster R-CNN algorithm.The spectral data from 400-1000 nm were preprocessed by the multiple scattering correction(MSC)algorithm.Three feature wavelengths(553 nm,682 nm and 714 nm)were selected by analyzing the X-loading weights(XLW)according to the absolute value of the peaks and troughs in different principal component(PC)load coefficient curves.Then,the different fusion methods of the three feature wavelengths were explored with different weight coefficients.Faster R-CNN was trained on the fusion and RGB datasets with VGG16,AlexNet,ZFNet,and ResNet-50 networks separately.Then,the other detection models SSD,YOLOv5,YOLOv7,CenterNet,and RetinaNet were used to compare with the Faster R-CNN model.As a result,the Faster R-CNN with VGG16 was best with the mAP(mean Average Precision)ranged from 97.7%to 98.8%.The model showed the best performance for the Fusion Image-1 dataset.Moreover,the Faster R-CNN model with VGG16 achieved an average detection accuracy of 99.00%,which was 23.89%,1.21%,0.75%,0.62%,and 8.46%higher than SSD,YOLOv5,YOLOv7,CenterNet,and RetinaNet models.Therefore,it was demonstrated that the Faster R-CNN model based on the hyperspectral feature image fusion dataset proposed in this paper was feasible for rapid evaluation of wheat FHB resistance.This study provided an important detection method for ensuring wheat food security.展开更多
高光谱图像以其高分辨率的空间和光谱信息在军事、航空航天及民用等遥感领域均有重要应用,具有重要的研究意义。深度学习具有学习能力强、覆盖范围广及可移植性强的优势,成为目前高精度高光谱图像分类技术研究的热点。其中卷积神经网络(...高光谱图像以其高分辨率的空间和光谱信息在军事、航空航天及民用等遥感领域均有重要应用,具有重要的研究意义。深度学习具有学习能力强、覆盖范围广及可移植性强的优势,成为目前高精度高光谱图像分类技术研究的热点。其中卷积神经网络(CNN)因强大的特征提取能力广泛应用于高光谱图像分类方法研究中,取得了有效的研究成果,但该类方法通常单独基于2D-CNN或3D-CNN进行,针对高光谱图像的单一特征,一是不能充分利用高光谱数据本身完整的特征信息;二是虽然相应提取网络局部特征优化性好,但是整体泛化能力不足,在深度挖掘HSI的空间和光谱信息方面存在局限性。鉴于此,提出了基于注意力机制的混合卷积神经网络模型(HybridSN_AM),使用主成分分析法对高光谱图像进行降维,采用卷积神经网络作为分类模型的主体,通过注意力机制筛选出更有区分度的特征,使模型能够提取到更精确、更核心的空间-光谱信息,实现高光谱图像的高精度分类。对Indian Pines(IP)、University of Pavia(UP)和Salinas(SA)三个数据集进行了应用实验,结果表明,基于该模型的目标图像总体分类精度、平均分类精度和Kappa系数均高于98.14%、97.17%、97.87%。与常规HybridSN模型对比表明,HybridSN_AM模型在三个数据集上的分类精度分别提升了0.89%、0.07%和0.73%。有效解决了高光谱图像空间-光谱特征提取与融合的难题,提高HSI分类的精度,具有较强的泛化能力,充分验证了注意力机制结合混合卷积神经网络在高光谱图像分类中的有效性和可行性,对高光谱图像分类技术的发展及应用具有重要的科学价值。展开更多
针对高光谱遥感图像复杂农作物分类问题,提出了一种基于空谱融合和随机多图的高光谱遥感图像农作物分类方法。通过使用一种潜在特征融合和最优聚类(Latent Features Fusion and Optimal Clustering Framework,LFFOCF)的波段选择方法和...针对高光谱遥感图像复杂农作物分类问题,提出了一种基于空谱融合和随机多图的高光谱遥感图像农作物分类方法。通过使用一种潜在特征融合和最优聚类(Latent Features Fusion and Optimal Clustering Framework,LFFOCF)的波段选择方法和分段主成分分析(Segmented Principal Component Analysis,SPCA)进行光谱降维,采用多尺度二维奇异谱分析(2-D-Singular Spectrum Analysis,2-D-SSA)应用于降维图像,以提取不同尺度的空间特征。将多尺度空间特征与主成分分析(Principal Component Analysis,PCA)得到的全局光谱特征融合送到随机多图(Random Multi-Graphs,RMG)中进行分类。在印度松树、萨利纳斯和龙口数据集上,所提出的方法与一些现有的方法进行了对比实验。实验结果表明,该方法的类别精度(Class Accuracy,CA)、总体分类精度(Overall Accuracy,OA)、平均分类精度(Average Accuracy,AA)和Kappa系数优于这些方法。展开更多
基金National Natural Science Foundation of China(No.61902060)Natural Science Foundation of Shanghai,China(No.19ZR1453800)Fundamental Research Funds for the Central Universities,China(No.2232021D-33)。
文摘In order to improve the spatial resolution of hyperspectral(HS)image and minimize the spectral distortion,an HS and multispectral(MS)image fusion approach based on convolutional neural network(CNN)is proposed.The proposed approach incorporates the linear spectral mixture model and spatial-spectral spread transform model into the learning phase of network,aiming to fully exploit the spatial-spectral information of HS and MS images,and improve the spectral fidelity of fusion images.Experiments on two real remote sensing data under different resolutions demonstrate that compared with some state-of-the-art HS and MS image fusion methods,the proposed approach achieves superior spectral fidelities and lower fusion errors.
基金the Researchers Supporting Project number(RSPD2024R848),King Saud University,Riyadh,Saudi Arabia.
文摘Disjoint sampling is critical for rigorous and unbiased evaluation of state-of-the-art(SOTA)models e.g.,Attention Graph and Vision Transformer.When training,validation,and test sets overlap or share data,it introduces a bias that inflates performance metrics and prevents accurate assessment of a model’s true ability to generalize to new examples.This paper presents an innovative disjoint sampling approach for training SOTA models for the Hyperspectral Image Classification(HSIC).By separating training,validation,and test data without overlap,the proposed method facilitates a fairer evaluation of how well a model can classify pixels it was not exposed to during training or validation.Experiments demonstrate the approach significantly improves a model’s generalization compared to alternatives that include training and validation data in test data(A trivial approach involves testing the model on the entire Hyperspectral dataset to generate the ground truth maps.This approach produces higher accuracy but ultimately results in low generalization performance).Disjoint sampling eliminates data leakage between sets and provides reliable metrics for benchmarking progress in HSIC.Disjoint sampling is critical for advancing SOTA models and their real-world application to large-scale land mapping with Hyperspectral sensors.Overall,with the disjoint test set,the performance of the deep models achieves 96.36%accuracy on Indian Pines data,99.73%on Pavia University data,98.29%on University of Houston data,99.43%on Botswana data,and 99.88%on Salinas data.
基金supported in part by the National Natural Science Foundation of China(62276192)。
文摘Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.
文摘Image fusion is performed between one band of multi-spectral image and two bands of hyperspectral image to produce fused image with the same spatial resolution as source multi-spectral image and the same spectral resolution as source hyperspeetral image. According to the characteristics and 3-Dimensional （3-D） feature analysis of multi-spectral and hyperspectral image data volume, the new fusion approach using 3-D wavelet based method is proposed. This approach is composed of four major procedures： Spatial and spectral resampling, 3-D wavelet transform, wavelet coefficient integration and 3-D inverse wavelet transform. Especially, a novel method, Ratio Image Based Spectral Resampling （RIBSR）method, is proposed to accomplish data resampling in spectral domain by utilizing the property of ratio image. And a new fusion rule, Average and Substitution （A＆S） rule, is employed as the fusion rule to accomplish wavelet coefficient integration. Experimental results illustrate that the fusion approach using 3-D wavelet transform can utilize both spatial and spectral characteristics of source images more adequately and produce fused image with higher quality and fewer artifacts than fusion approach using 2-D wavelet transform. It is also revealed that RIBSR method is capable of interpolating the missing data more effectively and correctly, and A＆S rule can integrate coefficients of source images in 3-D wavelet domain to preserve both spatial and spectral features of source images more properly.
文摘Hyperspectral image(HSI)classification has been one of themost important tasks in the remote sensing community over the last few decades.Due to the presence of highly correlated bands and limited training samples in HSI,discriminative feature extraction was challenging for traditional machine learning methods.Recently,deep learning based methods have been recognized as powerful feature extraction tool and have drawn a significant amount of attention in HSI classification.Among various deep learning models,convolutional neural networks(CNNs)have shown huge success and offered great potential to yield high performance in HSI classification.Motivated by this successful performance,this paper presents a systematic review of different CNN architectures for HSI classification and provides some future guidelines.To accomplish this,our study has taken a few important steps.First,we have focused on different CNN architectures,which are able to extract spectral,spatial,and joint spectral-spatial features.Then,many publications related to CNN based HSI classifications have been reviewed systematically.Further,a detailed comparative performance analysis has been presented between four CNN models namely 1D CNN,2D CNN,3D CNN,and feature fusion based CNN(FFCNN).Four benchmark HSI datasets have been used in our experiment for evaluating the performance.Finally,we concluded the paper with challenges on CNN based HSI classification and future guidelines that may help the researchers to work on HSI classification using CNN.
基金supported by the Natural Science Foundation of Jiangsu Province(Grant No.BK20221518)the Jiangsu Agriculture Science and Technology Innovation Fund(Grant No.CX(23)1002)。
文摘The breeding and selection of resistant varieties is an effective way to minimize wheat Fusarium head blight(FHB)hazards,so it is important to identify and evaluate resistant varieties.The traditional resistance phenotype identification is still largely dependent on time-consuming manual methods.In this paper,the method for evaluating FHB resistance in wheat ears was optimized based on the fusion feature wavelength images of the hyperspectral imaging system and the Faster R-CNN algorithm.The spectral data from 400-1000 nm were preprocessed by the multiple scattering correction(MSC)algorithm.Three feature wavelengths(553 nm,682 nm and 714 nm)were selected by analyzing the X-loading weights(XLW)according to the absolute value of the peaks and troughs in different principal component(PC)load coefficient curves.Then,the different fusion methods of the three feature wavelengths were explored with different weight coefficients.Faster R-CNN was trained on the fusion and RGB datasets with VGG16,AlexNet,ZFNet,and ResNet-50 networks separately.Then,the other detection models SSD,YOLOv5,YOLOv7,CenterNet,and RetinaNet were used to compare with the Faster R-CNN model.As a result,the Faster R-CNN with VGG16 was best with the mAP(mean Average Precision)ranged from 97.7%to 98.8%.The model showed the best performance for the Fusion Image-1 dataset.Moreover,the Faster R-CNN model with VGG16 achieved an average detection accuracy of 99.00%,which was 23.89%,1.21%,0.75%,0.62%,and 8.46%higher than SSD,YOLOv5,YOLOv7,CenterNet,and RetinaNet models.Therefore,it was demonstrated that the Faster R-CNN model based on the hyperspectral feature image fusion dataset proposed in this paper was feasible for rapid evaluation of wheat FHB resistance.This study provided an important detection method for ensuring wheat food security.
文摘高光谱图像以其高分辨率的空间和光谱信息在军事、航空航天及民用等遥感领域均有重要应用,具有重要的研究意义。深度学习具有学习能力强、覆盖范围广及可移植性强的优势,成为目前高精度高光谱图像分类技术研究的热点。其中卷积神经网络(CNN)因强大的特征提取能力广泛应用于高光谱图像分类方法研究中,取得了有效的研究成果,但该类方法通常单独基于2D-CNN或3D-CNN进行,针对高光谱图像的单一特征,一是不能充分利用高光谱数据本身完整的特征信息;二是虽然相应提取网络局部特征优化性好,但是整体泛化能力不足,在深度挖掘HSI的空间和光谱信息方面存在局限性。鉴于此,提出了基于注意力机制的混合卷积神经网络模型(HybridSN_AM),使用主成分分析法对高光谱图像进行降维,采用卷积神经网络作为分类模型的主体,通过注意力机制筛选出更有区分度的特征,使模型能够提取到更精确、更核心的空间-光谱信息,实现高光谱图像的高精度分类。对Indian Pines(IP)、University of Pavia(UP)和Salinas(SA)三个数据集进行了应用实验,结果表明,基于该模型的目标图像总体分类精度、平均分类精度和Kappa系数均高于98.14%、97.17%、97.87%。与常规HybridSN模型对比表明,HybridSN_AM模型在三个数据集上的分类精度分别提升了0.89%、0.07%和0.73%。有效解决了高光谱图像空间-光谱特征提取与融合的难题,提高HSI分类的精度,具有较强的泛化能力,充分验证了注意力机制结合混合卷积神经网络在高光谱图像分类中的有效性和可行性,对高光谱图像分类技术的发展及应用具有重要的科学价值。
文摘针对高光谱图像全色锐化中的光谱失真和纹理细节提升不足问题,结合交叉皮层神经网络模型(intersecting cortical model,ICM),提出一种自适应高光谱图像全色锐化算法。该算法采用ICM分割,先将高光谱图像与空间分辨率较为接近的多光谱图像进行匹配融合,再将结果与高分辨率的全色图像融合,以获得同时具有全色图像的高空间分辨率和高光谱图像的光谱分辨率融合结果。同时,在锐化融合中采用灰狼优化算法(grey wolf optimizer,GWO)自适应优化ICM模型参数,生成最优非规则分割区域,为高光谱图像提供更精准全面的细节和光谱信息。采用2组资源一号02D卫星高光谱数据集进行实验验证,结果表明,提出的新的锐化融合算法在空间细节和光谱信息评价指标上均表现最优,验证了该算法有效性。
