摘要
利用低空间分辨率高光谱(Low Resolution HyperSpectral Image,LR-HSI)和高空间分辨率多光谱图像(High Resolution Multi Spectral Image,HR-MSI)的有机结合,实现高光谱空间分辨率增强,是当前高光谱图像处理的热点问题.目前,深度学习已成为高光谱-多光谱图像融合超分辨率的代表性方法,然而如何有效挖掘两者的互补空谱信息,实现空间结构和细节注入,在提升高光谱图像空间分辨率的同时保持高保真光谱信息,依然存在诸多挑战.本文提出了一种多层小波深度聚合网络(Multilevel Wavelet-Deep Aggregation Network,MW-DAN).该网络有机结合非抽取小波(Un Decimated Wavelet Transform,UDWT)分解和深度残差网络,建立双分支互补信息融合网络,提升图像重建性能.其中,通过深度残差网络中引入跳层汇聚连接,设计信息聚合型结构,并对多光谱图像进行UDWT方向子带分解,逐层注入到网络中间隐层,增强了方向子带结构的细节注入和光谱保真能力.整个网络通过LR-HSI,HR-MSI和HRHSI(High Resolution HyperSpectral Image)端对端训练,能够学习性能优越的空-谱融合的超分辨非线性映射.大量仿真数据集和真实数据集上的大量融合实验表明,本文提出的方法在客观评价指标、光谱保持和视觉效果上优于目前主流的深度学习方法 .
The utilization of the synergistic fusion of low resolution hyperspectral image(LR-HSI)and high resolu-tion multispectral image(HR-MSI)for the purpose of achieving enhanced hyperspectral spatial resolution has emerged as a prominent and actively pursued research area within the domain of hyperspectral image processing.At the present time,deep learning has become an efficient tool for HSI-MSI fusion.Despite the potential of deep learning,there are still some challenging,such as how to effectively mine the complementary information of HSI and MSI,how to inject the spatial struc-ture and detail of MSI into HSI,and how to maintain the spectral fidelity of HSI.This study proposes a multilevel wavelet-deep aggregation network(MW-DAN).It has dual branches,which combine undecimated wavelet transform(UDWT)with deep residual network to promote the image reconstruction.Particularly,the UDWT directional subband decomposition of MSIs is performed by introducing jumper aggregation connections in the deep residual network to design an information ag-gregation type structure,and injected into the middle hidden layer of the network layer by layer to enhance the detail injec-tion and spectral fidelity of the directional subband structure.The entire network are trained from LR-HSI,HR-MSI and HR-HSI in an end-to-end fashion.It could learn the spatial-spectral fusion nonlinear mapping with superior performance.Experimental results on simulation and real datasets show that the proposed method is superior to the state-of-the-art fusion methods in terms of objective evaluation index,spectral fidelity and visual performance.
作者
方健
杨劲翔
肖亮
FANG Jian;YANG Jing-xiang;XIAO Liang(School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing,Jiangsu 210094,China;Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense,Nanjing,Jiangsu 210094,China)
出处
《电子学报》
EI
CAS
CSCD
北大核心
2024年第1期201-216,共16页
Acta Electronica Sinica
基金
国家自然科学基金(No.61871226,No.61571230,No.62001226)
江苏省重点研发计划(No.BE2018727)
江苏省自然科学基金(No.BK20200465)
中央高校基本科研业务费专项资金(No.30920021134)。
关键词
高光谱图像
图像融合
深度学习
非抽取小波变换
深度残差聚合模块
hyperspectral image
image fusion
deep learning
undecimated wavelet transform
deep residual aggre-gation module