基于多层小波深度聚合网络的高光谱图像超分辨率方法

MW-DAN:Multilevel Wavelet-Deep Aggregation Network for Hyperspectral Image Super-Resolution

利用低空间分辨率高光谱(Low Resolution HyperSpectral Image,LR-HSI)和高空间分辨率多光谱图像(High Resolution Multi Spectral Image,HR-MSI)的有机结合,实现高光谱空间分辨率增强,是当前高光谱图像处理的热点问题.目前,深度学习已成为高光谱-多光谱图像融合超分辨率的代表性方法,然而如何有效挖掘两者的互补空谱信息,实现空间结构和细节注入,在提升高光谱图像空间分辨率的同时保持高保真光谱信息,依然存在诸多挑战.本文提出了一种多层小波深度聚合网络(Multilevel Wavelet-Deep Aggregation Network,MW-DAN).该网络有机结合非抽取小波(Un Decimated Wavelet Transform,UDWT)分解和深度残差网络,建立双分支互补信息融合网络,提升图像重建性能.其中,通过深度残差网络中引入跳层汇聚连接,设计信息聚合型结构,并对多光谱图像进行UDWT方向子带分解,逐层注入到网络中间隐层,增强了方向子带结构的细节注入和光谱保真能力.整个网络通过LR-HSI,HR-MSI和HRHSI(High Resolution HyperSpectral Image)端对端训练,能够学习性能优越的空-谱融合的超分辨非线性映射.大量仿真数据集和真实数据集上的大量融合实验表明,本文提出的方法在客观评价指标、光谱保持和视觉效果上优于目前主流的深度学习方法 .

The utilization of the synergistic fusion of low resolution hyperspectral image(LR-HSI)and high resolu-tion multispectral image(HR-MSI)for the purpose of achieving enhanced hyperspectral spatial resolution has emerged as a prominent and actively pursued research area within the domain of hyperspectral image processing.At the present time,deep learning has become an efficient tool for HSI-MSI fusion.Despite the potential of deep learning,there are still some challenging,such as how to effectively mine the complementary information of HSI and MSI,how to inject the spatial struc-ture and detail of MSI into HSI,and how to maintain the spectral fidelity of HSI.This study proposes a multilevel wavelet-deep aggregation network(MW-DAN).It has dual branches,which combine undecimated wavelet transform(UDWT)with deep residual network to promote the image reconstruction.Particularly,the UDWT directional subband decomposition of MSIs is performed by introducing jumper aggregation connections in the deep residual network to design an information ag-gregation type structure,and injected into the middle hidden layer of the network layer by layer to enhance the detail injec-tion and spectral fidelity of the directional subband structure.The entire network are trained from LR-HSI,HR-MSI and HR-HSI in an end-to-end fashion.It could learn the spatial-spectral fusion nonlinear mapping with superior performance.Experimental results on simulation and real datasets show that the proposed method is superior to the state-of-the-art fusion methods in terms of objective evaluation index,spectral fidelity and visual performance.