基于深度展开和双流网络的高光谱图像融合

Hyperspectral Image Fusion via Deep Unfolding and Dual-stream Networks

针对基于深度学习的高光谱图像融合算法通常堆积多个卷积以学习映射关系、没有充分利用问题的特性以及缺乏可解释性等问题,提出一种结合深度展开与双流网络的深度网络。首先使用卷积稀疏编码建立融合模型,该模型将低分辨率高光谱图像(Low-resolution hyperspectral images,LR-HSI)和高分辨率多光谱图像(high-resolution multispectral images,HR-MSI)映射到低维子空间中。在融合模型设计中,考虑了LR-HSI和HR-MSI的共有信息以及LR-HSI的独有信息,并将HR-MSI作为辅助信息加入模型中。其次将该融合模型展开为可学习的可解释深度网络。最后,使用双流网络获取更精确的高分辨率高光谱图像(High-resolution hyperspectral images,HR-HSI)。实验表明,该网络在高光谱图像融合中可以获得出色的效果。

Hyperspectral image fusion algorithms based on deep learning typically stack multiple convolutional layers to learn mapping relationships,which suffer from the problems of not fully utilizing the characteristics of the task and lack of interpretability.To address these problems,this paper proposes a deep network combining deep unfolding and dual-stream networks.Firstly,an image fusion model is established using convolutional sparse coding,which maps low-resolution hyperspectral images(LR-HSI)and high-resolution multispectral images(HR-MSI)into a low-dimensional subspace.In the design of the fusion model,we consider the common information of LR-HSI and HR-MSI as well as the unique information of LR-HSI,and add HR-MSI to the model as auxiliary information.Next,the fusion model is unfolded into a learnable interpretable deep network.Finally,the dual-stream network is used to get more accurate high-resolution hyperspectral images(HR-HSI).Experiments prove that the network obtains excellent results in the hyperspectral image fusion task.