高频信息强化的双分支高光谱图像超分辨率网络

High-frequency Enhanced Dual-branch Hyperspectral Image Super-resolution Network

高光谱图像(hyperspectral image,HSI)的窄光谱波段为许多视觉任务提供了丰富信息,但也给特征提取带来了挑战.尽管许多研究者提出了各种深度学习方法,但尚未充分结合这些架构的优势.因此,本文提出了一种基于高频信息强化的双分支高光谱图像超分辨率网络(HFEDB-Net),将卷积神经网络(convolutional neural network,CNN)的图像空间特征提取优势与Transformer的自适应能力和长距离依赖提取优势相结合,有效地提取了HSI的空间和光谱信息.HFEDB-Net由高频信息强化分支和主干分支组成.在高频信息强化分支中,通过拉普拉斯金字塔提取低分辨率和高分辨率HSI的高频信息,并将结果作为高频分支的输入和标签,采用光谱强化Transformer来作为该分支的方法.在主干分支中,使用结合通道注意力的CNN充分提取空间特征和光谱信息.最后将两个分支的结果通过CNN进行结合以得到最终的重建图像.此外,采用多头注意力和多尺度策略分别改进了Transformer的注意力机制和编码器层,以更好地提取HSI的空间和光谱信息.实验结果表明,HFEDB-Net在两个公开数据集上的定量评价指标和视觉效果上优于当前主流方法.

The narrow spectral bands of hyperspectral images(HSI)provide rich information for many visual tasks,but also pose challenges for feature extraction.Despite various deep learning methods proposed by researchers,the advantages of these architectures are not fully combined.Therefore,this study proposes a high-frequency enhanced dualbranch hyperspectral image super-resolution network(HFEDB-Net)that effectively extracts spatial and spectral information of HSI by integrating the image spatial feature extraction advantage of convolutional neural network(CNN)with the adaptive capability and long-distance dependency extraction advantage of Transformers.HFEDB-Net consists of a high-frequency information enhancement branch and a backbone branch.In the high-frequency information enhancement branch,the high-frequency information of low-resolution and high-resolution HSI is extracted by using Laplacian pyramids,and the results serve as the input and label for the high-frequency branch.A spectral-enhanced Transformer is employed as the feature extraction method for this branch.In the backbone branch,a CNN with channel attention is utilized to extract spatial features and spectral information comprehensively.Finally,the results from both branches are combined through CNN to obtain the final reconstructed image.Additionally,the attention mechanism and encoder layers of the Transformer are respectively improved by using multi-head attention and multi-scale strategies to Research and Development研究开发217 better extract spatial and spectral information from HSI.Experimental results demonstrate that HFEDB-Net outperforms current state-of-the-art methods in terms of quantitative evaluation metrics and visual effects on two public datasets.