全局-局部注意力特征重用高光谱图像超分辨率网络

Hyperspectral Image Super-Resolution Network of Local-Global Attention Feature Reuse

为了解决高光谱图像超分辨率重建中物质本征光谱表达能力不足、图像尺度变化过程中存在细节信息损失的问题,提出一种全局-局部注意力特征重用网络。首先,通过多节点特征重用增强网络的多尺度信息提取能力。其次,引入局部注意力,利用空间注意力机制聚焦重点空间信息,并通过通道注意力机制增强特异性光谱提取能力。最后,设计全局修正模块,根据原始多光谱图像空间信息丰富与高光谱图像光谱保真度高的特性进一步补偿处理过程中损失的空间与光谱维度信息,提高网络可靠性。选取CAVE和Harvard数据集进行训练与测试,并与多种先进方法进行定量与定性评估。结果表明,所提网络能够重建出更高分辨率的高光谱图像,更好完成高光谱超分辨率任务。

Objective Hyperspectral images usually need to sacrifice spatial resolution to improve spectral resolution,which will lead to the emergence of a large number of mixed pixels and seriously affect the performance of subsequent applications.Convolutional neural networks(CNNs)can maximize the spatial resolution of hyperspectral images on the premise of ensuring spectral information integrity by fusing multispectral images.The 2D convolution scheme adopts 1D convolution and 2D convolution respectively for feature extraction of spectral and spatial information.However,1D convolution can only take global spectral information into account and lacks attention to complementary spectral information between adjacent pixels,which easily results in insufficient spectral feature extraction.3D convolution will introduce a large number of network parameters,limiting the depth and width of the design.Meanwhile,most network designs pay more attention to spatial feature extraction but ignore spectral dimension information to easily cause spectral confusion.In addition,as the network deepens,both spatial and spectral dimensions will lose information.The residual connection can alleviate this problem to a certain extent by ignoring the information differences among different input images,and it is difficult to employ the original input information to compensate the network.Therefore,hyperspectral image superresolution needs to enhance the extraction of spectral information and improve the spatial resolution of images.In addition,the design should strengthen the adaptability to hyperspectral images to ensure that the network can accurately take advantage of the characteristics of different input images.Methods To solve the insufficient utilization of intrinsic spectral features and supplement information more effectively in the fusion-based hyperspectral image super-resolution method,we propose a global-local attention feature reuse network(LGAR-Net).The network adopts low-resolution hyperspectral images and multispectral images with bicubic interpolation as the original input and designs a progressive structure.The progressive network leverages a few bands for the initial build first and gradually adds more band information to fine-tune the details for more accurate reconstruction effects.The network optimization features a reuse mechanism to preserve multi-scale spatial information while reducing the parameter number.Each progressive stage contains local attention blocks which employ spatial attention to enhance spatial information extraction and channel attention to supplement spectral information representation ability.Finally,we design a global correction module.According to the characteristics of a high spatial abundance of multispectral images and high spectral fidelity of hyperspectral images,The module adopts a global attention mechanism to focus information of different dimensions on the two kinds of original inputs to supplement targeted global information and ensure network stability.Results and Discussions To achieve the balance between performance and parameters,we design the module performance experiments to determine the specific number of extraction modules in feature reuse modules(Table 1).At the same time,we perform an ablation on the feature reuse connection,local attention blocks,and global correction module to determine the effectiveness of each core module(Table 2).In the comparative experiment,LGAR-Net is compared with other six representative advanced algorithms through quantitative evaluation,and two datasets of CAVE and Harvard are selected for evaluation.In the CAVE dataset results,LGAR-Netnet×4 magnification results in PSNR and SSIM,SAM,and EGRA respectively reach 51.244 dB,0.9644,1.703,and 0.392,and prove the network advancement(Table 3).Additionally,LGAR-Net yields the best performance in both×8 and×16 magnification tasks,which verifies its strong adaptability to different magnification factors.LGAR-Net still achieves the best results by comparing the results of×4,×8,and×16 magnification in the Harvard dataset,which further proves the network generalization(Table 4).We carry out some qualitative experiments to further evaluate the model performance.The absolute error map is employed to reflect the differences between the reconstructed image and the real image(Fig.6).In addition,we compare the spectral curve to reflect the spectral distortion condition(Fig.7).The results of qualitative experiments also prove the superior performance of LGAR-Net.Conclusions In this paper,we propose a hyperspectral image super-resolution network named LGAR-Net to obtain highresolution hyperspectral images by integrating information from low-resolution hyperspectral images and multispectral images.The network refines the reconstruction effect continuously through the progressive network and adopts the feature reuse mechanism to retain multiple granularity information.Local attention is utilized to enhance spectral information extraction,and global attention is to make information compensation by the original input image characteristics for strengthening the network adaptation to hyperspectral images.In addition,the optimal number of modules is analyzed in the network design experiments,and the effectiveness of innovation points is proven by the ablation experiment.In the comparative experiment,LGAR-Net conducts quantitative and qualitative evaluations alongside other six methods on the CAVE and Harvard datasets.Across various magnifications,LGAR-Net consistently achieves outstanding results,demonstrating its effectiveness and advanced capabilities.