基于注意力残差网络的快照式多光谱相机图像重构

Image reconstruction of snapshot multispectral camera based on an attention residual network

随着光谱成像技术的飞速发展,使用多光谱滤光片阵列(multispectral filter array,MSFA)采集多光谱图像的空间和光谱信息已经成为研究热点。如何利用低采样率且具有强频谱互相关性的原始数据进行重构成为制约其发展的瓶颈。本文基于一种含有全通波段的8波段4×4 MSFA,提出了一种空谱联合的多分支注意力残差网络模型。使用多分支模型对各个波段插值后的图像特征进行学习。利用本文设计的空间通道注意力模型对8个波段和全通波段的特征信息进行联合处理。该模型通过多层卷积和卷积注意力模块以及残差补偿机制,有效减小了各波段的颜色差异,增强了边缘纹理等相关特征信息。对于初步插值的全通波段和其他波段的特征信息,通过无需进行批量归一化的残差密集块对多光谱图像空间和光谱相关性进行特征学习,以匹配各个波段的光谱信息。实验结果表明,对于在D65光源下测试图像,本文所提模型的峰值信噪比、结构相似度和光谱角相似度分别较最先进的深度学习方法提升了3.46%、0.27%和6%。该方法不仅减少了伪影还获得了更多的纹理细节。

With the rapid advancement of spectral imaging technology,the use of multispectral filter array(MSFA)to collect the spatial and spectral information of multispectral images has become a research hotspot.The uses of the original data are limited because of its low sampling rate and strong spectral inter-correlation for reconstruction.Therefore,we propose a multi-branch attention residual network model for spatial-spectral association based on an 8-band 4×4 MSFA with all-pass bands.First,the multi-branch model was used to learn the image features after interpolation in each band;second,the feature information of the eight bands and the all-pass band were united by the spatial channel attention model designed in this paper,and the application of multi-layer convolution and the convolutional attention module and the use of residual compensation effectively compensated the color difference of each band and enriched the edge texture-related feature information.Finally,the preliminary interpolated full-pass band and the rest of the band feature information were used for feature learning of the spatial and spectral correlations of multispectral images through residual dense blocks without batch normalization to match the spectral information of each band.Experimental results show that the peak signal-to-noise ratio,structural similarity,and spectral angular similarity of the test image under the D65 light source outperform the state-of-the-art deep learning method by 3.46%,0.27%,and 6%,respectively.This method not only reduces artifacts but also obtains more texture details.