基于Transformer和多尺度CNN的图像去模糊

Image Deblurring Based on Transformer and Multi-scale CNN

卷积神经网络(CNN)单独应用于图像去模糊时感受野受限,Transformer能有效缓解这一问题但计算复杂度随输入图像空间分辨率的增加呈2次方增长。为此,提出一种基于Transformer和多尺度CNN的图像去模糊网络(T-MIMO-UNet)。利用多尺度CNN提取空间特征,并嵌入Transformer全局特性捕获远程像素信息。设计局部增强Transformer模块、局部多头自注意力计算网络和增强前馈网络,采用窗口的方式进行局部逐块多头自注意力计算,通过增加深度可分离卷积层,加强不同窗口之间的信息交互。在GoPro测试数据集上的实验结果表明,T-MIMO-UNet的峰值信噪比相比于MIMO-UNet、DeepDeblur、DeblurGAN、SRN网络分别提升了0.39 dB、2.89 dB、3.42 dB、1.86 dB,参数量相比于MPRNet减少了1/2,能有效解决动态场景下的图像模糊问题。

Convolutional Neural Network(CNN)has limitations when applied solely to image deblurring tasks with restricted receptive fields.Transformer can effectively mitigate these limitations.However,the computational complexity increases quadratically as the spatial resolution of the input image increases.Therefore,this study proposes an image deblurring network based on Transformer and multi-scale CNN called T-MIMO-UNet.The multi-scale CNN is used to extract spatial features while the global feature of the Transformer is employed to capture remote pixel information.The local enhanced Transformer module,local Multi-Head Self-Attention(MHSA)computing network,and Enhanced Feed-Forward Network(EFFN)are designed.The block-by-block MHSA computation is performed using a windowing approach.The information interaction between different windows is enhanced by increasing the depth of the separable convolution layer.The results of the experiment conducted using the GoPro test dataset demonstrate that the Peak Signal-to-Noise Ratio(PSNR)of the T-MIMO-UNet increases by 0.39 dB,2.89 dB,3.42 dB,and 1.86 dB compared to the MIMO-UNet,DeepDeblur,DeblurGAN,and SRN networks,respectively.Additionally,the number of parameters is reduced by 1/2 compared to MPRNet.These findings prove that the T-MIMO-UNet effectively addresses the challenge of image blurring in dynamic scenes.