Research on Restoration of Murals Based on Diffusion Model and Transformer

Due to the limitations of a priori knowledge and convolution operation,the existing image restoration techniques cannot be directly applied to the cultural relics mural restoration,in order to more accurately restore the original appearance of the cultural relics mural images,an image restoration based on the denoising diffusion probability model(Denoising Diffusion Probability Model(DDPM))and the Transformer method.The process involves two steps:in the first step,the damaged mural image is firstly utilized as the condition to generate the noise image,using the time,condition and noise image patch as the inputs to the noise prediction network,capturing the global dependencies in the input sequence through the multi-attentionmechanismof the input sequence and feedforward neural network processing,and designing a long skip connection between the shallow and deep layers in the Transformer blocks between the shallow and deep layers using long skip connections to fuse the feature information of global and local outputs to maintain the overall consistency of the restoration results;In the second step,taking the noisy image as a condition to direct the diffusion model to back sample to generate the restored image.Experiment results show that the PSNR and SSIM of the proposedmethod are improved by 2%to 9%and 1%to 3.3%,respectively,which are compared to the comparison methods.This study proposed synthesizes the advantages of the diffusionmodel and deep learningmodel to make themural restoration results more accurate.

Back in time:digital restoration techniques for the millennium Dunhuang murals

In the long history of more than 1 500 years,Dunhuang murals suffered from various deteriorations causing irreversible damage such as falling off,fading,and so on.However,the existing Dunhuang mural restoration methods are time-consuming and not feasible to facilitate cultural dissemination and permanent inheritance.Inspired by cultural computing using artificial intelligence,gated-convolution-based dehaze net(GD-Net) was proposed for Dunhuang mural refurbishment and comprehensive protection.First,a neural network with gated convolution was applied to restore the falling off areas of the mural to ensure the integrity of the mural content.Second,a dehaze network was applied to enhance image quality to cope with the fading of the mural.Besides,a Dunhuang mural dataset was presented to meet the needs of deep learning approach,containing 1 180 images from the Cave 290 and Cave 112 of the Mogao Grottoes.The experimental results demonstrate the effectiveness and superiority of GD-Net.