RESEARCH ON WEIGHTED PRIORITY OF EXEMPLAR-BASED IMAGE INPAINTING

The priority of the filled patch play a key role in the exemplar-based image inpainting, and it should be determined firstly to optimize the process of image inpainting. A modified image inpainting algorithm is proposed by weighted-priority based on the Criminisi algorithm. The improved algorithm demonstrates better relationship between the data term and the confidence term for the optimization of the priority than the classical Criminisi algorithm. By comparing the effect of the inpainted images with different structure, conclusion can be drawn that the optimal priority should be chosen properly for different images with different structures.

Deep learning-based inpainting of saturation artifacts in optical coherence tomography images

Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images.We proposed a deep learning-based inpainting method of saturation artifacts in this paper.The generation mechanism of saturation artifacts was analyzed,and experimental and simulated datasets were built based on the mechanism.Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs.The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility,strong generalization,and robustness.

Image Inpainting Technique Incorporating Edge Prior and Attention Mechanism

Recently,deep learning-based image inpainting methods have made great strides in reconstructing damaged regions.However,these methods often struggle to produce satisfactory results when dealing with missing images with large holes,leading to distortions in the structure and blurring of textures.To address these problems,we combine the advantages of transformers and convolutions to propose an image inpainting method that incorporates edge priors and attention mechanisms.The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details.This method divides the inpainting task into two phases:edge prediction and image inpainting.Specifically,in the edge prediction phase,a transformer architecture is designed to combine axial attention with standard self-attention.This design enhances the extraction capability of global structural features and location awareness.It also balances the complexity of self-attention operations,resulting in accurate prediction of the edge structure in the defective region.In the image inpainting phase,a multi-scale fusion attention module is introduced.This module makes full use of multi-level distant features and enhances local pixel continuity,thereby significantly improving the quality of image inpainting.To evaluate the performance of our method.comparative experiments are conducted on several datasets,including CelebA,Places2,and Facade.Quantitative experiments show that our method outperforms the other mainstream methods.Specifically,it improves Peak Signal-to-Noise Ratio(PSNR)and Structure Similarity Index Measure(SSIM)by 1.141~3.234 db and 0.083~0.235,respectively.Moreover,it reduces Learning Perceptual Image Patch Similarity(LPIPS)and Mean Absolute Error(MAE)by 0.0347~0.1753 and 0.0104~0.0402,respectively.Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details.Furthermore,our model exhibits impressive performance in terms of the number of parameters,memory cost,and testing time.

Spatial and Contextual Path Network for Image Inpainting

Image inpainting is a kind of use known area of information technology to repair the loss or damage to the area.Image feature extraction is the core of image restoration.Getting enough space for information and a larger receptive field is very important to realize high-precision image inpainting.However,in the process of feature extraction,it is difficult to meet the two requirements of obtaining sufficient spatial information and large receptive fields at the same time.In order to obtain more spatial information and a larger receptive field at the same time,we put forward a kind of image restoration based on space path and context path network.For the space path,we stack three convolution layers for 1/8 of the figure,the figure retained the rich spatial details.For the context path,we use the global average pooling layer,where the accept field is the maximum of the backbone network,and the pooling module can provide global context information for the maximum accept field.In order to better integrate the features extracted from the spatial and contextual paths,we study the fusion module of the two paths.Features fusionmodule first path output of the space and context path,and then through themass normalization to balance the scale of the characteristics,finally the characteristics of the pool will be connected into a feature vector and calculate the weight vector.Features of images in order to extract context information,we add attention to the context path refinement module.Attention modules respectively from channel dimension and space dimension to weighted images,in order to obtain more effective information.Experiments show that our method is better than the existing technology in the quality and quantity of themethod,and further to expand our network to other inpainting networks,in order to achieve consistent performance improvements.

Multi-Layer Deep Sparse Representation for Biological Slice Image Inpainting

Biological slices are an effective tool for studying the physiological structure and evolutionmechanism of biological systems.However,due to the complexity of preparation technology and the presence of many uncontrollable factors during the preparation processing,leads to problems such as difficulty in preparing slice images and breakage of slice images.Therefore,we proposed a biological slice image small-scale corruption inpainting algorithm with interpretability based on multi-layer deep sparse representation,achieving the high-fidelity reconstruction of slice images.We further discussed the relationship between deep convolutional neural networks and sparse representation,ensuring the high-fidelity characteristic of the algorithm first.A novel deep wavelet dictionary is proposed that can better obtain image prior and possess learnable feature.And multi-layer deep sparse representation is used to implement dictionary learning,acquiring better signal expression.Compared with methods such as NLABH,Shearlet,Partial Differential Equation(PDE),K-Singular Value Decomposition(K-SVD),Convolutional Sparse Coding,and Deep Image Prior,the proposed algorithm has better subjective reconstruction and objective evaluation with small-scale image data,which realized high-fidelity inpainting,under the condition of small-scale image data.And theOn2-level time complexitymakes the proposed algorithm practical.The proposed algorithm can be effectively extended to other cross-sectional image inpainting problems,such as magnetic resonance images,and computed tomography images.

Color Texture Image Inpainting Using the Non Local CTV Model

The classical TV (Total Variation) model has been applied to gray texture image denoising and inpainting previously based on the non local operators, but such model can not be directly used to color texture image inpainting due to coupling of different image layers in color images. In order to solve the inpainting problem for color texture images effectively, we propose a non local CTV (Color Total Variation) model. Technically, the proposed model is an extension of local TV model for gray images but we take account of the coupling of different layers in color images and make use of concepts of the non-local operators. As the coupling of different layers for color images in the proposed model will in-crease computational complexity, we also design a fast Split Bregman algorithm. Finally, some numerical experiments are conducted to validate the performance of the proposed model and its algorithm.

A 360-Degree Panoramic Image Inpainting Network Using a Cube Map

Inpainting has been continuously studied in the field of computer vision.As artificial intelligence technology developed,deep learning technology was introduced in inpainting research,helping to improve performance.Currently,the input target of an inpainting algorithm using deep learning has been studied from a single image to a video.However,deep learning-based inpainting technology for panoramic images has not been actively studied.We propose a 360-degree panoramic image inpainting method using generative adversarial networks(GANs).The proposed network inputs a 360-degree equirectangular format panoramic image converts it into a cube map format,which has relatively little distortion and uses it as a training network.Since the cube map format is used,the correlation of the six sides of the cube map should be considered.Therefore,all faces of the cube map are used as input for the whole discriminative network,and each face of the cube map is used as input for the slice discriminative network to determine the authenticity of the generated image.The proposed network performed qualitatively better than existing single-image inpainting algorithms and baseline algorithms.

Image inpainting using complex 2-D dual-tree wavelet transform

The dual-tree complex wavelet transform is a useful tool in signal and image process- ing. In this paper, we propose a dual-tree complex wavelet transform （CWT） based algorithm for image inpalnting problem. Our approach is based on Cai, Chan, Shen and Shen＇s framelet-based algorithm. The complex wavelet transform outperforms the standard real wavelet transform in the sense of shift-invariance, directionality and anti-aliasing. Numerical results illustrate the good performance of our algorithm.

UFC-Net with Fully-Connected Layers and Hadamard Identity Skip Connection for Image Inpainting

Image inpainting is an interesting technique in computer vision and artificial intelligence for plausibly filling in blank areas of an image by referring to their surrounding areas.Although its performance has been improved significantly using diverse convolutional neural network(CNN)-based models,these models have difficulty filling in some erased areas due to the kernel size of the CNN.If the kernel size is too narrow for the blank area,the models cannot consider the entire surrounding area,only partial areas or none at all.This issue leads to typical problems of inpainting,such as pixel reconstruction failure and unintended filling.To alleviate this,in this paper,we propose a novel inpainting model called UFC-net that reinforces two components in U-net.The first component is the latent networks in the middle of U-net to consider the entire surrounding area.The second component is the Hadamard identity skip connection to improve the attention of the inpainting model on the blank areas and reduce computational cost.We performed extensive comparisons with other inpainting models using the Places2 dataset to evaluate the effectiveness of the proposed scheme.We report some of the results.

Image Inpainting Detection Based on High-Pass Filter Attention Network

Image inpainting based on deep learning has been greatly improved.The original purpose of image inpainting was to repair some broken photos, suchas inpainting artifacts. However, it may also be used for malicious operations,such as destroying evidence. Therefore, detection and localization of imageinpainting operations are essential. Recent research shows that high-pass filteringfull convolutional network (HPFCN) is applied to image inpainting detection andachieves good results. However, those methods did not consider the spatial location and channel information of the feature map. To solve these shortcomings, weintroduce the squeezed excitation blocks (SE) and propose a high-pass filter attention full convolutional network (HPACN). In feature extraction, we apply concurrent spatial and channel attention (scSE) to enhance feature extraction and obtainmore information. Channel attention (cSE) is introduced in upsampling toenhance detection and localization. The experimental results show that the proposed method can achieve improvement on ImageNet.

ORTHOGONAL-DIRECTIONAL FORWARD DIFFUSION IMAGE INPAINTING AND DENOISING MODEL

In this paper,an orthogonal-directional forward diffusion Partial Differential Equation(PDE) image inpainting and denoising model which processes image based on variation problem is proposed.The novel model restores the damaged information and smoothes the noise in image si-multaneously.The model is morphological invariant which processes image based on the geometrical property.The regularization item of it diffuses along and cross the isophote,and then the known image information is transported into the target region through two orthogonal directions.The cross isophote diffusion part is the TV(Total Variation) equation and the along isophote diffusion part is the inviscid Helmholtz vorticity equation.The equivalence between the Helmholtz equation and the inpainting PDEs is proved.The model with the fidelity item which is used in the whole image domain denoises while preserving edges.So the novel model could inpaint and denoise simultaneously.Both theoretical analysis and experiments have verified the validity of the novel model proposed in this paper.

IDENTIFICATION OF INPAINTED IMAGES AND NATURAL IMAGES FOR DIGITAL FORENSICS

Image forensics is a form of image analysis for finding out the condition of an image in the complete absence of any digital watermark or signature.It can be used to authenticate digital images and identify their sources.While the technology of exemplar-based inpainting provides an approach to remove objects from an image and play visual tricks.In this paper, as a first attempt, a method based on zero-connectivity feature and fuzzy membership is proposed to discriminate natural images from inpainted images.Firstly, zero-connectivity labeling is applied on block pairs to yield matching degree feature of all blocks in the region of suspicious, then the fuzzy memberships are computed and the tampered regions are identified by a cut set.Experimental results demonstrate the effectiveness of our method in detecting inpainted images.

图像Inpainting技术原理及在包装印刷图像处理中的应用

在包装印刷行业图像修复问题需要有经验的技术人员进行复杂的手工处理,随着计算机图像处理领域对图像自动处理技术的讨论,INPAINTING技术对包装印刷图像处理提供了新的方法和方向。主要介绍了图像自动修复技术的原理、发展,以及在包装印刷行业的应用。

Image completion algorithm based on texture synthesis

A new algorithm is proposed for completing the missing parts caused by the removal of foreground or background elements from an image of natural scenery in a visually plausible way. The major contributions of the proposed algorithm are： （1） for most natural images, there is a strong orientation of texture or color distribution. So a method is introduced to compute the main direction of the texture and complete the image by limiting the search to one direction to carry out image completion quite fast; （2） there exists a synthesis ordering for image completion. The searching order of the patches is defined to ensure the regions with more known information and the structures should be completed before filling in other regions; （3） to improve the visual effect of texture synthesis, an adaptive scheme is presented to determine the size of the template window for capturing the features of various scales. A number of examples are given to demonstrate the effectiveness of the proposed algorithm.

Wavelet Inpainting Based on p-Laplace Operator

填失踪或损坏的小浪系数的问题在这篇论文被考虑。禅宗，沈，和周建议了二个全部的变化(电视) 小浪 inpainting 模型解决这个问题。电视模型的主要利益是它能保留边很好，但是这个方法受不了楼梯效果。克服这个缺点，我们在本地坐标分析电视模型和 p-Laplace 操作员的物理特征，并且解释 p-Laplace 的那散开表演实质上比电视模型的优异。以后，一个 inpainting 模型基于为损坏小浪系数的 p-Laplace 操作员被介绍。而它能仍然使边以及电视模型锋利，这个新模型能有效地在电视模型减少楼梯效果。实验结果证明更好的 inpaingting 质量能与与我们的模型一起计算时间的少得多被完成。

Overview of Digital Image Restoration

Image restoration is an image processing technology with great practical value in the field of computer vision.It is a computer technology that estimates the image information of the damaged area according to the residual image information of the damaged image and carries out automatic repair.This article firstly classify and summarize image restoration algorithms,and describe recent advances in the research respectively from three aspects including image restoration based on partial differential equation,based on the texture of image restoration and based on deep learning,then make the brief analysis of digital image restoration of subjective and objective evaluation method,and briefly summarize application of digital image restoration technique in the future and prospects,provide direction for the research on image after repair.

An Optimal Weight Method for CT Image Denoising

This paper proposes a novel exemplar- based method for reducing noise in computed tomography （CT） images. In the proposed method, denoising is performed on each block with the help of a given database of standard image blocks. For each noisy block, its denoised version is the best sparse positive linear combination of the blocks in the database. We formulate the problem as a constrained optimization problem such that the solution is the denoised block. Experimental results demonstrate the good performance of the proposed method over current state-of-the-art denoising methods, in terms of both objective and subjective evaluations.

Super-resolution inpainting

Image or video resources are often received in poor condition, mostly with noise or defects making the resources hard to read. We propose an effective algorithm based on digital image inpainting. The mechanism can be used in restoring images or video frames with very high noise or defect ratio (e.g., 90%). The algorithm is based on the concept of image subdivision and estimation of color variations. Noises inside blocks of different sizes are inpainted with different levels of surrounding information. The results showed that an almost unrecognizable image can be recovered with visually good result. The algorithm can be further extended for processing motion picture with high percentage of noise.

Pyramid-VAE-GAN:Transferring hierarchical latent variables for image inpainting

Significant progress has been made in image inpainting methods in recent years.However,they are incapable of producing inpainting results with reasonable structures,rich detail,and sharpness at the same time.In this paper,we propose the Pyramid-VAE-GAN network for image inpainting to address this limitation.Our network is built on a variational autoencoder(VAE)backbone that encodes high-level latent variables to represent complicated high-dimensional prior distributions of images.The prior assists in reconstructing reasonable structures when inpainting.We also adopt a pyramid structure in our model to maintain rich detail in low-level latent variables.To avoid the usual incompatibility of requiring both reasonable structures and rich detail,we propose a novel cross-layer latent variable transfer module.This transfers information about long-range structures contained in high-level latent variables to low-level latent variables representing more detailed information.We further use adversarial training to select the most reasonable results and to improve the sharpness of the images.Extensive experimental results on multiple datasets demonstrate the superiority of our method.Our code is available at https://github.com/thy960112/Pyramid-VAE-GAN.