RESEARCH ON WEIGHTED PRIORITY OF EXEMPLAR-BASED IMAGE INPAINTING

The priority of the filled patch play a key role in the exemplar-based image inpainting, and it should be determined firstly to optimize the process of image inpainting. A modified image inpainting algorithm is proposed by weighted-priority based on the Criminisi algorithm. The improved algorithm demonstrates better relationship between the data term and the confidence term for the optimization of the priority than the classical Criminisi algorithm. By comparing the effect of the inpainted images with different structure, conclusion can be drawn that the optimal priority should be chosen properly for different images with different structures.

Deep learning-based inpainting of saturation artifacts in optical coherence tomography images

Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images.We proposed a deep learning-based inpainting method of saturation artifacts in this paper.The generation mechanism of saturation artifacts was analyzed,and experimental and simulated datasets were built based on the mechanism.Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs.The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility,strong generalization,and robustness.

Image Inpainting Technique Incorporating Edge Prior and Attention Mechanism

Recently,deep learning-based image inpainting methods have made great strides in reconstructing damaged regions.However,these methods often struggle to produce satisfactory results when dealing with missing images with large holes,leading to distortions in the structure and blurring of textures.To address these problems,we combine the advantages of transformers and convolutions to propose an image inpainting method that incorporates edge priors and attention mechanisms.The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details.This method divides the inpainting task into two phases:edge prediction and image inpainting.Specifically,in the edge prediction phase,a transformer architecture is designed to combine axial attention with standard self-attention.This design enhances the extraction capability of global structural features and location awareness.It also balances the complexity of self-attention operations,resulting in accurate prediction of the edge structure in the defective region.In the image inpainting phase,a multi-scale fusion attention module is introduced.This module makes full use of multi-level distant features and enhances local pixel continuity,thereby significantly improving the quality of image inpainting.To evaluate the performance of our method.comparative experiments are conducted on several datasets,including CelebA,Places2,and Facade.Quantitative experiments show that our method outperforms the other mainstream methods.Specifically,it improves Peak Signal-to-Noise Ratio(PSNR)and Structure Similarity Index Measure(SSIM)by 1.141~3.234 db and 0.083~0.235,respectively.Moreover,it reduces Learning Perceptual Image Patch Similarity(LPIPS)and Mean Absolute Error(MAE)by 0.0347~0.1753 and 0.0104~0.0402,respectively.Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details.Furthermore,our model exhibits impressive performance in terms of the number of parameters,memory cost,and testing time.

Spatial and Contextual Path Network for Image Inpainting

Image inpainting is a kind of use known area of information technology to repair the loss or damage to the area.Image feature extraction is the core of image restoration.Getting enough space for information and a larger receptive field is very important to realize high-precision image inpainting.However,in the process of feature extraction,it is difficult to meet the two requirements of obtaining sufficient spatial information and large receptive fields at the same time.In order to obtain more spatial information and a larger receptive field at the same time,we put forward a kind of image restoration based on space path and context path network.For the space path,we stack three convolution layers for 1/8 of the figure,the figure retained the rich spatial details.For the context path,we use the global average pooling layer,where the accept field is the maximum of the backbone network,and the pooling module can provide global context information for the maximum accept field.In order to better integrate the features extracted from the spatial and contextual paths,we study the fusion module of the two paths.Features fusionmodule first path output of the space and context path,and then through themass normalization to balance the scale of the characteristics,finally the characteristics of the pool will be connected into a feature vector and calculate the weight vector.Features of images in order to extract context information,we add attention to the context path refinement module.Attention modules respectively from channel dimension and space dimension to weighted images,in order to obtain more effective information.Experiments show that our method is better than the existing technology in the quality and quantity of themethod,and further to expand our network to other inpainting networks,in order to achieve consistent performance improvements.

Multi-Layer Deep Sparse Representation for Biological Slice Image Inpainting

Biological slices are an effective tool for studying the physiological structure and evolutionmechanism of biological systems.However,due to the complexity of preparation technology and the presence of many uncontrollable factors during the preparation processing,leads to problems such as difficulty in preparing slice images and breakage of slice images.Therefore,we proposed a biological slice image small-scale corruption inpainting algorithm with interpretability based on multi-layer deep sparse representation,achieving the high-fidelity reconstruction of slice images.We further discussed the relationship between deep convolutional neural networks and sparse representation,ensuring the high-fidelity characteristic of the algorithm first.A novel deep wavelet dictionary is proposed that can better obtain image prior and possess learnable feature.And multi-layer deep sparse representation is used to implement dictionary learning,acquiring better signal expression.Compared with methods such as NLABH,Shearlet,Partial Differential Equation(PDE),K-Singular Value Decomposition(K-SVD),Convolutional Sparse Coding,and Deep Image Prior,the proposed algorithm has better subjective reconstruction and objective evaluation with small-scale image data,which realized high-fidelity inpainting,under the condition of small-scale image data.And theOn2-level time complexitymakes the proposed algorithm practical.The proposed algorithm can be effectively extended to other cross-sectional image inpainting problems,such as magnetic resonance images,and computed tomography images.

Gait recognition based on Wasserstein generating adversarial image inpainting network

Aiming at the problem of small area human occlusion in gait recognition,a method based on generating adversarial image inpainting network was proposed which can generate a context consistent image for gait occlusion area.In order to reduce the effect of noise on feature extraction,the stacked automatic encoder with robustness was used.In order to improve the ability of gait classification,the sparse coding was used to express and classify the gait features.Experiments results showed the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases CASIA-B and TUM-GAID for gait recognition.

Self-adaptively inpainting gaps between electrical image pads according to the image fabric

Images created from measurements made by wireline microresistivity imaging tools have longitudinal gaps when the well circumference exceeds the total width of the pad-mounted electrode arrays.The gap size depends on the tool design and borehole size,and the null data in these gaps negatively aff ect the quantitative evaluation of reservoirs.Images with linear and texture features obtained from microresistivity image logs have distinct dual fabric features because of logging principles and various geological phenomena.Linear image features usually include phenomena such as fractures,bedding,and unconformities.Contrarily,texture-based image features usually indicate phenomena such as vugs and rock matrices.According to the characteristics of this fabric-based binary image structure and guided by the practice of geological interpretation,an adaptive inpainting method for the blank gaps in microresistivity image logs is proposed.For images with linear features,a sinusoidal tracking inpainting algorithm based on an evaluation of the validity and continuity of pixel sets is used.Contrarily,the most similar target transplantation algorithm is applied to texture-based images.The results obtained for measured electrical imaging data showed that the full borehole image obtained by the proposed method,whether it was a linear structural image refl ecting fracture and bedding or texture-based image refl ecting the matrix and pore of rock,had substantially good inpainting quality with enhanced visual connectivity.The proposed method was eff ective for inpainting electrical image logs with large gaps and high angle fractures with high heterogeneity.Moreover,ladder and block artifacts were rare,and the inpainting marks were not obvious.In addition,detailed full borehole images obtained by the proposed method will provide an essential basis for interpreting geological phenomena and reservoir parameters.

A 360-Degree Panoramic Image Inpainting Network Using a Cube Map

Inpainting has been continuously studied in the field of computer vision.As artificial intelligence technology developed,deep learning technology was introduced in inpainting research,helping to improve performance.Currently,the input target of an inpainting algorithm using deep learning has been studied from a single image to a video.However,deep learning-based inpainting technology for panoramic images has not been actively studied.We propose a 360-degree panoramic image inpainting method using generative adversarial networks(GANs).The proposed network inputs a 360-degree equirectangular format panoramic image converts it into a cube map format,which has relatively little distortion and uses it as a training network.Since the cube map format is used,the correlation of the six sides of the cube map should be considered.Therefore,all faces of the cube map are used as input for the whole discriminative network,and each face of the cube map is used as input for the slice discriminative network to determine the authenticity of the generated image.The proposed network performed qualitatively better than existing single-image inpainting algorithms and baseline algorithms.

Layered image inpainting based on image decomposition

We propose a layered image inpainting scheme based on image decomposition. The damaged image is first decomposed into three layers： cartoon, edge, and texture. The cartoon and edge layers are repaired using an adaptive offset operator that can fill-in damaged image blocks while preserving sharpness of edges. The missing information in the texture layer is generated with a texture synthesis method. By using discrete cosine transform （DCT） in image decomposition and trading between resolution and computation complexity in texture synthesis, the processing time is kept at a reasonable level.

Color Texture Image Inpainting Using the Non Local CTV Model

The classical TV (Total Variation) model has been applied to gray texture image denoising and inpainting previously based on the non local operators, but such model can not be directly used to color texture image inpainting due to coupling of different image layers in color images. In order to solve the inpainting problem for color texture images effectively, we propose a non local CTV (Color Total Variation) model. Technically, the proposed model is an extension of local TV model for gray images but we take account of the coupling of different layers in color images and make use of concepts of the non-local operators. As the coupling of different layers for color images in the proposed model will in-crease computational complexity, we also design a fast Split Bregman algorithm. Finally, some numerical experiments are conducted to validate the performance of the proposed model and its algorithm.

Digital image inpainting by example-based image synthesis method

A simple and effective image inpainting method is proposed in this paper, which is proved to be suitable for different kinds of target regions with shapes from little scraps to large unseemly objects in a wide range of images. It is an important improvement upon the traditional image inpainting techniques. By introducing a new bijeetive-mapping term into the matching cost function, the artificial repetition problem in the final inpainting image is practically solved. In addition, by adopting an inpainting error map, not only the target pixels are refined gradually during the inpainting process but also the overlapped target patches are combined more seamlessly than previous method. Finally, the inpainting time is dramatically decreased by using a new acceleration method in the matching process.

Image inpainting using complex 2-D dual-tree wavelet transform

The dual-tree complex wavelet transform is a useful tool in signal and image process- ing. In this paper, we propose a dual-tree complex wavelet transform （CWT） based algorithm for image inpalnting problem. Our approach is based on Cai, Chan, Shen and Shen＇s framelet-based algorithm. The complex wavelet transform outperforms the standard real wavelet transform in the sense of shift-invariance, directionality and anti-aliasing. Numerical results illustrate the good performance of our algorithm.

UFC-Net with Fully-Connected Layers and Hadamard Identity Skip Connection for Image Inpainting

Image inpainting is an interesting technique in computer vision and artificial intelligence for plausibly filling in blank areas of an image by referring to their surrounding areas.Although its performance has been improved significantly using diverse convolutional neural network(CNN)-based models,these models have difficulty filling in some erased areas due to the kernel size of the CNN.If the kernel size is too narrow for the blank area,the models cannot consider the entire surrounding area,only partial areas or none at all.This issue leads to typical problems of inpainting,such as pixel reconstruction failure and unintended filling.To alleviate this,in this paper,we propose a novel inpainting model called UFC-net that reinforces two components in U-net.The first component is the latent networks in the middle of U-net to consider the entire surrounding area.The second component is the Hadamard identity skip connection to improve the attention of the inpainting model on the blank areas and reduce computational cost.We performed extensive comparisons with other inpainting models using the Places2 dataset to evaluate the effectiveness of the proposed scheme.We report some of the results.

ORTHOGONAL-DIRECTIONAL FORWARD DIFFUSION IMAGE INPAINTING AND DENOISING MODEL

In this paper,an orthogonal-directional forward diffusion Partial Differential Equation(PDE) image inpainting and denoising model which processes image based on variation problem is proposed.The novel model restores the damaged information and smoothes the noise in image si-multaneously.The model is morphological invariant which processes image based on the geometrical property.The regularization item of it diffuses along and cross the isophote,and then the known image information is transported into the target region through two orthogonal directions.The cross isophote diffusion part is the TV(Total Variation) equation and the along isophote diffusion part is the inviscid Helmholtz vorticity equation.The equivalence between the Helmholtz equation and the inpainting PDEs is proved.The model with the fidelity item which is used in the whole image domain denoises while preserving edges.So the novel model could inpaint and denoise simultaneously.Both theoretical analysis and experiments have verified the validity of the novel model proposed in this paper.

Image Inpainting Detection Based on High-Pass Filter Attention Network

Image inpainting based on deep learning has been greatly improved.The original purpose of image inpainting was to repair some broken photos, suchas inpainting artifacts. However, it may also be used for malicious operations,such as destroying evidence. Therefore, detection and localization of imageinpainting operations are essential. Recent research shows that high-pass filteringfull convolutional network (HPFCN) is applied to image inpainting detection andachieves good results. However, those methods did not consider the spatial location and channel information of the feature map. To solve these shortcomings, weintroduce the squeezed excitation blocks (SE) and propose a high-pass filter attention full convolutional network (HPACN). In feature extraction, we apply concurrent spatial and channel attention (scSE) to enhance feature extraction and obtainmore information. Channel attention (cSE) is introduced in upsampling toenhance detection and localization. The experimental results show that the proposed method can achieve improvement on ImageNet.

SemID: Blind Image Inpainting with Semantic Inconsistency Detection

Most existing image inpainting methods aim to fill in the missing content in the inside-hole region of the target image. However, the areas to be restored in realistically degraded images are unspecified. Previous studies have failed to recover the degradations due to the absence of the explicit mask indication. Meanwhile, inconsistent patterns are blended complexly with the image content. Therefore, estimating whether certain pixels are out of distribution and considering whether the object is consistent with the context is necessary. Motivated by these observations, a two-stage blind image inpainting network, which utilizes global semantic features of the image to locate semantically inconsistent regions and then generates reasonable content in the areas, is proposed. Specifically, the representation differences between inconsistent and available content are first amplified, iteratively predicting the region to be restored from coarse to fine. A confidence-driven inpainting network based on prediction masks is then used to estimate the information regarding missing regions. Furthermore, a multiscale contextual aggregation module is introduced for spatial feature transfer to refine the generated contents. Extensive experiments over multiple datasets demonstrate that the proposed method can generate visually plausible and structurally complete results that are particularly effective in recovering diverse degraded images.

IDENTIFICATION OF INPAINTED IMAGES AND NATURAL IMAGES FOR DIGITAL FORENSICS

Image forensics is a form of image analysis for finding out the condition of an image in the complete absence of any digital watermark or signature.It can be used to authenticate digital images and identify their sources.While the technology of exemplar-based inpainting provides an approach to remove objects from an image and play visual tricks.In this paper, as a first attempt, a method based on zero-connectivity feature and fuzzy membership is proposed to discriminate natural images from inpainted images.Firstly, zero-connectivity labeling is applied on block pairs to yield matching degree feature of all blocks in the region of suspicious, then the fuzzy memberships are computed and the tampered regions are identified by a cut set.Experimental results demonstrate the effectiveness of our method in detecting inpainted images.

图像Inpainting技术原理及在包装印刷图像处理中的应用

在包装印刷行业图像修复问题需要有经验的技术人员进行复杂的手工处理,随着计算机图像处理领域对图像自动处理技术的讨论,INPAINTING技术对包装印刷图像处理提供了新的方法和方向。主要介绍了图像自动修复技术的原理、发展,以及在包装印刷行业的应用。

Image completion algorithm based on texture synthesis

A new algorithm is proposed for completing the missing parts caused by the removal of foreground or background elements from an image of natural scenery in a visually plausible way. The major contributions of the proposed algorithm are： （1） for most natural images, there is a strong orientation of texture or color distribution. So a method is introduced to compute the main direction of the texture and complete the image by limiting the search to one direction to carry out image completion quite fast; （2） there exists a synthesis ordering for image completion. The searching order of the patches is defined to ensure the regions with more known information and the structures should be completed before filling in other regions; （3） to improve the visual effect of texture synthesis, an adaptive scheme is presented to determine the size of the template window for capturing the features of various scales. A number of examples are given to demonstrate the effectiveness of the proposed algorithm.

Wavelet Inpainting Based on p-Laplace Operator

填失踪或损坏的小浪系数的问题在这篇论文被考虑。禅宗，沈，和周建议了二个全部的变化(电视) 小浪 inpainting 模型解决这个问题。电视模型的主要利益是它能保留边很好，但是这个方法受不了楼梯效果。克服这个缺点，我们在本地坐标分析电视模型和 p-Laplace 操作员的物理特征，并且解释 p-Laplace 的那散开表演实质上比电视模型的优异。以后，一个 inpainting 模型基于为损坏小浪系数的 p-Laplace 操作员被介绍。而它能仍然使边以及电视模型锋利，这个新模型能有效地在电视模型减少楼梯效果。实验结果证明更好的 inpaingting 质量能与与我们的模型一起计算时间的少得多被完成。