基于稀疏表示的物体图像修复

Object Image Inpainting Based on Sparse Representation

针对图像中物体缺失区域较大且具有复杂结构和纹理的情况,现有方法的修复结果会出现修复区域模糊和与周围已知区域结构衔接不连贯等情况.本文以修复缺失结构信息和复杂纹理的物体图像为目标,提出一种基于稀疏表示的物体图像修复算法.该问题被分解为轮廓修复和其他区域的纹理修复,物体缺失区域的轮廓块用已知区域轮廓块稀疏表示,而轮廓块间的稀疏关系来自相应的结构高度相似的参考物体轮廓.同时,为降低对参考物体选择的要求,本文提出的算法建立了两个轮廓之间的相似变换模型对参考物体做变形编辑以提高其轮廓与待修复物体轮廓形状的匹配度.在其他区域纹理修复中,本文提出一种基于图像平滑的修复优先级和搜索区域划分方法,减少了纹理中幅值较大的梯度对修复顺序计算的影响,更好地修复纹理结构.实验表明,该算法能利用参考图修复缺失独特结构的物体,较好地修复各种物体图像的弯曲轮廓和不规则纹理.与现有图像修复方法比较,该方法在修复具有复杂结构与纹理的物体方面获得了更好的结果.

Image inpainting is a tough task. When the region to be filled is large and complex structure and texture of an object is absent in the region, the existing image inpainting methods will result in blurred object contour or inconsistent structure with surrounding known region. In this paper, an image inpainting algorithm based on sparse representation is proposed to inpaint the object image with missing structure and complex texture. The problem is decomposed into two sub-problems: contour inpainting and texture inpainting. For contour inpainting, each contour block in the missing region is sparsely represented by contour blocks in the known region. If two contour shapes are highly similar, the structure sparse relation is approximated to the same. To avoid sparse relation unreasonable by missing information in inpainted contour block, sparse coefficients are derived from corresponding reference object contour with high similarity. A series of contour block pairs between two contours are obtained by contour matching. That is, for each unknown contour block (UB) in the object to be inpainted, there is a corresponding contour block (UCB) in reference object. Similarly, for each known contour block (KB) in the object to be inpainted, there is also a corresponding contour block (KCB) in reference object. Each UCB can be sparsely represented by KCBs and the sparse coefficients are calculated with those known UCB and KCBs. Then UB is expressed by those sparse coefficients and KBs. To improve the contour similarity of reference object to inpainted object, the algorithm provides a similarity transformation model to edit the reference object. Moving least squares technique is adopted to make the shape of reference object contour match that of inpainted object contour and preserve the shape of reference object. Specially, first reference contour is deformed following unpainted contour. By matching contour point pairs of two contours, a similarity matrix and a translation vector are calculated and act on unpainted contour points to get target positions of reference contour points. Then reference image is deformed to make contour points of reference object reach target positions. The transformation rule is that spatially-close pixels follow similar transformation. Using original positions and target positions of reference contour points, a similarity matrix and a translation vector for each pixel in reference image are calculated using moving least squares with the original spatial similarity between pixel and reference contour points as weights. The pixel new position is obtained by transforming original position with similarity matrix and translation vector. For texture inpainting, priority calculation and search region partition based on image smoothing is proposed. Image structure extraction based texture smoothing technique is adopted to reduce the influence of gradients with large amplitudes on priority order, so as to obtain better texture structure. Search region partition can reduce the time for searching similar image blocks. Experiments show that the algorithm can inpaint the object with missing unique structure by reference object, and can better inpaint the curved contour and irregular texture of various object images. Compared with the existing traditional and deep learning image inpainting methods, this method achieves better results in inpainting objects with complex structures and textures.