Depth-aided inpainting for disocclusion restoration of multi-view images using depth-image-based rendering

A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi＇s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of flling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio （PSNR） statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.

Detection of the Single Image from DIBR Based on 3D Warping Trace and Edge Matching

Recently, the popularity of 3D content is on the rise because of its immersive experience to view- ers. While demands for 3D contents and 3D technologies increase, only a few copyright protection methods for 3D contents have been proposed. The simplest infringement is the illegal distribution of the single 2D image from 3D content. The leaked image is still valuable as 2D content and the leakage can be occurred in DIBR system. To detect the leaked image, we focus on the hole-filled region which is caused by the hole-filling procedure mandatory in DIBR system. To estimate the hole-filled regions, two different procedures are conducted to extract edges and to estimate 3D warping traces, respectively. After that, the hole-filled regions are estimated and the left-right-eye image discrimination (LR discrimination) is also conducted. Experimental results demonstrate the effectiveness of the proposed method using quantitative measures.

No-reference synthetic image quality assessment with convolutional neural network and local image saliency

Depth-image-based rendering(DIBR) is widely used in 3 DTV, free-viewpoint video, and interactive 3 D graphics applications. Typically, synthetic images generated by DIBR-based systems incorporate various distortions, particularly geometric distortions induced by object dis-occlusion. Ensuring the quality of synthetic images is critical to maintaining adequate system service. However, traditional 2 D image quality metrics are ineffective for evaluating synthetic images as they are not sensitive to geometric distortion. In this paper, we propose a novel no-reference image quality assessment method for synthetic images based on convolutional neural networks, introducing local image saliency as prediction weights. Due to the lack of existing training data, we construct a new DIBR synthetic image dataset as part of our contribution. Experiments were conducted on both the public benchmark IRCCyN/IVC DIBR image dataset and our own dataset. Results demonstrate that our proposed metric outperforms traditional 2 D image quality metrics and state-of-the-art DIBR-related metrics.