Human Mouth-State Recognition Based on Image Warping and Sparse Representation Combined with Homotopy

It is often necessary to recognize human mouth-states for detecting the number of audio sources and improving the speech recognition capability of an intelligent robot auditory system. A human mouth-state recognition method based on image warping and sparse representation( SR) combined with homotopy is proposed.Using properly warped training mouth-state images as atoms of the overcomplete dictionary overcomes the impact of the diversity of the mouths' scales,shapes and positions so that further improvement of the robustness can be achieved and the requirement for a large number of training samples can be relieved. The homotopy method is employed to compute the expansion coefficients effectively,i. e.,for sparse coding. The orthogonal matching pursuit( OMP) is also tested and compared with the homototy method. Experimental results and comparisons with the state-of-the-art methods have proved the effectiveness of the proposed approach.

AN IMPROVED RADIAL BASIS FUNCTION BASED METHOD FOR IMAGE WARPING

A new image warping method is proposed in this letter, which can warp a given image by some manual defined features. Based on the radial basis interpolation function algorithm, the proposed method can transform the original optimized problem into nonsingular linear problem by adding one-order term and affine differentiable condition. This linear system can get the steady unique solution by choosing suitable kernel function. Furthermore, the proposed method demonstrates how to set up the radial basis function in the target image so as to achieve supports to adopt the backward re-sampling technology accordingly which could gain the very slippery warping image. Theexperimental result shows that the proposed method can implement smooth and gradual image warping with multi-anchor points' accurate interpolation.

Polygonal finite element-based content-aware image warping

Mesh-based image warping techniques typically represent image deformation using linear functions on triangular meshes or bilinear functions on rectangular meshes.This enables simple and efficient implementation,but in turn,restricts the representation capability of the deformation,often leading to unsatisfactory warping results.We present a novel,flexible polygonal finite element(poly-FEM)method for content-aware image warping.Image deformation is represented by high-order poly-FEMs on a content-aware polygonal mesh with a cell distribution adapted to saliency information in the source image.This allows highly adaptive meshes and smoother warping with fewer degrees of freedom,thus significantly extending the flexibility and capability of the warping representation.Benefiting from the continuous formulation of image deformation,our polyFEM warping method is able to compute the optimal image deformation by minimizing existing or even newly designed warping energies consisting of penalty terms for specific transformations.We demonstrate the versatility of the proposed poly-FEM warping method in representing different deformations and its superiority by comparing it to other existing state-ofthe-art methods.

Depth-aided inpainting for disocclusion restoration of multi-view images using depth-image-based rendering

A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi＇s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of flling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio （PSNR） statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.

Image-based clothes changing system

Current image-editing tools do not match up to the demands of personalized image manipulation,one application of which is changing clothes in usercaptured images. Previous work can change single color clothes using parametric human warping methods.In this paper, we propose an image-based clothes changing system, exploiting body factor extraction and content-aware image warping. Image segmentation and mask generation are first applied to the user input.Afterwards, we determine joint positions via a neural network. Then, body shape matching is performed and the shape of the model is warped to the user's shape. Finally, head swapping is performed to produce realistic virtual results. We also provide a supervision and labeling tool for refinement and further assistance when creating a dataset.

A retargeting method for stereoscopic 3D video

We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we model distortion energies to prevent important video contents from deforming. Then, to maintain depth mapping stability, we model disparity variation energies to constraint the disparity range both in spatial and temporal domains. The last component of our method is a non-uniform, pixel-wise warp to the target resolution based on these energy models. Using this method, we can process the original stereoscopic video to generate new, high-perceptual-quality versions at different display resolutions. For evaluation, we conduct a user study; we also discuss the performance of our method.