MusicFace: Music-driven expressive singing face synthesis

It remains an interesting and challenging problem to synthesize a vivid and realistic singing face driven by music. In this paper, we present a method for this task with natural motions for the lips, facial expression, head pose, and eyes. Due to the coupling of mixed information for the human voice and backing music in common music audio signals, we design a decouple-and-fuse strategy to tackle the challenge. We first decompose the input music audio into a human voice stream and a backing music stream. Due to the implicit and complicated correlation between the two-stream input signals and the dynamics of the facial expressions, head motions, and eye states, we model their relationship with an attention scheme, where the effects of the two streams are fused seamlessly. Furthermore, to improve the expressivenes of the generated results, we decompose head movement generation in terms of speed and direction, and decompose eye state generation into short-term blinking and long-term eye closing, modeling them separately. We have also built a novel dataset, SingingFace, to support training and evaluation of models for this task, including future work on this topic. Extensive experiments and a user study show that our proposed method is capable of synthesizing vivid singing faces, qualitatively and quantitatively better than the prior state-of-the-art.

Effects of mesh loop modes on performance of unstructured finite volume GPU simulations

In unstructured finite volume method,loop on different mesh components such as cells,faces,nodes,etc is used widely for the traversal of data.Mesh loop results in direct or indirect data access that affects data locality significantly.By loop on mesh,many threads accessing the same data lead to data dependence.Both data locality and data dependence play an important part in the performance of GPU simulations.For optimizing a GPU-accelerated unstructured finite volume Computational Fluid Dynamics(CFD)program,the performance of hot spots under different loops on cells,faces,and nodes is evaluated on Nvidia Tesla V100 and K80.Numerical tests under different mesh scales show that the effects of mesh loop modes are different on data locality and data dependence.Specifically,face loop makes the best data locality,so long as access to face data exists in kernels.Cell loop brings the smallest overheads due to non-coalescing data access,when both cell and node data are used in computing without face data.Cell loop owns the best performance in the condition that only indirect access of cell data exists in kernels.Atomic operations reduced the performance of kernels largely in K80,which is not obvious on V100.With the suitable mesh loop mode in all kernels,the overall performance of GPU simulations can be increased by 15%-20%.Finally,the program on a single GPU V100 can achieve maximum 21.7 and average 14.1 speed up compared with 28 MPI tasks on two Intel CPUs Xeon Gold 6132.

蒙古族唐卡与藏族唐卡艺术特征比较研究

提到唐卡艺术,人们更多的会联想到雪域高原藏族画师精妙绝伦的技艺以及对于信仰的无限虔诚。观者在精美的唐卡面前,无不感受到藏传佛教的博大精深。从元至清600多年的时间里,藏传佛教在蒙古族地区广泛而深入的传播,使藏传佛教对世界的观念逐渐移植于蒙古社会,其带来的深刻影响,至今依然可见。

Feature-based RGB-D camera pose optimization for real-time 3D reconstruction

In this paper we present a novel featurebased RGB-D camera pose optimization algorithm for real-time 3D reconstruction systems. During camera pose estimation, current methods in online systems suffer from fast-scanned RGB-D data, or generate inaccurate relative transformations between consecutive frames. Our approach improves current methods by utilizing matched features across all frames and is robust for RGB-D data with large shifts in consecutive frames. We directly estimate camera pose for each frame by efficiently solving a quadratic minimization problem to maximize the consistency of3 D points in global space across frames corresponding to matched feature points. We have implemented our method within two state-of-the-art online 3D reconstruction platforms. Experimental results testify that our method is efficient and reliable in estimating camera poses for RGB-D data with large shifts.

Stable haptic interaction based on adaptive hierarchical shape matching

In this paper, we present a framework allowing users to interact with geometrically complex3 D deformable objects using(multiple) haptic devices based on an extended shape matching approach. There are two major challenges for haptic-enabled interaction using the shape matching method. The first is how to obtain a rapid deformation propagation when a large number of shape matching clusters exist. The second is how to robustly handle the collision response when the haptic interaction point hits the particlesampled deformable volume. Our framework extends existing multi-resolution shape matching methods,providing an improved energy convergence rate. This is achieved by using adaptive integration strategies to avoid insignificant shape matching iterations during the simulation. Furthermore, we present a new mechanism called stable constraint particle coupling which ensures consistent deformable behavior during haptic interaction. As demonstrated in our experimental results, the proposed method provides natural and smooth haptic rendering as well as efficient yet stable deformable simulation of complex models in real time.