This paper proposes a technique for synthesizing a pixel-based photo-realistic talking face animation using two-step synthesis with HMMs and DNNs. We introduce facial expression parameters as an intermediate represent...This paper proposes a technique for synthesizing a pixel-based photo-realistic talking face animation using two-step synthesis with HMMs and DNNs. We introduce facial expression parameters as an intermediate representation that has a good correspondence with both of the input contexts and the output pixel data of face images. The sequences of the facial expression parameters are modeled using context-dependent HMMs with static and dynamic features. The mapping from the expression parameters to the target pixel images are trained using DNNs. We examine the required amount of the training data for HMMs and DNNs and compare the performance of the proposed technique with the conventional PCA-based technique through objective and subjective evaluation experiments.展开更多
Hyperbolic metamaterials(HMMs) are novel artificial materials that excite the surface plasmon resonance(SPR) because of their unique hyperbolic dispersion properties. Herein, to the best of our knowledge, we propose t...Hyperbolic metamaterials(HMMs) are novel artificial materials that excite the surface plasmon resonance(SPR) because of their unique hyperbolic dispersion properties. Herein, to the best of our knowledge, we propose the first HMM-based fiber SPR(HMM-SPR) sensor for vector magnetic detection. By selecting the composite materials and structural parameters of the HMM dispersion management, HMM-SPR sensors can achieve a high refractive index sensitivity of 14.43 μm/RIU. Vector magnetic field detection was performed with the HMM-SPR sensor encapsulated with a magnetic fluid. Compared with other ferrofluidbased magnetic field fiber sensors, the proposed sensor shows pronounced advantages in intensity and direction sensitivity of 1.307 nm/Oe and 7.116 nm/°, respectively. The sensor design approach presented in this paper provides an excellent demonstration of HMM-SPR sensors in various applications.展开更多
文摘This paper proposes a technique for synthesizing a pixel-based photo-realistic talking face animation using two-step synthesis with HMMs and DNNs. We introduce facial expression parameters as an intermediate representation that has a good correspondence with both of the input contexts and the output pixel data of face images. The sequences of the facial expression parameters are modeled using context-dependent HMMs with static and dynamic features. The mapping from the expression parameters to the target pixel images are trained using DNNs. We examine the required amount of the training data for HMMs and DNNs and compare the performance of the proposed technique with the conventional PCA-based technique through objective and subjective evaluation experiments.
基金supported by the National Natural Science Foundation of China (Grant Nos. 62175094, 61904067, 61805108, and 62075088)Basic and Applied Basic Research Foundation of Guangdong Province (Grant Nos. 2022A1515011671, 2022A1515010272, and 2020A1515011498)+2 种基金Basic and Applied Basic Research Foundation of Guangzhou (Grant No. 202102020758)Science and Technology R&D Project of Shenzhen (Grant Nos. JSGG20201102163800003, and JSGG20210713091806021)Fundamental Research Funds for the Central Universities (Grant Nos. 21621405, and 21620328)。
文摘Hyperbolic metamaterials(HMMs) are novel artificial materials that excite the surface plasmon resonance(SPR) because of their unique hyperbolic dispersion properties. Herein, to the best of our knowledge, we propose the first HMM-based fiber SPR(HMM-SPR) sensor for vector magnetic detection. By selecting the composite materials and structural parameters of the HMM dispersion management, HMM-SPR sensors can achieve a high refractive index sensitivity of 14.43 μm/RIU. Vector magnetic field detection was performed with the HMM-SPR sensor encapsulated with a magnetic fluid. Compared with other ferrofluidbased magnetic field fiber sensors, the proposed sensor shows pronounced advantages in intensity and direction sensitivity of 1.307 nm/Oe and 7.116 nm/°, respectively. The sensor design approach presented in this paper provides an excellent demonstration of HMM-SPR sensors in various applications.
