We propose a novel end-to-end deep learning framework, the Joint Matting Network(JMNet), to automatically generate alpha mattes for human images.We utilize the intrinsic structures of the human body as seen in images ...We propose a novel end-to-end deep learning framework, the Joint Matting Network(JMNet), to automatically generate alpha mattes for human images.We utilize the intrinsic structures of the human body as seen in images by introducing a pose estimation module,which can provide both global structural guidance and a local attention focus for the matting task. Our network model includes a pose network, a trimap network, a matting network, and a shared encoder to extract features for the above three networks. We also append a trimap refinement module and utilize gradient loss to provide a sharper alpha matte. Extensive experiments have shown that our method outperforms state-of-theart human matting techniques;the shared encoder leads to better performance and lower memory costs.Our model can process real images downloaded from the Internet for use in composition applications.展开更多
In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new bac...In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new background. In this paper, we use a color line model to improve the Gaussian mixture model in the background cut method to obtain a binary foreground segmentation result that is less sensitive to brightness differences. Based on the high quality binary segmentation results, we can automatically create a reliable trimap for alpha matting to refine the segmentation boundary. To make the composition result more realistic, an automatic foreground color adjustment step is added to make the foreground look consistent with the new background. Compared to previous approaches, our method can produce higher quality binary segmentation results, and to the best of our knowledge, this is the first time such an automatic and integrated background substitution system has been proposed which can run in real time, which makes it practical for everyday applications.展开更多
基金supported by National Natural Science Foundation of China(Grant Nos.61561146393 and61521002)supported by a Victoria Early-Career Research Excellence Award。
文摘We propose a novel end-to-end deep learning framework, the Joint Matting Network(JMNet), to automatically generate alpha mattes for human images.We utilize the intrinsic structures of the human body as seen in images by introducing a pose estimation module,which can provide both global structural guidance and a local attention focus for the matting task. Our network model includes a pose network, a trimap network, a matting network, and a shared encoder to extract features for the above three networks. We also append a trimap refinement module and utilize gradient loss to provide a sharper alpha matte. Extensive experiments have shown that our method outperforms state-of-theart human matting techniques;the shared encoder leads to better performance and lower memory costs.Our model can process real images downloaded from the Internet for use in composition applications.
基金supported by the National HighTech R&D Program of China (Project No. 2012AA011903)the National Natural Science Foundation of China (Project No. 61373069)+1 种基金the Research Grant of Beijing Higher Institution Engineering Research CenterTsinghua–Tencent Joint Laboratory for Internet Innovation Technology
文摘In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new background. In this paper, we use a color line model to improve the Gaussian mixture model in the background cut method to obtain a binary foreground segmentation result that is less sensitive to brightness differences. Based on the high quality binary segmentation results, we can automatically create a reliable trimap for alpha matting to refine the segmentation boundary. To make the composition result more realistic, an automatic foreground color adjustment step is added to make the foreground look consistent with the new background. Compared to previous approaches, our method can produce higher quality binary segmentation results, and to the best of our knowledge, this is the first time such an automatic and integrated background substitution system has been proposed which can run in real time, which makes it practical for everyday applications.
