Existing lip synchronization(lip-sync)methods generate accurately synchronized mouths and faces in a generated video.However,they still confront the problem of artifacts in regions of non-interest(RONI),e.g.,backgroun...Existing lip synchronization(lip-sync)methods generate accurately synchronized mouths and faces in a generated video.However,they still confront the problem of artifacts in regions of non-interest(RONI),e.g.,background and other parts of a face,which decreases the overall visual quality.To solve these problems,we innovatively introduce diverse image inpainting to lip-sync generation.We propose Modulated Inpainting Lip-sync GAN(MILG),an audio-constraint inpainting network to predict synchronous mouths.MILG utilizes prior knowledge of RONI and audio sequences to predict lip shape instead of image generation,which can keep the RONI consistent.Specifically,we integrate modulated spatially probabilistic diversity normalization(MSPD Norm)in our inpainting network,which helps the network generate fine-grained diverse mouth movements guided by the continuous audio features.Furthermore,to lower the training overhead,we modify the contrastive loss in lipsync to support small-batch-size and few-sample training.Extensive experiments demonstrate that our approach outperforms the existing state-of-the-art of image quality and authenticity while keeping lip-sync.展开更多
This paper investigates human mobility patterns in an urban taxi transportation system. This work focuses on predicting human mobility from discovering patterns of in the number of passenger pick-ups quantity （PUQ） ...This paper investigates human mobility patterns in an urban taxi transportation system. This work focuses on predicting human mobility from discovering patterns of in the number of passenger pick-ups quantity （PUQ） from urban hotspots. This paper proposes an improved ARIMA based prediction method to forecast the spatial-temporal variation of passengers in a hotspot. Evaluation with a large-scale real- world data set of 4 000 taxis＇ GPS traces over one year shows a prediction error of only 5.8%. We also explore the applica- tion of the pl^di^fioti approach to help drivers find their next passetlgerS, The sinatllation results using historical real-world data demonstrate that, with our guidance, drivers can reduce the time taken and distance travelled, to find their next pas- senger＋ by 37.1% and 6.4% respectively,展开更多
基金partially funded by the National Key Research and Development Program of China(Grant No.2020AAA0140004).
文摘Existing lip synchronization(lip-sync)methods generate accurately synchronized mouths and faces in a generated video.However,they still confront the problem of artifacts in regions of non-interest(RONI),e.g.,background and other parts of a face,which decreases the overall visual quality.To solve these problems,we innovatively introduce diverse image inpainting to lip-sync generation.We propose Modulated Inpainting Lip-sync GAN(MILG),an audio-constraint inpainting network to predict synchronous mouths.MILG utilizes prior knowledge of RONI and audio sequences to predict lip shape instead of image generation,which can keep the RONI consistent.Specifically,we integrate modulated spatially probabilistic diversity normalization(MSPD Norm)in our inpainting network,which helps the network generate fine-grained diverse mouth movements guided by the continuous audio features.Furthermore,to lower the training overhead,we modify the contrastive loss in lipsync to support small-batch-size and few-sample training.Extensive experiments demonstrate that our approach outperforms the existing state-of-the-art of image quality and authenticity while keeping lip-sync.
文摘This paper investigates human mobility patterns in an urban taxi transportation system. This work focuses on predicting human mobility from discovering patterns of in the number of passenger pick-ups quantity （PUQ） from urban hotspots. This paper proposes an improved ARIMA based prediction method to forecast the spatial-temporal variation of passengers in a hotspot. Evaluation with a large-scale real- world data set of 4 000 taxis＇ GPS traces over one year shows a prediction error of only 5.8%. We also explore the applica- tion of the pl^di^fioti approach to help drivers find their next passetlgerS, The sinatllation results using historical real-world data demonstrate that, with our guidance, drivers can reduce the time taken and distance travelled, to find their next pas- senger＋ by 37.1% and 6.4% respectively,
