基于口型运动速度的视频序列对齐方法

VIDEO SEQUENCE ALIGNMENT METHOD BASED ON VELOCITY OF MOUTH MOVEMENT

基于视频人脸替换系统中降低视频抖动和满足语音跟口型匹配的要求,提出一种基于视频图像中口型运动速度特征的视频口型对齐方法。视频中将ASM和光流法相结合,获取稳定的口型运动速度特征;利用平行线约束条件对动态时间规整(DTW)算法的路径搜索范围进行约束以快速求得最优的匹配路径;得到对齐后的视频并给出相似性评价。实验结果表明,该方法可以使得两段视频中说话的口型保持一致。口型对齐不仅满足人物正脸不动的情况,而且能满足人物有任意小角度(<45°)的头部运动的情况。最后对模仿视频给出比较客观的相似性评价,评价结果显示该方法切实可行。

Based on the requirement of reducing video jitter and video matching between mouth and voice in video face replacement system, a video mouth alignment method based on the characteristics of mouth speed in video image is proposed. In the video, the ASM and the optical flow method were combined to obtain the stable velocity characteristics of the mouth, and then the path search range of the Dynamic Time Warping （DTW） algorithm was constrained by the parallel line constraint condition to obtain the optimal matching path quickly. Finally, the aligned video was obtained and the similarity evaluation was given. Experimental results show that the proposed method can make the mouth speak two videos in consistent alignment. This method not only satisfies the situation of human face immobility, but also satisfies the human head imitation video, and the motion at any degree less than 45 degrees. Finally objective similarity evaluation is given to the evaluation results show that the method is feasible.