基于动态规划的视频自动裁剪

Automatic video cropping based on dynamic programming

传统视频裁剪方法大多只考虑内容保持,而忽略了由裁剪窗口频繁移动造成的时间不连续问题.为此,提出了一种基于动态规划的视频自动裁剪算法,并通过平滑因子和面积约束将视差控制在舒适区间.算法主要分为目标检测、轨迹定义、轨迹找寻三个阶段.首先将输入的视频序列进行语义处理并输出,运用YOLOX模型检测输出视频中的重要内容;然后定义一个对数线性模型融合每帧图像的特征、颜色直方图和灰度图,以得到一组帧系数,并对裁剪窗口进行面积和距离上的约束;最后运用动态规划中最短关键路径方法为视频帧找到一系列裁剪窗口,在相邻帧之间设置过渡因子来保证裁剪轨迹的平滑,同时定义了一个视觉惩罚函数平衡内容丢失与裁剪窗口偏移的问题.实验结果表明:所提出的算法在裁剪率、扭曲度和稳定性方面都优于以往的方法.

Traditional video cropping methods mostly only consider content retention,and ignore the problem of time discontinuity caused by frequent movement of the crop window.An automatic video cropping algorithm based on dynamic programming is proposed in this paper,and the parallax is controlled in the comfort area by smoothing factor and area constraint.The algorithm is mainly divided into three stages:object detection,trajectory definition,and trajectory search.Firstly,the input video sequence is semantically processed and output.The important content of output video is detected by YOLOX.Then,a set of frame coefficients are got by defining a log-linear model,which fuse the features,color histogram and grayscale diagram of each frame,the area and distance of the crop window are constrained at the same time.Finally,a series of crop windows for the video frame are found by the shortest critical path method in dynamic programming,while the transition factor is set between adjacent frames to ensure the smoothness of the crop trajectory,and a visual penalty function is also defined to balance content loss with crop window offset.Experimental results show that the proposed algorithm is better than the previous method in terms of crop ratios,distortion and stability.