去除光流中冗余信息的动作预测方法

Action Prediction Method for Removing Redundant Information in Optical Flow

近年来使用光流作为输入特征的基于深度学习的动作预测方法逐渐成为主流,但是光流由于环境因素等影响,极易引入无关的冗余信息,从而降低动作预测的精度,而现有方法并没有考虑到光流中的冗余信息。可以从三方面去除光流图中的冗余信息:消除视频中静止部分光流所带来的冗余信息;选取合理的运动区域以消除无关背景因素引入的光流冗余信息;评估相机的运动去除相机运动产生的光流冗余信息。针对去除冗余信息的光流图,提出了一种基于深度学习的动作预测框架,通过使用空间卷积和时间卷积来减少模型的参数,使用基于时间权重的投票机制实现了对动作的预测。在UT-Interaction set1和set2数据集上的实验表明了该方法的有效性。

In recent years,the deep learning-based action prediction method which uses optical flow as input features has gradually become the mainstream.Due to the environmental factors,the optical flow can easily introduce irrele-vant redundant information to reduce the accuracy of action prediction.However,the existing methods do not take the redundant information in optical flow into consideration.In this paper,the redundant information in the optical flow is removed from three aspects:the redundant information is brought by the static part in the video;the reasonable motion area is selected to eliminate the redundant information introduced by irrelevant background factors;the motion of the camera is evaluated to remove the redundant information generated by the camera motion.Aiming at the optical flow after removing redundant information,an action prediction framework based on deep learning is proposed.The parameters of the model are reduced by using space convolution and time convolution.The voting mechanism based on time weight is used to predict the action.Experiments on UT-Interaction set1 and set2 demonstrate the effectiveness of this method.