一种视频时空特征提取算法及其应用研究

Research and application of a video spatiotemporal feature extraction algorithm

目前基于内容特征的视频检索系统,大多采用提取视频关键帧的颜色、纹理、形状等底层特征,来进行视频相识度匹配,这些底层特征是基于全局统计或者人工设计的特征提取方式,存在泛化抽象能力不强、易受光照和噪声的影响等问题,同时,由于未考虑视频帧时序上的关联性,导致视频检索精度偏低。为此,基于深度学习框架,提出了一种视频时空特征提取算法。该算法以关键帧附近16帧图像作为学习源,采用三维卷积神经网络,融合帧的内容特性和时序变化特性,获取4096维特征向量作为新的视视时空特征描述子。在标准动作视频数据集UCF-101上进行实验,结果表明该特征能显著提高视频检索精度,在查全率为90%的情况下,平均查准率不低于84%,检索效果优于传统视频检索方法。

At present,most of the content-based video retrieval systems use the underlying features such as color,texture and shape of the key frames of the video for video similarity matching.These underlying features are based on global statistics or artificially designed methods of feature extraction,which have weak generalization and abstraction ability,and are easily affected by light and noise,etc.In addition,the relevance of video frame timing is not considered,so the video retrieval accuracy is not ideal.Therefore,this paper proposes an algorithm of video spatial-temporal feature extraction based on the deep learning framework.It takes 16 frames near the key frame as the learning source and adopts the 3D convolutional neural network to fuse the spatial and temporal characteristics of the frame,to obtain the 4096D feature vector as the new spatial-temporal feature descriptor of the video.Experiments on the standard action video dataset UCF-101 show that this feature can significantly improve the video retrieval accuracy.The average accuracy of the video retrieval is not less than 84%when the recall rate is 90%,which is better than the traditional video retrieval method.