基于卷积神经网络的多模态视频场景分割优化算法

Multi-modal video scene segmentation optimization algorithm based on convolutional neural network

针对基于内容的视频检索中场景分割效率有待提高的问题,提出了一种基于卷积神经网络提取特征的多模态视频场景分割优化算法。首先利用改进的VGG19网络从视频镜头中提取多种模态的底层特征和语义特征,再将这些特征组成向量,然后通过三重损失学习与镜头相似度计算等方法,使场景分割问题转换为对镜头边界的二分类问题,最后建立评分机制优化所得结果,获取分割好的视频场景及对应的场景边界,完成场景分割任务。实验结果表明,该算法能对视频场景进行有效分割,整体查全率与查准率分别能达到85.77%、87.01%。

Aiming at the problem that the efficiency of scene segmentation in content-based video retrieval needs to be improved,this paper proposed a multi-modal video scene segmentation optimization algorithm based on feature extraction of convolutional neural network.Firstly,the algorithm applied the improved VGG19 network to extract underlying features and semantic features from each video shots.Secondly,this paper combined these features into vectors and applied the method of triplet loss learning and shot similarity calculation,so that converted the scene segmentation task to a binary classification problem for shot boundary.Finally,this paper established a scoring mechanism to optimize the results and obtained the segmented video scene and corresponding scene boundary.Experimental results show that the algorithm can be effective in video scene segmentation,and the overall recall and precision indicators can reach 85.77% and 87.01%.