采用图融合的多模态半协同训练视频分类算法

Multi-modality semi co-training algorithm with graph fusion in video classification

针对视频数据库的分类,提出了一种将视频数据的多模态特性与基于图的半监督算法相结合的新方法.首先,算法假设数据分散在不同模态的流形上,通过使用给定的每种模态构成一个图.然后,将流形正则化的结果融合在不同的图上,并利用每个图的流形正则化输出上基于排名的判决准则来对排名度量中最有效的样本进行标记.算法迭代执行,直至全部样本被标记.为了进一步提高算法的性能,对前面提出的算法进行改进,在每次迭代中采用更多的标记视频,同时又避免了误差传播.在CCTV-3数据库上运行的仿真实验结果表明,提出的算法无论是在平均精度,还是在精度-召回率性能指标方面都优于现有的单模态流形正则化算法。

Aiming at the classification of video database,a new method to combine the multimodal characteristics of video data with the graph based semi-supervised algorithm is proposed in this paper. Firstly,the assumption of the algorithm is that data are scattered over a manifold for different modalities,and a graph is formed by using each of the given modalities. Then,the results of manifold regularization are fused on different graphs,and a ranking-based decision criterion over manifold regularization output of each graph is utilized to label the most efficient samples in ranking metric. The algorithm iterates until all samples are labeled. In order to further improve the performance of the algorithm,the proposed algorithm at the head is modified,and more labeled videos are used in each iteration while avoiding error propagation. Simulation experiments running on CCTV-3 database show that the proposed algorithm is superior to the existing single-modality manifold regularization algorithm in terms of both average precision performance and precision-recall performance.