基于时空信息可信融合的视频监控暴力检测算法

Violence detection algorithm based on trustworthy fusion of spatial and temporal information

在暴力行为检测任务中,监控视频的静态红-绿-蓝(RGB)信息和光流特征对任务的支持度受场景变化而产生动态变化。因此,提出一种基于时空信息可信融合的视频监控暴力检测算法。首先,利用每一帧静态RGB信息和相邻帧的动态光流信息提取特征,并通过空间特征网络和时序特征网络获得静态RGB帧和多帧光流的分类证据;然后,在得到时间网络和空间网络的分类证据后,应用狄利克雷分布估计每个模态分类结果的不确定性;最后,考虑两个模态特征质量的动态变化特性,并基于Dempster-Shafer证据理论对两个模态预测结果进行融合。实验结果表明,与现有时空改进的三维轨迹(TS+IDT)算法相比,所提算法在violent crowd数据集和hockfights数据集上暴力检测准确率分别提升4.79和0.23个百分点,验证了所提方法的合理性和有效性。此外,该算法不仅可以提供暴力行为识别结果,还可以在部署阶段提供决策的可信程度,从而使得算法在实际应用中更加健壮、可靠,降低后续分析任务出现错误预测的风险。

In the violence detection task,the support of the static Red-Green-Blue(RGB)information and optical flow features in the surveillance video for the task are dynamically varied by scene changes.Therefore,a violence detection algorithm based on trustworthy fusion of spatial and temporal information was proposed.Firstly,the static RGB information of each frame and the dynamic optical flow information of adjacent frames were utilized to extract features,and the classification evidence of static RGB frames and multiple frames of optical flow was obtained through spatial feature network and temporal feature network.Then,after obtaining the classification evidence from the temporal network and the spatial network,the uncertainty of each modal classification result was estimated using the Dirichlet distribution.Finally,the dynamic variation characteristics of the quality of the two modal features were considered,and the fusion of the two modal prediction results was performed based on the Dempster-Shafer evidence theory.Experimental results show that compared to the existing Temporal Spatial Improved three-Dimensional Trajectories(TS+IDT)algorithm,the proposed algorithm can improve the accuracy of violence detection by 4.79 and 0.23 percentage points on the violent crowd dataset and hockfights dataset,respectively,which verifies the rationality and effectiveness of this fusion strategy.In addition,the proposed algorithm can not only provide violent behavior recognition results but also provide the trustworthiness of decision-making during deployment,making the algorithm more robust and reliable in practical applications,and reducing the risk of erroneous predictions in subsequent analysis tasks.