基于图卷积及振幅约束的弱监督时序行为检测

Weakly supervised temporal action detection based on graph convolution and amplitude constraint

针对弱监督时序行为检测缺乏精确的行为起始和结束时间标注,导致时间维度信息匮乏等问题,提出基于挖掘视频片段间联系的方法,捕获一定程度上的时间维度信息,提高行为检测能力,本研究采用图卷积建模弱监督时序行为检测任务,用图节点表达视频片段的特征,图的边表达视频片段间的联系,使得行为检测网络不仅考虑了各视频片段的特征,还考虑了视频片段之间的联系.此外,利用振幅约束和背景约束进一步建模视频片段特征.在公开数据集上的实验结果表明本文方法相对于已有方法具有一定的性能优势.

Aiming at the problems such as the lack of accurate start and end time marking in weakly supervised temporal action detection,which leads to the lack of time dimension information,a method based on mining the connection between video clips was proposed to capture a certain degree of time dimension information and improve the ability of action detection.In this paper,graph convolution was used to model the weakly supervised temporal action detection task.The graph nodes were used to express the features of video segments,and the graph edges were used to express the connections between video segments,so that the action detection network not only considers the features of each video segment,but also considers the connections between video segments.In addition,amplitude constraints and background constraints were used to further model video segment features.Experimental results on public datasets show that the proposed method has certain performance advantages over the existing methods.