基于条件变分自编码器的井下配电室巡检行为检测

Inspection behavior detection of underground power distribution room based on conditional variational auto-encoder

现有井下配电室巡检行为检测方法的研究重点在于视频动作的分类,但在实际应用中,对于端到端的视频检测任务,不仅需要识别巡检动作的类别,还需要预测巡检动作发生的开始时间和结束时间。且现有基于监督学习的研究方法训练网络时需要标注视频的每一帧,存在数据集制作繁琐、训练时间较长等问题,基于弱监督学习的研究方法也依赖视频分类模型,导致在没有视频帧级别标注的条件下很难区分动作帧和背景帧。针对以上问题,提出了一种基于条件变分自编码器的弱监督井下配电室巡检行为检测模型。该模型主要由判别注意力模型和生成注意力模型2个部分组成,将井下配电室巡检行为检测分为巡检动作的分类和定位2种任务。首先利用特征提取模型分别提取出井下配电室监控视频的RGB特征与光流特征;然后将获取到的RGB特征与光流特征输入注意力模块中进行训练,得到特征帧的注意力,通过判别注意力模型得到软分类,根据注意力的得分情况判断出动作帧和背景帧;最后对判别注意力模型的输出进行后处理,输出视频中包含巡检动作的时间区间、动作标签及置信度,即完成了巡检动作的分类及定位。为了提高定位任务的精度,加入基于条件变分自编码器的生成注意力模型,利用条件变分自编码器与解码器的生成对抗对视频的潜在特征进行学习。利用井下配电室监控视频,将巡检行为分为站立检测、下蹲检测、来回走动、站立记录和坐下记录,制作了巡检行为数据集进行实验,结果表明:基于条件变分自编码器的巡检行为检测模型可同时完成巡检行为分类和定位任务,在THUMOS14数据集上mAP@0.5达到17.0%,在自制的巡检行为数据集上mAP@0.5达到24.0%,满足井下配电室巡检行为检测要求。

The research focus of the existing inspection behavior detection methods in underground power distribution room is on the classification of video action.However,in practical application,for end-to-end video detection tasks,it is necessary not only to identify the category of inspection actions,but also to predict the start time and end time of inspection actions.Moreover,the existing research method based on supervised learning needs to label each frame of the video when training the network,so there are problems of complicated data set production and long training time.And the research method based on weakly supervised learning also relies on a video classification model,so it is difficult to distinguish the action frame and the background frame without video frame-level labeling.In order to solve the above problems,this paper proposes an inspection behavior detection model of weakly supervised underground power distribution room based on conditional variational auto-encoder.The model consists of two parts,namely discriminative attention model and generative attention model.The inspection behavior detection form of the underground power distribution room is divided into two tasks,namely classification and positioning of inspection action.Firstly,the RGB characteristics and light flow characteristics of the monitoring video of the underground power distribution room are extracted by using the characteristic extraction model.Secondly,the obtained RGB characteristics and the light flow characteristics are input into an attention module for training to obtain the attention of the characteristic frame.The soft classification is obtained by judging an attention model,and the action frame and background frame are distinguished according to the attention score.Finally,the output of the discriminative attention model is post-processed,and the output video contains the time interval,action label and confidence of the inspection action,that is,the classification and positioning of the inspection action are completed.In order to improve the precision of the positioning task,the generative attention model based on conditional variational auto-encoder is added,and the potential characteristics of the video are learned by using the generative confrontation between conditional variational auto-encoder and decoder.The inspection behavior is divided into standing detection,squatting detection,walking back and forth,standing record and sitting record by using the monitoring video of the underground power distribution room,and the inspection behavior data set is made for experiment.The result shows that the inspection behavior detection model based on the conditional variational auto-encoder can complete the inspection behavior classification and positioning tasks simultaneously.And the mAP@0.5 reaches 17.0%on the THUMOS14 data set,and the mAP@0.5 reaches 24.0%on the self-made inspection behavior data set,which meets the requirements for inspection behavior detection in underground power distribution rooms.