一种用于动作识别的双分支网络

A dual-branch network for action recognition

动作识别是计算机视觉领域的一项重要任务,主要有基于RGB视频和人体骨架两种数据模态的领域,主流方法分别是3D卷积神经网络和图卷积神经网络。针对视频和人体骨架两种数据模态的不同特点,设计了双分支网络分别对两种数据模态进行建模。对于人体骨架数据,基于自注意力机制设计了图卷积神经网络,该算法能在基于骨架的动作识别任务中达到先进的性能。对于视频数据,采用3D卷积网络进行特征提取。同时,利用深监督方法对两种数据模态的中间特征进行监督,提高两种数据特征的耦合度,进一步提高网络效率。这种算法的网络结构简单,在NTU-RGBD60(CS)数据集上仅用3.37×10^(7)的参数量可达到95.6%的精度。

Action recognition has always been an important task in the field of computer vision.There are mainly two tasks based on RGB video and human skeleton.The mainstream methods are3D convolutional neural network and graph convolutional neural network.For the data modality of human skeleton,this work designs a graph convolutional neural network based on the self-attention mechanism.The algorithm can achieve advanced performance on skeleton-based action recognition tasks.In addition,a method is proposed to use deep supervision methods to supervise the intermediate features of video and human skeleton,which improves the coupling of the two data features and further improves network efficiency.The network structure of this algorithm is simple,and only 3.37×10^(7) parameters are used to achieve an accuracy of 95.6%on the NTU-RGBD60(CS)dataset.