基于骨架的自适应尺度图卷积动作识别

Scale Adaptive Graph Convolutional Network for Skeleton-Based Action Recognition

基于骨架的动作识别任务中,一般将骨骼序列表示为预定义的时空拓扑图.然而,由于样本的多样性,固定尺度的拓扑图往往不是最优结构,针对样本特性构建自适应尺度的骨骼拓扑图能够更好地捕捉时空特征;另外,不同尺度的骨骼图能够表达不同粒度的人体结构特征,因此对多个不同尺度的拓扑图进行特征提取与融合是有必要的.针对这些问题,提出了一种自适应尺度的图卷积动作识别模型.该模型包含自适应尺度图卷积模块和多尺度融合模块两部分.自适应尺度图卷积模块基于先验与空间注意力机制,构建关键点的活跃度判决器,将活跃点细化为小尺度结构、非活跃点聚合为大尺度结构,在加速节点间特征传递的同时最小化特征损耗;多尺度融合模块基于通道注意力机制,动态融合不同尺度的特征,进一步提升网络的灵活性;最后,综合关键点、骨骼、运动信息实现多路特征聚合的动作判别,丰富模型的特征表达.结果表明:该算法在NTU-RGBD数据集的CS和CV子集上分别取得了89.7%和96.1%的分类准确率,显著提高了动作识别的准确性.

In skeleton-based action recognition,graph convolutional network(GCN),which models the human skeleton sequences as spatiotemporal graphs,have achieved excellent performance.However,in existing GCNbased methods,the topology of the graph is set manually,and it is fixed over all layers and input samples.This approach may not be optimal for diverse samples.Constructing an scale adaptive graph based on sample characteristics can better capture spatiotemporal features.Moreover,most methods do not explicitly exploit the multiple scales of body components,which carry crucial information for action recognition.In this paper,we proposed a scale adaptive graph convolutional network comprising a dynamic scale graph convolution module and a multiscale fusion module.Specifically,we first used an a priori and attention mechanism to construct an activity judger,which can divide each keypoint into two parts based on whether it is active;thereafter,a scale adaptive graph was automatically learned.This module accelerated the feature transfer between nodes while minimizing the feature loss.Furthermore,we proposed a multiscale fusion module based on the channel attention mechanism to extract features at different scales and fuse features across scales.Moreover,we used a four-stream framework to model the first-order,second-order,and motion information of a skeleton,which shows notable improvement in terms of recognition accuracy.Extensive experiments on the NTU-RGBD dataset demonstrate the effectiveness of our method.Results show that the algorithm achieves 89.7%and 96.1%classification accuracy on the cross-subject(CS)and cross-view(CV)subsets of the NTU-RGBD dataset,respectively,thus significantly improving the accuracy of action recognition.