分层残差结构的时空图网络多目标在线康复动作识别

MULTI-TARGET ONLINE REHABILITATION ACTION RECOGNITION BASED ON SPATIO-TEMPORAL GRAPH NETWORK WITH HIERARCHICAL RESIDUAL STRUCTURE

时空图卷积网络(ST-GCN)可以自动学习骨架数据的空间和时间特征,不受外界复杂环境的干扰。针对原有模型存在的骨架信息特征提取不充分、局部信息建模不强等问题,提出一种分层残差结构的骨架识别模型(Res2-STGCN)。构造分层残差结构的时空图卷积模块结合原模块组成新的网络模型。通过改变模块的尺度来进一步扩大感受野。调整学习率间隔等参数解决过拟合问题。将Res2-STGCN与检测、姿态估计与跟踪算法结合实现多目标康复动作识别。在NTU-RGB+D和自建数据集上设计实验,对比基准算法ST-GCN,改进后最优模型的识别准确率在两种不同的数据划分标准下分别提升了5.61百分点和6.03百分点,在自建数据集上的平均识别准确率为99.5%,对复杂动作的识别具有较强的鲁棒性。

Spatio-temporal graph convolutional network(ST-GCN)can automatically learn the spatial and temporal characteristics of skeleton data without interference from the external complex environment.In order to solve the problems of inadequate skeleton information feature extraction and weak local information modeling in the original model,a skeleton recognition model with layered residual structure(Res2-STGCN)is proposed.The spatio-temporal convolution module with layered residual structure was combined with the original module to form a new network model.The receptive field was further expanded by changing the size of the module.Parameters such as learning rate interval were adjusted to solve the overfitting problem.Res2-STGCN was combined with detection,pose estimation and tracking algorithm to realize multi-target rehabilitation action recognition.Experiments were designed on NTU-RGB+D and self-built data sets.Compared with the benchmark algorithm ST-GCN,the recognition accuracy of the improved optimal model is improved by 5.61 and 6.03 percentage points respectively under the two different data partitioning standards.The average recognition accuracy of the optimized model on self-built data sets is 99.5%,showing strong robustness for the recognition of complex actions.