基于时空张量融合的人体骨架行为自适应识别方法

Adaptive recognition method of human skeleton action with spatial-temporal tensor fusion

针对人体行为的空间复杂性和时间差异性问题,提出了一种基于时空张量融合的人体骨架行为自适应识别方法。首先充分利用人体行为骨架序列的帧内空间关系和帧间时间关系,构建相邻帧时空特征张量;其次通过计算相邻帧时空特征张量的差异性获取关键相邻帧时空特征张量并组成行为时空特征张量;之后利用行为时空特征张量的空间特征差异和多尺度时间卷积构建行为时空特征张量自适应注意力机制,完成行为时空特征融合;最后,使用深度随机配置网络根据行为时空特征融合张量识别人体行为。使用NTU RGB-D数据集进行实验仿真,识别准确率达到84.57%,并且设计相应的系统进行实际应用验证,结果表明本文所提方法是一种适合应对人体行为空间复杂性和时间差异性问题的人体行为识别方法。

To address space complexity and time difference of human action,an adaptive recognition method of human skeleton action with spatial-temporal tensor fusion is proposed.Firstly,the spatial-temporal feature tensors of adjacent frames are established by making full use of the intra-frame spatial relationship and the inter-frame temporal relationship of the human action skeleton sequences.Secondly,the difference of spatial-temporal feature tensors of adjacent frames is calculated to achieve the spatial-temporal feature tensors of key adjacent frames and compose the behavior spatial-temporal feature tensors.Then,the spatial feature difference of the action spatial-temporal feature tensors and the multi-scale temporal convolution is used to construct the adaptive attention mechanism of the behavior spatio-temporal feature tensors to complete the fusion of action spatial-temporal features.Finally,a deep stochastic configuration network is used to recognize human action according to the spatial-temporal feature fusion tensor of action.The NTU RGB-D data set was used for experimental simulation,and the recognition accuracy reached 84.57%.The corresponding system is designed for practical application verification.The results show that the proposed method is suitable for dealing with the space complexity and time difference of human action recognition.