摘要
传统的基于RGB和骨骼特征的行为识别算法,普遍存在两种特征互补性不足及视频关键时序性不强等问题。为解决这一问题,提出一种自适应融合RGB和骨骼特征的行为识别算法。首先,面向RGB图像和骨骼图像,联合双向长短时记忆(LSTM)网络和自注意力机制提取两者的时空特征;然后,构建自适应权重计算网络(AWCN),并以两者的空间特征为输入计算出自适应权重;最后,利用自适应权重得到上述时空特征的融合特征,实现了最终的动作分类。通过在Penn Action、JHMDB和NTU RGB-D人体行为数据集上与现有的方法进行比较,实验结果表明,本文算法有效地提高了行为识别精度。
In this paper,we proposed an action recognition algorithm based on the adaptive fusion of RGB and skeleton features to efficiently improve the accuracy of action recognition.However,the conventional action recognition algorithms based on RGB and skeleton features generally suffer from the inability to effectively utilize the complementarity of the two features and thus fail to focus on important frames in the video.Considering this,we first used the bidirectional long short-term memory network(Bi-LSTM)combined with a self-attention mechanism to extract spatial-temporal features of RGB and skeleton images.Next,we constructed an adaptive weight computing network(AWCN)and computed these adaptive weights based on the spatial features of two types of images.Finally,the extracted spatial-temporal features were fused by the adaptive weights to implement action recognition.Using Penn Action,JHMDB,and NTU RGB-D dataset,the experimental results show that our proposed algorithm effectively improves the accuracy of action recognition compared with existing methods.
作者
郭伏正
孔军
蒋敏
Guo Fuzheng;Kong Jun;Jiang Min(International Joint Laboratory for Pattern Recognition and Computational Intelligence,Jiangnan University,Wuxi,Jiangsu 214122,China)
出处
《激光与光电子学进展》
CSCD
北大核心
2020年第20期302-311,共10页
Laser & Optoelectronics Progress
基金
国家自然科学基金(61362030,61201429)
中国博士后科学基金(2015M581720,2016M600360)
科技援疆专项计划(2017E0279)
江苏博士后科学基金(1601216C)
关键词
机器视觉
行为识别
姿态估计
自适应权重计算网络
长短时记忆网络
自注意力
machine vision
action recognition
pose estimation
adaptive weight computing network
long short-term memory network
self-attention