基于深度学习和姿态驱动下特征集成的视频人体动作识别方法

Deep Learning Based Pose-driven Feature Learning Method for Video Human Action Recognition

基于姿态的方法仅将一系列人体骨骼信息提取出来作为输入,但仍然会存在姿态估计不准确的问题。因此该方法尝试将RGB图像和姿态信息结合起来,然而训练一个既有表冠又有姿态的模型十分具有挑战性,通常会严重偏向表冠信息,导致泛化能力不足。该方法提出基于姿态驱动下的特征集成方式,通过观察姿态特征来动态地组合表冠和姿态信息,使得姿态流可以根据给定的姿态信息是否可靠来决定在特征集成过程中使用多少权重以及哪些表冠信息。实验结果表明,文章提出的动态融合算法在上下文内和上下文外的动作视频数据集上实现了优异且鲁棒的性能。

The pose based method only extracted a series of human bone information as input, but there still existed the problem of inaccurate pose estimation. Therefore, this method attempted to combine RGB image and pose information. However, training a model with both appearance and pose was very challenging. It usually biased the appearance information seriously, resulting in insufficient generalization ability. Thus this method proposed a pose-driven feature integration method, which dynamically combined the appearance and pose information by observing the pose features, so that the pose flow could determine how much weight and which appearance information to use in the feature integration process according to whether the given pose information was reliable or not. Experimental results showed that the proposed dynamic fusion algorithm achieved excellent and robust performance on action video datasets in and out of context.