基于浅三维稠密网的多模态手势识别算法

Multimodal Gesture Recognition Algorithm Based on Shallow 3D Dense Networks

手势识别旨在理解人体的动态手势,是人机交互领域极其重要的交互方式之一。该方法通过将二维稠密网扩展为三维稠密网,并加入Inception结构,提出了一种基于浅三维稠密网的多模态手势识别方法,将其命名为Spatial Temporal 3D(ST3D) dense network。所提出的方法在手势识别公开数据集大规模离散手势数据集(IsoGD)上进行了评估,并取得了目前最好效果。实验证明,所提方法能够有效地学习到视频样本中手势的短期、中期以及长期时空特征。

Gesture recognition aims at understanding dynamic gestures of the human body,and is one of the most important ways of human-computer interaction.A multimodal gesture recognition method based on a shallow 3D dense network is proposed by extending the two-dimensional dense network into a 3D dense network and adding the Inception structure, which is named Spatial Temporal 3D(ST3D)dense network.The proposed method is evaluated on the Charlearn LAP large- scale Isolated Gesture Dataset(IsoGD)and achieves the best results.Experimental results show that the proposed method can effectively learn short,mid and long term spatiotemporal features of gestures in video samples.