基于加权DTW的多特征分级融合动态手势识别

Multi-feature hierarchical fusion for dynamic gesture recognition based on DTW

AR设备以第一视角获取的动态手势往往存在手掌角度偏转问题,各关节点位移轨迹相较正视角度发生变化,现有的手掌关节点位移特征无法有效识别,并且受限于AR设备性能,部分神经网络模型无法取得良好表现。针对该应用场景,提出一种使用多特征分级融合的手势识别方法。该方法构造位移、长度、角度三个特征对手势进行描述,并进行向量编码与归一化以消除抖动干扰。根据关节点与标准手势的相似度距离,仿照sigmoid函数分配关节点权重,以加权的动态时间规整(DTW)距离进行KNN匹配,并根据最佳的KNN置信度与特征优先级筛选出最可信的特征识别结果。实验结果表明:该方法能有效识别9种存在角度偏转的动态手势;相较于传统的位移特征方法,该方法的平均准确率提高了4%,能有效应对手掌偏转情况下的动态手势识别问题。

Hand angle variation in dynamic motions captured from a first-person perspective is a common problem for augmented reality(AR)systems.Existing palm key point displacement features are ineffective for recognition because different key points′displacement trajectories differ from the frontal perspective.AR device limits also impose performance limitations for some neural network models.A gesture recognition method using multi feature hierarchical fusion is proposed for this application scenario.In this method,the displacement,length,and angle features are constructed to describe gestures,and vector encoding and normalization are conformed to eliminate jitter interference.Based on the similarity distance between the joint points and the standard gesture,the joint weights are assigned by means of the sigmoid function,and KNN matching is performed by means of the weighted dynamic time warming(DTW)distance.The most reliable feature recognition result is selected based on the best KNN confidence and feature priority.The experimental results show that this method can effectively recognize 9 dynamic gestures with angle deviation;in comparison with the traditional displacement feature methods,this method has an average accuracy improvement of 4%and can effectively address the dynamic gesture recognition problem under palm deflection.