深度学习的双人交互行为识别与预测算法研究

Human interaction recognition and prediction algorithm based on deep learning

基于卷积神经网络的双人交互行为识别算法存在提取的深度特征无法有效表征交互行为序列特性的问题,本文将长短期记忆网络与卷积神经网络模型相结合,提出了一种基于深度学习的双人交互行为识别与预测一体化方法。该方法在训练过程中,完成对卷积神经网络和长短期记忆网络模型的参数训练。在识别与预测过程中,将不同时间比例长度的未知动作类别的视频图像分别送入已经训练好的卷积神经网络模型提取深度特征,再将卷积神经网络提取的深度特征送入长短期记忆网络模型完成对双人交互行为的识别与预测。在国际公开的UT-interaction双人交互行为数据库进行测试的结果表明,该方法在保证计算量适当的同时对交互行为的正确识别率达到了92.31%,并且也可完成对未知动作的初步预测。

A drawback of the human interaction recognition algorithm based on a convolutional neural network(CNN)is that the extracted depth features cannot effectively represent the characteristics of interaction sequences.Instead,this paper proposes a human interaction recognition and prediction algorithm based on deep learning,by combining the Long Short-Term Memory(LSTM)network with the CNN model.In the process,video images of unknown action categories of different time lengths are sent to a trained CNN model to extract the depth features.The depth features are then sent to a trained LSTM model to complete the recognition and prediction of the interaction behavior.When tested on the UTinteraction human interaction behavior dataset,the algorithm demonstrates a 92.31%correct human interaction recognition rate and can complete the preliminary prediction of unknown actions.