摘要
本文是对SKIG RGB-D多模态的孤立手势视频进行手势识别研究.首先将RGB和Depth两种单模态视频提取成图片的形式保存,然后采样成长度为32帧的手势序列分别输入到本文提出的稠密连接的3DCNN组件学习短期的时空域特征,然后将提取的时空域特征输入到卷积GRU网络进行长期的时空域特征学习,最终对单模态训练好的网络进行多模态融合,提升网络识别准确率.本文在SKIG数据集上取得了99.07%的识别准确率,达到了极高的准确率,证明了本文提出的网络模型的有效性.
In this study,the gesture recognition based on SKIG RGB-D multimodal isolated gesture video is studied.The RGB and depth videos are extracted into the form of images.Then the sampled 32frames from images are input to the densely connected 3DCNN component to learn short-term spatiotemporal features,after that the features input to the convolutional GRU to learn long-term spatiotemporal features.Finally,the trained networks for single modal are used to multimodal fusion to improve the recognition accuracy.99.07%recognition accuracy is obtained on the SKIG dataset, which achieves high accuracy and proves the validity of the network model proposed in this study.
作者
马正文
蔡坚勇
刘磊
欧阳乐峰
李楠
MA Zheng-Wen;CAI Jian-Yong;LIU Lei;OUYANG Le-Feng;LI Nan(College of Photonic and Electronic Engineering,Fujian Normal University,Fuzhou 350007,China;Key Laboratory of Optoelectronie Science and Technology for Medicine of Ministry of Education,Fujian Normal University,Fuzhou 350007,China;Fujian Provincial Key Laboratory of Photonics Technology,Fujian Normal University,Fuzhou 350007,China;Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application,Fujian Normal University,Fuzhou 350007,China;Intelligent Optoelectronic Systems Research Center,Fujian Normal University,Fuzhou 350007,China)
出处
《计算机系统应用》
2018年第12期234-239,共6页
Computer Systems & Applications
基金
福建省自然科学基金(2017J01744)~~
