基于肌音信号图像化和卷积神经网络的手语识别研究

Sign Language Recognition Based on Image-interpreted Mechanomyography and Convolution Neural Network

时间序列信号被广泛应用于各种模式识别的场合,针对大量目标的时间序列信号模式识别率低的问题,借助多种图像化手段,将时间序列信号转换为图像,采用图像分类算法实现模式识别。实验中采集了前臂上30种手语对应的肌音信号(Mechanomyography,MMG),将其转换为不同风格的图像,设计卷积神经网络(Convolution Neural Network,CNN)框架,对图像化的肌音信号训练集建立模式识别的分类模型,并且应用迁移学习(transfer learning)算法对模型进行多次优化,建立的分类模型识别率达98.7%,高于普通机器学习算法的识别率。实验结果证明了图像化处理时间序列信号可以有效提高多分类肌音信号模式识别的识别率,该研究可以为其他时间序列信号的模式识别研究提供参考。

Time series signals are widely used in various pattern recognition applications.In order to solve the problem of low pattern recognition rate of time series signals for a large number of targets,this article uses a variety of transform methods to convert time series signals into images,and performes pattern recognition using image classification algorithms.In the experiment,the mechanomyography(MMG)corresponding to 30 sign languages on the forearm are collected and converted into different image styles,and a convolution neural network(CNN)framework is designed to establish pattern recognition classification models for the images.The models are optimized twice with the application of transfer learning algorithm,and the recognition rate of the best classification model reaches 98.7%,which is much higher than the recognition rate of traditional machine learning algorithms.The experimental results imply that the image processing of time series signals can effectively improve the recognition rate of multi-target pattern recognition of MMG.This paper can provide references for pattern recognition of other time series signal.