基于多输入三维卷积神经网络的脑电解码模型

EEG decoding model based on three dimensional convolutionalneural network with multi input

脑机接口是实现人脑与外界交互的关键技术,脑电解码是其中的核心环节,并随着深度学习发展在预测大脑意图的精度方面得到显著提升。然而个体内脑电信号差异降低了脑电解码模型的鲁棒性。为此,提出一种多输入三维卷积神经网络(MT-3D-CNN)的脑电解码模型,通过融合脑电数据的空间排布与时间排布,形成两种三维矩阵数据作为卷积网络的多输入,采用三维卷积核沿时-空方向进行特征提取与解码。10名被试参加了间隔6 h和12 h的脑机接口实验,并采用MT-3D-CNN进行跨时间的解码预测。MT-3D-CNN的单次解码准确率在长时间下分别维持在78.15%和72.56%,高于单输入的3D-CNN(62.89%和52.35%),表明MT-3D-CNN通过对脑电数据的时间和空间的多种排布方式形成的多输入能够充分利用其三维卷积核学习与提取特征的能力,并且针对个体内脑电差异具有更强的解码性能,有助于推动脑机接口系统的普及使用。

The brain⁃computer interface(BCI)is a key technology for achieving interaction between the human brain and the outside world,and EEG decoding is the central part.With the development of deep learning,the accuracy of predicting brain intentions has been significantly improved by EEG decoding.However,intra⁃individual differences in EEG signals reduce the robustness of the EEG decoding model.Therefore,a multi input three dimensional convolutional neural network(MT⁃3D⁃CNN)EEG decoding model is proposed,which combines the spatial and temporal layouts of EEG data to form two types of 3D matrix data as the multi input of the convolutional network,and the 3D convolutional kernel is used for feature extraction and decoding along the spatiotemporal direction.Ten participants participated in the BCI experiments with an interval of 6 h and 12 h,and MT⁃3D⁃CNN is used for cross⁃time decoding prediction.The single decoding accuracy of MT⁃3D⁃CNN was maintained at 78.15%and 72.56%,respectively,over a long period of time,which were higher than those of 3D⁃CNN(62.89%and 52.35%)with single input.This indicates that the MT⁃3D⁃CNN can make full use of its ability of 3D convolution kernel learning and feature extraction by the multi⁃input formed with various arrangements of EEG data in time and space.In addition,it has stronger decoding performance for intra⁃individual EEG differences,which is helpful for promoting the popularization and use of BCI system.