基于偶极子成像和3D卷积神经网络的源域运动想象解码方法

Motor imagery decoding in the source domain based on dipole imaging and three-dimensional convolutional neural network

目的为充分保留和利用运动想象(motor imagery,MI)时偶极子的时空信息,本文提出一种新的偶极子成像(dipoles imaging,DI)结合3维卷积神经网络(3D convolutional neural network,3DCNN)的源域MI解码方法(DI-3DCNN)。方法首先,基于脑源成像(electroencephalography source imaging,ESI)技术计算运动想象脑电信号的偶极子源估计;接着,获取每类MI任务的平均偶极子源估计,基于数据驱动自动选择每类任务中偶极子激活水平较高且最大区分于其他任务的时刻作为中心采样点,再对中心采样点进行前后延伸并按任务顺序组合,形成感兴趣时间(time of interest,TOI);其次,选择覆盖高激活偶极子的Desikan-Killiany(DK)神经分区,并对局部保持投影方法(local preserving projection,LPP)增加DK分区约束,获得一种改进的有监督LPP(LPP DK);进而,基于LPP DK分别将所选择左、右半脑分区内的偶极子坐标从3维(three dimensional,3D)降成2维,获得具有神经生理先验信息的偶极子2D坐标,再结合TOI内各采样点处偶极子的幅值信息进行成像,并进行插值、下采样操作,得到偶极子的2D幅值图;随后,将TOI内偶极子的2D幅值图按时间顺序堆叠,获得左、右半脑的3D偶极子特征图,并将其作为网络的输入数据;最后,根据输入数据的特点,设计一种双分支3D卷积神经网络(dual-branched 3DCNN,DB3DCNN)实现MI解码。结果基于BCI competition IV 2a数据集进行实验研究,取得了86.50%的平均解码准确率。结论基于DI所得3D偶极子特征图能够较好地保留偶极子的最佳激活时间、程度及生理空间信息,且与DB3DCNN性能匹配。

Objective To fully preserve and utilize the spatiotemporal information of dipoles during motor imagery(MI),this paper proposes a novel dipole imaging(DI)method combined with a 3D convolutional neural network(3DCNN)for source-domain MI decoding(DI-3DCNN).Methods Firstly,based on the electroencephalography source imaging(ESI)technique,dipole source estimation of MI-EEG is computed.Secondly,the average dipole source estimation for each MI task is obtained.By automatically selecting the time points with high dipole activation levels and maximum discrimination from other tasks,the moments are chosen as central sampling points.These central points are extended forwards and backwards in time and combined in task order to form time of interest(TOI).Subsequently,the Desikan-Killiany(DK)neuroanatomical partitions covering highly activated dipoles is chosen,and a local preserving projection method(LPP)with DK atlas constraint(LPP DK)is employed.Then,dipole coordinates within the selected left and right brain partitions are separately reduced from 3D to 2D using LPP DK,obtaining dipole 2D coordinates with neurophysiological prior information.These 2D coordinates are combined with dipole amplitude information at each sampling point within TOI for imaging.Interpolation and downsampling are performed to obtain 2D amplitude maps of dipoles.After that,2D amplitude maps of dipoles within TOI are stacked in chronological order to obtain 3D dipole feature maps of left and right brains,which serve as the input data for the network.At last,a dual-branch 3D convolutional neural network(DB3DCNN)is designed to decode MI based on the characteristics of input data.Results Experimental studies conducted on the BCI competition IV 2a dataset demonstrate an average decoding accuracy of 86.50%.Conclusions The 3D dipole feature maps obtained through DI effectively preserve the optimal activation time,intensity,and physiological spatial information of dipoles,and are suitable for DB3DCNN.