基于视听交互刺激的认知机理与脑机接口范式研究进展

Research on cognitive mechanism and brain-computer interface application in visual-auditory crossmodal stimuli

与人类视听觉感知密切相关的图像、语音和文本(语言)信息在社会经济发展与国家安全保障等领域中扮演着重要角色,脑机接口(BCI)是无需外围神经肌肉便可操控外部设备、使"思想"直接变成"行动"的创新技术,基于视、听觉刺激的BCI系统具有极其广阔应用前景。目前认知心理学研究普遍认为视听交互刺激模式可产生比视或听单模态刺激更强的事件相关电位信号,因而可使识别大脑思维模式的正确率更高和响应速度更快,然而BCI研究至今尚难有充分验证其优势的成果。分别从认知心理机制和BCI实验范式两方面综述了视听交互刺激的信息整合机制、视听刺激之间的协同与竞争关系、匹配度对刺激效果的影响及相关BCI实验范式的研究现状;深入分析了视听交互刺激范式未在现有BCI系统中体现其优越性的原因;最后从BCI实验范式选择、系统硬件性能改进、脑电信号处理等方面提出了改进思路并展望了基于视听交互刺激BCI范式研究未来动向。

The information of image, voice, and text （language）, which is closely related to the human perception of visual and auditory, plays an important role in socioeconomic development and country security assurance.Brain-computer interface （BCI） is an innovative technology that directly controls peripheral equipment and makes thought into action with non-muscular methods.Visual and auditory stimuli based BCI system has extremely broad application prospects.It is commonly believed that brain has stronger event-related potential, higher reaction accuracy and faster reaction speed on visual-auditory crossmodal stimuli （VACS） than single modal stimuli （visual or auditory） in current cognitive psychology researches, while BCI researches have difficulty on the verification of the VACS superiority.In this paper, the VACS researches in cognitive psychology and BCI paradigms were reviewed respectively, which included the information integration mechanism of VACS, the facilitation and competition relationship between visual stimuli and auditory stimuli in VACS system, the influence of congruence and the research status of VACS based BCI paradigms.Also, the reasons for not showing the superiority of VACS in BCI system were discussed.Then, the countermeasure of defusing the difficulties were presented from aspects of the selection of BCI paradigms, the performance improvement of system hardware, the signal processing of electroencephalography（EEG） and so on.In the end, the future trends of VACS based BCI research were presented.