基于增强现实脑机接口和计算机视觉的机械臂控制系统

Robotic arm control system based on augmented reality brain-computer interface and computer vision

脑机接口(BCI)在替代丧失的上肢功能方面具有巨大的潜力。因此,基于脑机接口控制的机械臂的开发备受关注。但是,少有研究尝试利用基于头皮脑电的无创脑机接口来实现对机械臂的高级控制。本文设计了一种结合增强现实(AR)脑机接口和计算机视觉的高级控制体系结构,以控制机械臂执行拾取和放置任务。基于稳态视觉诱发电位(SSVEP)的脑机接口范式被用来实现脑机接口系统。微软的HoloLens用于构建增强现实环境,并用作诱发稳态视觉诱发电位的视觉刺激器。所构建的增强现实脑机接口用于选择需要由机械臂操作的物体。计算机视觉负责提供物体的位置、颜色和形状信息。根据增强现实和计算机视觉的输出,机械臂可以自动拾取物体并将其放置到特定位置。11名健康受试者的在线结果表明,该系统的平均分类准确率为91.41%。这些结果验证了结合增强现实、脑机接口和计算机视觉技术来控制机械臂的可行性,并有望为创新机械臂控制方法提供新的思路。

Brain-computer interface(BCI)has great potential to replace lost upper limb function.Thus,there has been great interest in the development of BCI-controlled robotic arm.However,few studies have attempted to use noninvasive electroencephalography(EEG)-based BCI to achieve high-level control of a robotic arm.In this paper,a highlevel control architecture combining augmented reality(AR)BCI and computer vision was designed to control a robotic arm for performing a pick and place task.A steady-state visual evoked potential(SSVEP)-based BCI paradigm was adopted to realize the BCI system.Microsoft’s HoloLens was used to build an AR environment and served as the visual stimulator for eliciting SSVEPs.The proposed AR-BCI was used to select the objects that need to be operated by the robotic arm.The computer vision was responsible for providing the location,color and shape information of the objects.According to the outputs of the AR-BCI and computer vision,the robotic arm could autonomously pick the object and place it to specific location.Online results of 11 healthy subjects showed that the average classification accuracy of the proposed system was 91.41%.These results verified the feasibility of combing AR,BCI and computer vision to control a robotic arm,and are expected to provide new ideas for innovative robotic arm control approaches.