In the digital age,non-touch communication technologies are reshaping human-device interactions and raising security concerns.A major challenge in current technology is the misinterpretation of gestures by sensors and...In the digital age,non-touch communication technologies are reshaping human-device interactions and raising security concerns.A major challenge in current technology is the misinterpretation of gestures by sensors and cameras,often caused by environmental factors.This issue has spurred the need for advanced data processing methods to achieve more accurate gesture recognition and predictions.Our study presents a novel virtual keyboard allowing character input via distinct hand gestures,focusing on two key aspects:hand gesture recognition and character input mechanisms.We developed a novel model with LSTM and fully connected layers for enhanced sequential data processing and hand gesture recognition.We also integrated CNN,max-pooling,and dropout layers for improved spatial feature extraction.This model architecture processes both temporal and spatial aspects of hand gestures,using LSTM to extract complex patterns from frame sequences for a comprehensive understanding of input data.Our unique dataset,essential for training the model,includes 1,662 landmarks from dynamic hand gestures,33 postures,and 468 face landmarks,all captured in real-time using advanced pose estimation.The model demonstrated high accuracy,achieving 98.52%in hand gesture recognition and over 97%in character input across different scenarios.Its excellent performance in real-time testing underlines its practicality and effectiveness,marking a significant advancement in enhancing human-device interactions in the digital age.展开更多
现有的脑-机接口系统大都只基于单模式的脑电特征,系统能实现的功能非常有限,从而制约了脑-机接口系统的应用。采用基于多种模式脑电信号(electroencephalogram,EEG)的脑-机接口技术来实现虚拟键鼠系统,使得被试可以利用自身的脑电信号...现有的脑-机接口系统大都只基于单模式的脑电特征,系统能实现的功能非常有限,从而制约了脑-机接口系统的应用。采用基于多种模式脑电信号(electroencephalogram,EEG)的脑-机接口技术来实现虚拟键鼠系统,使得被试可以利用自身的脑电信号控制鼠标和键盘的操作。研究了脑-机接口中常用的3种脑电信号,分别是P300波、alpha波以及稳态视觉诱发电位(steady state visual evoked potential,SSVEP),通过设计实验成功的诱发出了被试相应的特征脑电信号。利用SSVEP的脑电特征设计6频率LED闪烁刺激的虚拟鼠标系统,实现控制鼠标光标移动、单击左键和单击右键的任务;利用P300波的脑电特征设计6×6的字符矩阵虚拟键盘系统,实现字符输入的任务;利用被试自主闭眼增强alpha波的脑电特征,实现鼠标和键盘应用切换的任务。研究了适宜这3种脑电特征的最佳测量电极组合及模式识别算法,使得对3种脑电信号的识别正确率均达到了85%以上。测试结果显示,文中设计的基于多模式EEG的脑-机接口虚拟键鼠系统能有效地实现鼠标控制以及键盘输入的任务。展开更多
文摘In the digital age,non-touch communication technologies are reshaping human-device interactions and raising security concerns.A major challenge in current technology is the misinterpretation of gestures by sensors and cameras,often caused by environmental factors.This issue has spurred the need for advanced data processing methods to achieve more accurate gesture recognition and predictions.Our study presents a novel virtual keyboard allowing character input via distinct hand gestures,focusing on two key aspects:hand gesture recognition and character input mechanisms.We developed a novel model with LSTM and fully connected layers for enhanced sequential data processing and hand gesture recognition.We also integrated CNN,max-pooling,and dropout layers for improved spatial feature extraction.This model architecture processes both temporal and spatial aspects of hand gestures,using LSTM to extract complex patterns from frame sequences for a comprehensive understanding of input data.Our unique dataset,essential for training the model,includes 1,662 landmarks from dynamic hand gestures,33 postures,and 468 face landmarks,all captured in real-time using advanced pose estimation.The model demonstrated high accuracy,achieving 98.52%in hand gesture recognition and over 97%in character input across different scenarios.Its excellent performance in real-time testing underlines its practicality and effectiveness,marking a significant advancement in enhancing human-device interactions in the digital age.
文摘现有的脑-机接口系统大都只基于单模式的脑电特征,系统能实现的功能非常有限,从而制约了脑-机接口系统的应用。采用基于多种模式脑电信号(electroencephalogram,EEG)的脑-机接口技术来实现虚拟键鼠系统,使得被试可以利用自身的脑电信号控制鼠标和键盘的操作。研究了脑-机接口中常用的3种脑电信号,分别是P300波、alpha波以及稳态视觉诱发电位(steady state visual evoked potential,SSVEP),通过设计实验成功的诱发出了被试相应的特征脑电信号。利用SSVEP的脑电特征设计6频率LED闪烁刺激的虚拟鼠标系统,实现控制鼠标光标移动、单击左键和单击右键的任务;利用P300波的脑电特征设计6×6的字符矩阵虚拟键盘系统,实现字符输入的任务;利用被试自主闭眼增强alpha波的脑电特征,实现鼠标和键盘应用切换的任务。研究了适宜这3种脑电特征的最佳测量电极组合及模式识别算法,使得对3种脑电信号的识别正确率均达到了85%以上。测试结果显示,文中设计的基于多模式EEG的脑-机接口虚拟键鼠系统能有效地实现鼠标控制以及键盘输入的任务。
