Virtual Keyboard:A Real-Time Hand Gesture Recognition-Based Character Input System Using LSTM and Mediapipe Holistic

In the digital age,non-touch communication technologies are reshaping human-device interactions and raising security concerns.A major challenge in current technology is the misinterpretation of gestures by sensors and cameras,often caused by environmental factors.This issue has spurred the need for advanced data processing methods to achieve more accurate gesture recognition and predictions.Our study presents a novel virtual keyboard allowing character input via distinct hand gestures,focusing on two key aspects:hand gesture recognition and character input mechanisms.We developed a novel model with LSTM and fully connected layers for enhanced sequential data processing and hand gesture recognition.We also integrated CNN,max-pooling,and dropout layers for improved spatial feature extraction.This model architecture processes both temporal and spatial aspects of hand gestures,using LSTM to extract complex patterns from frame sequences for a comprehensive understanding of input data.Our unique dataset,essential for training the model,includes 1,662 landmarks from dynamic hand gestures,33 postures,and 468 face landmarks,all captured in real-time using advanced pose estimation.The model demonstrated high accuracy,achieving 98.52%in hand gesture recognition and over 97%in character input across different scenarios.Its excellent performance in real-time testing underlines its practicality and effectiveness,marking a significant advancement in enhancing human-device interactions in the digital age.

Gesture Recognition Based on Time-of-Flight Sensor and Residual Neural Network

With the advancement of technology and the increase in user demands, gesture recognition played a pivotal role in the field of human-computer interaction. Among various sensing devices, Time-of-Flight (ToF) sensors were widely applied due to their low cost. This paper explored the implementation of a human hand posture recognition system using ToF sensors and residual neural networks. Firstly, this paper reviewed the typical applications of human hand recognition. Secondly, this paper designed a hand gesture recognition system using a ToF sensor VL53L5. Subsequently, data preprocessing was conducted, followed by training the constructed residual neural network. Then, the recognition results were analyzed, indicating that gesture recognition based on the residual neural network achieved an accuracy of 98.5% in a 5-class classification scenario. Finally, the paper discussed existing issues and future research directions.

Vision Based Hand Gesture Recognition Using 3D Shape Context

Hand gesture recognition is a popular topic in computer vision and makes human-computer interaction more flexible and convenient.The representation of hand gestures is critical for recognition.In this paper,we propose a new method to measure the similarity between hand gestures and exploit it for hand gesture recognition.The depth maps of hand gestures captured via the Kinect sensors are used in our method,where the 3D hand shapes can be segmented from the cluttered backgrounds.To extract the pattern of salient 3D shape features,we propose a new descriptor-3D Shape Context,for 3D hand gesture representation.The 3D Shape Context information of each 3D point is obtained in multiple scales because both local shape context and global shape distribution are necessary for recognition.The description of all the 3D points constructs the hand gesture representation,and hand gesture recognition is explored via dynamic time warping algorithm.Extensive experiments are conducted on multiple benchmark datasets.The experimental results verify that the proposed method is robust to noise,articulated variations,and rigid transformations.Our method outperforms state-of-the-art methods in the comparisons of accuracy and efficiency.

A Hand Gesture Recognition Method Based on SVM

A hand gesture recognition method is presented for human-computer interaction, which is based on fingertip localization. First, hand gesture is segmented from the background based on skin color characteristics. Second, feature vectors are selected with equal intervals on the boundary of the gesture, and then gestures＇ length normalization is accomplished. Third, the fingertip positions are determined by the feature vectors＇ parameters, and angles of feature vectors are normalized. Finally the gestures are classified by support vector machine. The experimental results demonstrate that the proposed method can recognize 9 gestures with an accuracy of 94.1%.

Appearance Based Dynamic Hand Gesture Recognition Using 3D Separable Convolutional Neural Network

Appearance-based dynamic Hand Gesture Recognition(HGR)remains a prominent area of research in Human-Computer Interaction(HCI).Numerous environmental and computational constraints limit its real-time deployment.In addition,the performance of a model decreases as the subject’s distance from the camera increases.This study proposes a 3D separable Convolutional Neural Network(CNN),considering the model’s computa-tional complexity and recognition accuracy.The 20BN-Jester dataset was used to train the model for six gesture classes.After achieving the best offline recognition accuracy of 94.39%,the model was deployed in real-time while considering the subject’s attention,the instant of performing a gesture,and the subject’s distance from the camera.Despite being discussed in numerous research articles,the distance factor remains unresolved in real-time deployment,which leads to degraded recognition results.In the proposed approach,the distance calculation substantially improves the classification performance by reducing the impact of the subject’s distance from the camera.Additionally,the capability of feature extraction,degree of relevance,and statistical significance of the proposed model against other state-of-the-art models were validated using t-distributed Stochastic Neighbor Embedding(t-SNE),Mathew’s Correlation Coefficient(MCC),and the McNemar test,respectively.We observed that the proposed model exhibits state-of-the-art outcomes and a comparatively high significance level.

Gesture Recognition Summarization

Gesture recognition is an important research in the field of human-computer interaction. Hand Gestures are strong variable and flexible, so the gesture recognition has always been an important challenge for the researchers. In this paper, we first outlined the development of gestures recognition, and different classification of gestures based on different purposes. Then we respectively introduced common methods used in the process of gesture segmentation, feature extraction and recognition. Finally, the gesture recognition was summarized and the studying prospects were given.

基于Kinect的手势识别算法研究及应用

手势识别技术是人机交互技术的重要研究内容。为了提高基于Kinect的手势识别性能,提出基于深度人手定位和hog特征的静态手势识别算法及基于改进HMMs的动态手势识别算法。静态手势识别算法首先通过Kinect的深度信息完成人手定位,而后在定位区域内提取基于梯度方向直方图的形状特征并利用级联Adaboost训练的手势模型,实现对静态手势的准确识别,在公开手势数据库中测试的实验结果表明提出的静态手势识别算法具有较高的识别率。动态手势识别算法首先通过Kinect获取手心轨迹并提取轨迹切线角度作为特征,利用改进的隐马尔科夫模型实现动态手势的判别,实验结果表明提出的动态手势识别算法相比于传统HMMs算法有效地排除了无效手势。此外利用提出的动静态手势识别算法有效地控制了模拟的数字电视。

Virtools环境下基于Kinect的手势识别与手部跟踪

为了实现在Virtools环境下利用手势对虚拟物体进行交互操作,设计开发了一个针对Kinect设备的插件。该插件基于Kinect的程序开发接口和Virtools的软件开发工具包,以Visual C++2010.Net为程序开发工具设计实现。首先根据Virtools的软件架构和模块开发技术,设计插件的管理器和行为模组,它们提供手势信息与手点位置;然后设计相应的扩展行为模组,利用手势选中物体零部件,零部件跟随手的运动做平移或缩放;最后通过实例验证了该插件的手势识别和手部跟踪功能的有效性,扩展了Virtools软件的应用领域。

基于Kinect骨骼跟踪技术的人机交互

基于Kinect提供的深度信息和骨骼跟踪技术进行手势识别,利用手势代替鼠标实现人机交互。首先,引入以人体为参考系的人体坐标系统,将手势位置通过坐标系变换映射到计算机屏幕,实现计算机屏幕上光标的位置显示和跟踪。其次,研究计算机操作系统响应鼠标事件的工作模式,设计固定的手势静态及动态工作模式,通过一定的映射关系将特定的一组手势与计算机系统基本操作指令进行对应。并定义状态机,根据手势状态触发系统操作事件,从而达到对计算机系统的体感控制。实验表明,该人机交互系统自然、合理、有效。

基于Kinect的手部分割混合识别技术

基于人手的人机交互由于其便利性,是未来人机交互技术发展的重要方向之一。基于Kinect硬件,提出了一种自适应的混合型手部分割方法,结合基于深度图像的阈值分割方法和基于彩色图像的肤色分割方法,对人体图像进行手部分割。该混合型手部分割方法,能够较好地克服光线以及复杂背景所带来的消极影响,也能够更好地适应复杂的人体动作,达到更好的分割效果。最终,搭建了硬件实测平台,验证该算法的有效性。

Kinect体感器的人体姿势交互漫游方法

针对三维虚拟场景人机交互漫游采用传统的鼠标、键盘等交互感不强的问题,提出了一种基于Kinect人体姿势交互漫游的方法。根据Kinect骨骼跟踪原理,对用户进行实时骨骼跟踪,通过骨骼点之间的相对空间位置识别人体姿势,利用特定的人体姿势实现虚拟场景的交互漫游。

基于Kinect的空中手势跟踪识别的研究与实现

以肢体语言中的手势为出发点,利用微软的Kinect体感设备,采集RGB彩色图像和深度图像,研究并提出一套手势跟踪识别的设计思路与实现方法。首先,利用Kinect所获取的色彩及深度信息,建立人体骨骼图从而更方便地定位手部位置并获取手部轮廓。其次,从HSI彩色空间对手部的深度图像进行二值化,并进行腐蚀、细化处理,去除手掌掌心部分。最后,通过对手指的计数定义相应操作。经实验测试,该系统具有对手势的识别功能,可以达到用空中手势动作进行人机交互的目的。

基于kinect传感器的全方位运输平台控制系统研究

设计一种麦克纳姆轮全向行走运输平台的体感交互控制系统。该系统应用kinect体感器提出骨骼运动信息识别和基于深度手势识别的两种控制方式,应用于不同场景。基于骨骼运动信息识别控制方式通过kinect获取人体深度图像数据,然后利用骨骼追踪技术提取人体应用关节点,并建立空间坐标系,最后通过向量计算法来计算出人体关节转动角度实现动态的动作识别进而转换为控制指令实现平台控制。基于深度手势识别控制方式利用kinect获取的深度信息实现手部从背景中分割,然后运用模板匹配的方式识别手势转换为控制指令实现平台控制。实验表明,通过该控制系统能对全方位运输平台进行有效灵活的控制。

基于Kinect和变形雅可比-傅里叶矩的手势识别

为了使人机交互变得更加自然,提出利用Kinect体感器获取手势深度图像;利用变形雅可比—傅里叶矩对手势图像进行特征提取;利用最小欧氏距离分类器进行建模、分类,实现手势识别。用Kinect体感器获取手部深度数据流,深度数据结合阈值分割法,可以有效地实现手势的分割。变形雅可比—傅里叶矩是一种不变矩,不变矩具有灰度、平移、旋转和尺度不变性,适合用于多畸变不变图像的特征提取。实验对5种手势进行了测试,平均识别率为95.2%,实验结果表明:该方法具有较高的识别率。

基于Kinect的手势交互课件应用系统的设计

针对增强现实(Augmented Reality,AR)可视化交互教学系统中,需要对课件进行手势控制的需求,本文完成了基于Kinect的手势交互课件应用系统的设计.首先通过Kinect采集深度图像,然后根据Kinect骨骼跟踪技术完成手势识别,并最终依据手势识别结果实现对课件播放的控制.实验结果表明,本方法实现课件播控能提高播控的准确率和灵活性,从而为基于AR交互课件的播控奠定基础.

基于HMM算法的Kinect工作台控制系统研究

手势识别是一种自然直接、简单丰富的人机交互方式,手势识别在日常生活中将会越来越多的被应用。本课题设计了一种基于Kinect设备的工作台控制系统。系统利用Kinect设备采集用户的手势信息,利用HMM算法实现训练数据与存储的手势数据进行比较,最后输出评价结果。得到的手势识别的结果经过ARM9处理器控制工作台的移动。本课题最终实现以较高的识别率控制了工作台的左移、右移、上移和下移。本课题创新地将手势识别应用于可改造的工作台,系统具有使用方便,成本低的特点,还可将工作台系统根据需求改造成物流分发小车或特殊人群用小车,可实现多种功能。

基于手势跟踪的人-机器人协作研究

针对人-机器人协作团队中的交互问题,本文通过对手部对象的建模,提出了一种基于均值变换的粒子滤波算法以实现手势的实时准确的跟踪;针对移动机器人运动控制特点,提出了一个简单而实用的基于手势跟踪的动态手势理解方法,并应用于一个实际的人-多机器人协作团队。实验表明通过人提供辅助性关键信息的方法,能够有效地提高了机器人团队的任务执行效率。

基于ICONDENSATION算法的人手跟踪与手势识别算法

为实现基于手势的智能人机交互,提出了一种基于视觉的人手跟踪与手势识别算法.该方法以ICON-DENSATION算法为基础,融合颜色与深度信息进行重要采样,结合轮廓信息并通过ASM模型的能量函数进行观测,根据最大后验概率对几种预定义的手势进行识别.实验结果表明,该算法能够有效地实现复杂背景下的人手跟踪与手势识别,为进一步实现智能人机交互奠定了基础.

基于计算机视觉的3D手势交互系统

随着计算机的广泛发展,键盘、鼠标等传统的人机交互方式很难满足用户自然、便捷的交互需求。研究手势建模、人手跟踪和手势交互系统的应用成为热点趋势。提出了一种简化的2D人手模型,该模型将人手建模为掌心点和5根手指,同时设计了一种基于粒子群优化(PSO)算法的人手跟踪方法,通过建模人手的生理和运动学约束关系,实现了基于2D/3D人手模型的PSO人手跟踪,该手势交互系统框架更具适用性和扩展性,融合了语义和反馈信息,提高了人手跟踪的鲁棒性和手势识别的准确度。