Quantitative analysis of the morphing wing mechanism of raptors:IMMU-based motion capture system and its application on gestures of a Falco peregrinus

This paper presented a novel tinny motion capture system for measuring bird posture based on inertial and magnetic measurement units that are made up of micromachined gyroscopes, accelerometers, and magnetometers. Multiple quaternion-based extended Kalman filters were implemented to estimate the absolute orientations to achieve high accuracy.Under the guidance of ornithology experts, the extending/contracting motions and flapping cycles were recorded using the developed motion capture system, and the orientation of each bone was also analyzed. The captured flapping gesture of the Falco peregrinus is crucial to the motion database of raptors as well as the bionic design.

Implementation of natural hand gestures in holograms for 3D object manipulation

Holograms provide a characteristic manner to display and convey information, and have been improved to provide better user interactions Holographic interactions are important as they improve user interactions with virtual objects. Gesture interaction is a recent research topic, as it allows users to use their bare hands to directly interact with the hologram. However, it remains unclear whether real hand gestures are well suited for hologram applications. Therefore, we discuss the development process and implementation of three-dimensional object manipulation using natural hand gestures in a hologram. We describe the design and development process for hologram applications and its integration with real hand gesture interactions as initial findings. Experimental results from Nasa TLX form are discussed. Based on the findings, we actualize the user interactions in the hologram.

Dynamic Hand Gesture-Based Person Identification Using Leap Motion and Machine Learning Approaches

Person identification is one of the most vital tasks for network security. People are more concerned about theirsecurity due to traditional passwords becoming weaker or leaking in various attacks. In recent decades, fingerprintsand faces have been widely used for person identification, which has the risk of information leakage as a resultof reproducing fingers or faces by taking a snapshot. Recently, people have focused on creating an identifiablepattern, which will not be reproducible falsely by capturing psychological and behavioral information of a personusing vision and sensor-based techniques. In existing studies, most of the researchers used very complex patternsin this direction, which need special training and attention to remember the patterns and failed to capturethe psychological and behavioral information of a person properly. To overcome these problems, this researchdevised a novel dynamic hand gesture-based person identification system using a Leap Motion sensor. Thisstudy developed two hand gesture-based pattern datasets for performing the experiments, which contained morethan 500 samples, collected from 25 subjects. Various static and dynamic features were extracted from the handgeometry. Randomforest was used to measure feature importance using the Gini Index. Finally, the support vectormachinewas implemented for person identification and evaluate its performance using identification accuracy. Theexperimental results showed that the proposed system produced an identification accuracy of 99.8% for arbitraryhand gesture-based patterns and 99.6% for the same dynamic hand gesture-based patterns. This result indicatedthat the proposed system can be used for person identification in the field of security.

HgaNets:Fusion of Visual Data and Skeletal Heatmap for Human Gesture Action Recognition

Recognition of human gesture actions is a challenging issue due to the complex patterns in both visual andskeletal features. Existing gesture action recognition (GAR) methods typically analyze visual and skeletal data,failing to meet the demands of various scenarios. Furthermore, multi-modal approaches lack the versatility toefficiently process both uniformand disparate input patterns.Thus, in this paper, an attention-enhanced pseudo-3Dresidual model is proposed to address the GAR problem, called HgaNets. This model comprises two independentcomponents designed formodeling visual RGB (red, green and blue) images and 3Dskeletal heatmaps, respectively.More specifically, each component consists of two main parts: 1) a multi-dimensional attention module forcapturing important spatial, temporal and feature information in human gestures;2) a spatiotemporal convolutionmodule that utilizes pseudo-3D residual convolution to characterize spatiotemporal features of gestures. Then,the output weights of the two components are fused to generate the recognition results. Finally, we conductedexperiments on four datasets to assess the efficiency of the proposed model. The results show that the accuracy onfour datasets reaches 85.40%, 91.91%, 94.70%, and 95.30%, respectively, as well as the inference time is 0.54 s andthe parameters is 2.74M. These findings highlight that the proposed model outperforms other existing approachesin terms of recognition accuracy.

Chemical simulation teaching system based on virtual reality and gesture interaction

Background Most existing chemical experiment teaching systems lack solid immersive experiences,making it difficult to engage students.To address these challenges,we propose a chemical simulation teaching system based on virtual reality and gesture interaction.Methods The parameters of the models were obtained through actual investigation,whereby Blender and 3DS MAX were used to model and import these parameters into a physics engine.By establishing an interface for the physics engine,gesture interaction hardware,and virtual reality(VR)helmet,a highly realistic chemical experiment environment was created.Using code script logic,particle systems,as well as other systems,chemical phenomena were simulated.Furthermore,we created an online teaching platform using streaming media and databases to address the problems of distance teaching.Results The proposed system was evaluated against two mainstream products in the market.In the experiments,the proposed system outperformed the other products in terms of fidelity and practicality.Conclusions The proposed system which offers realistic simulations and practicability,can help improve the high school chemistry experimental education.

Virtual Keyboard:A Real-Time Hand Gesture Recognition-Based Character Input System Using LSTM and Mediapipe Holistic

In the digital age,non-touch communication technologies are reshaping human-device interactions and raising security concerns.A major challenge in current technology is the misinterpretation of gestures by sensors and cameras,often caused by environmental factors.This issue has spurred the need for advanced data processing methods to achieve more accurate gesture recognition and predictions.Our study presents a novel virtual keyboard allowing character input via distinct hand gestures,focusing on two key aspects:hand gesture recognition and character input mechanisms.We developed a novel model with LSTM and fully connected layers for enhanced sequential data processing and hand gesture recognition.We also integrated CNN,max-pooling,and dropout layers for improved spatial feature extraction.This model architecture processes both temporal and spatial aspects of hand gestures,using LSTM to extract complex patterns from frame sequences for a comprehensive understanding of input data.Our unique dataset,essential for training the model,includes 1,662 landmarks from dynamic hand gestures,33 postures,and 468 face landmarks,all captured in real-time using advanced pose estimation.The model demonstrated high accuracy,achieving 98.52%in hand gesture recognition and over 97%in character input across different scenarios.Its excellent performance in real-time testing underlines its practicality and effectiveness,marking a significant advancement in enhancing human-device interactions in the digital age.

A Survey of Gesture Recognition Using Frequency Modulated Continuous Wave Radar

With technology advances and human requirements increasing, human-computer interaction plays an important role in our daily lives. Among these interactions, gesture-based recognition offers a natural and intuitive user experience that does not require physical contact and is becoming increasingly prevalent across various fields. Gesture recognition systems based on Frequency Modulated Continuous Wave (FMCW) millimeter-wave radar are receiving widespread attention due to their ability to operate without wearable sensors, their robustness to environmental factors, and the excellent penetrative ability of radar signals. This paper first reviews the current main gesture recognition applications. Subsequently, we introduce the system of gesture recognition based on FMCW radar and provide a general framework for gesture recognition, including gesture data acquisition, data preprocessing, and classification methods. We then discuss typical applications of gesture recognition systems and summarize the performance of these systems in terms of experimental environment, signal acquisition, signal processing, and classification methods. Specifically, we focus our study on four typical gesture recognition systems, including air-writing recognition, gesture command recognition, sign language recognition, and text input recognition. Finally, this paper addresses the challenges and unresolved problems in FMCW radar-based gesture recognition and provides insights into potential future research directions.

Gesture Recognition Based on Time-of-Flight Sensor and Residual Neural Network

With the advancement of technology and the increase in user demands, gesture recognition played a pivotal role in the field of human-computer interaction. Among various sensing devices, Time-of-Flight (ToF) sensors were widely applied due to their low cost. This paper explored the implementation of a human hand posture recognition system using ToF sensors and residual neural networks. Firstly, this paper reviewed the typical applications of human hand recognition. Secondly, this paper designed a hand gesture recognition system using a ToF sensor VL53L5. Subsequently, data preprocessing was conducted, followed by training the constructed residual neural network. Then, the recognition results were analyzed, indicating that gesture recognition based on the residual neural network achieved an accuracy of 98.5% in a 5-class classification scenario. Finally, the paper discussed existing issues and future research directions.

Design and Implementation of Hand Gesture Detection System Using HM Model for Sign Language Recognition Development

Gesture detection is the primary and most significant step for sign language detection and sign language is the communication medium for people with speaking and hearing disabilities. This paper presents a novel method for dynamic hand gesture detection using Hidden Markov Models (HMMs) where we detect different English alphabet letters by tracing hand movements. The process involves skin color-based segmentation for hand isolation in video frames, followed by morphological operations to enhance image trajectories. Our system employs hand tracking and trajectory smoothing techniques, such as the Kalman filter, to monitor hand movements and refine gesture paths. Quantized sequences are then analyzed using the Baum-Welch Re-estimation Algorithm, an HMM-based approach. A maximum likelihood classifier is used to identify the most probable letter from the test sequences. Our method demonstrates significant improvements over traditional recognition techniques in real-time, automatic hand gesture recognition, particularly in its ability to distinguish complex gestures. The experimental results confirm the effectiveness of our approach in enhancing gesture-based sign language detection to alleviate the barrier between the deaf and hard-of-hearing community and general people.

Robust Interactive Method for Hand Gestures Recognition Using Machine Learning

The Hand Gestures Recognition(HGR)System can be employed to facilitate communication between humans and computers instead of using special input and output devices.These devices may complicate communication with computers especially for people with disabilities.Hand gestures can be defined as a natural human-to-human communication method,which also can be used in human-computer interaction.Many researchers developed various techniques and methods that aimed to understand and recognize specific hand gestures by employing one or two machine learning algorithms with a reasonable accuracy.Thiswork aims to develop a powerful hand gesture recognition model with a 100%recognition rate.We proposed an ensemble classification model that combines the most powerful machine learning classifiers to obtain diversity and improve accuracy.The majority voting method was used to aggregate accuracies produced by each classifier and get the final classification result.Our model was trained using a self-constructed dataset containing 1600 images of ten different hand gestures.The employing of canny’s edge detector and histogram of oriented gradient method was a great combination with the ensemble classifier and the recognition rate.The experimental results had shown the robustness of our proposed model.Logistic Regression and Support Vector Machine have achieved 100%accuracy.The developed model was validated using two public datasets,and the findings have proved that our model outperformed other compared studies.

Load on Bicycle Frame During Cycling with Different Speeds and Gestures

Experiment and dynamic simulation were combined to obtain the loads on bicycle frame. A dynamic model of body-bicycle system was built in ADAMS. Then the body gestures under different riding conditions were captured by a motion analysis system. Dynamic simulation was carried out after the data of body motions were input into the simulation system in ADAMS and a series of loads that the body applied on head tube, seat pillar and bottom bracket were obtained. The results show that the loads on frame and their distribution are apparently different under various riding conditions. Finally, finite element analysis was done in ANSYS, which showed that the stress and its distribution on frame were apparently different when the frame was loaded according to the bicycle testing standard and simulation respectively. An efficient way to obtain load on bicycle frame accurately was proposed, which is significant for the safety of cycling and will also be the basis for the bicycle design of digitalization, lightening and customization.

A Novel Machine Learning-Based Hand Gesture Recognition Using HCI on IoT Assisted Cloud Platform

Machine learning is a technique for analyzing data that aids the construction of mathematical models.Because of the growth of the Internet of Things(IoT)and wearable sensor devices,gesture interfaces are becoming a more natural and expedient human-machine interaction method.This type of artificial intelligence that requires minimal or no direct human intervention in decision-making is predicated on the ability of intelligent systems to self-train and detect patterns.The rise of touch-free applications and the number of deaf people have increased the significance of hand gesture recognition.Potential applications of hand gesture recognition research span from online gaming to surgical robotics.The location of the hands,the alignment of the fingers,and the hand-to-body posture are the fundamental components of hierarchical emotions in gestures.Linguistic gestures may be difficult to distinguish from nonsensical motions in the field of gesture recognition.Linguistic gestures may be difficult to distinguish from nonsensical motions in the field of gesture recognition.In this scenario,it may be difficult to overcome segmentation uncertainty caused by accidental hand motions or trembling.When a user performs the same dynamic gesture,the hand shapes and speeds of each user,as well as those often generated by the same user,vary.A machine-learning-based Gesture Recognition Framework(ML-GRF)for recognizing the beginning and end of a gesture sequence in a continuous stream of data is suggested to solve the problem of distinguishing between meaningful dynamic gestures and scattered generation.We have recommended using a similarity matching-based gesture classification approach to reduce the overall computing cost associated with identifying actions,and we have shown how an efficient feature extraction method can be used to reduce the thousands of single gesture information to four binary digit gesture codes.The findings from the simulation support the accuracy,precision,gesture recognition,sensitivity,and efficiency rates.The Machine Learning-based Gesture Recognition Framework(ML-GRF)had an accuracy rate of 98.97%,a precision rate of 97.65%,a gesture recognition rate of 98.04%,a sensitivity rate of 96.99%,and an efficiency rate of 95.12%.

A Novel SE-CNN Attention Architecture for sEMG-Based Hand Gesture Recognition

In this article,to reduce the complexity and improve the generalization ability of current gesture recognition systems,we propose a novel SE-CNN attention architecture for sEMG-based hand gesture recognition.The proposed algorithm introduces a temporal squeeze-and-excite block into a simple CNN architecture and then utilizes it to recalibrate the weights of the feature outputs from the convolutional layer.By enhancing important features while suppressing useless ones,the model realizes gesture recognition efficiently.The last procedure of the proposed algorithm is utilizing a simple attention mechanism to enhance the learned representations of sEMG signals to performmulti-channel sEMG-based gesture recognition tasks.To evaluate the effectiveness and accuracy of the proposed algorithm,we conduct experiments involving multi-gesture datasets Ninapro DB4 and Ninapro DB5 for both inter-session validation and subject-wise cross-validation.After a series of comparisons with the previous models,the proposed algorithm effectively increases the robustness with improved gesture recognition performance and generalization ability.

On the Importance of Bodily Gestures,Facial Expressions,and Intonations to Thinking Expression and Interpretation

Bodily gestures,facial expressions,and intonations are argued to be notably important features of spoken languagewhich are opposed to written language.Bodily gestures with or without spoken words can influence the clarity and density of expres-sion and involvement of listeners.Facial expressions whether or not correspond with exact thought could be"decoded"to influencethe extent of intelligibility of expression.Intonation can always reflect the mutual beliefs concerning the propositional content andstates of consciousness relating to the expression and interpretation.Therefore,these can considerably improve or abate the accura-cy of expression and interpretation of thought.

Hand Gestures Recognition Based on One-Channel Surface EMG Signal

This paper presents an experiment using OPENBCI to collect data of two hand gestures and decoding the signal to distinguish gestures. The signal was extracted with three electrodes on the subiect’s forearm and transferred in one channel. After utilizing a Butterworth bandpass filter, we chose a novel way to detect gesture action segment. Instead of using moving average algorithm, which is based on the calculation of energy, We developed an algorithm based on the Hilbert transform to find a dynamic threshold and identified the action segment. Four features have been extracted from each activity section, generating feature vectors for classification. During the process of classification, we made a comparison between K-nearest-neighbors (KNN) and support vector machine (SVM), based on a relatively small amount of samples. Most common experiments are based on a large quantity of data to pursue a highly fitted model. But there are certain circumstances where we cannot obtain enough training data, so it makes the exploration of best method to do classification under small sample data imperative. Though KNN is known for its simplicity and practicability, it is a relatively time-consuming method. On the other hand, SVM has a better performance in terms of time requirement and recognition accuracy, due to its application of different Risk Minimization Principle. Experimental results show an average recognition rate for the SVM algorithm that is 1.25% higher than for KNN while SVM is 2.031 s shorter than that KNN.

Holographic Raman Tweezers Controlled by Hand Gestures and Voice Commands

Several attempts have appeared recently to control optical trapping systems via touch tablets and cameras instead of a mouse and joystick. Our approach is based on a modern low-cost hardware combined with fingertips and speech recognition software. Positions of operator's hands or fingertips control the positions of trapping beams in holographic optical tweezers that provide optical manipulation with microobjects. We tested and adapted two systems for hands position detection and gestures recognition – Creative Interactive Gesture Camera and Leap Motion. We further enhanced the system of Holographic Raman tweezers (HRT) by voice commands controlling the micropositioning stage and acquisition of Raman spectra. Interface communicates with HRT either directly by which requires adaptation of HRT firmware, or indirectly by simulating mouse and keyboard messages. Its utilization in real experiments speeded up the operator’s communication with the system cca. Two times in comparison with the traditional control by the mouse and the keyboard.

Engagement Detection Based on Analyzing Micro Body Gestures Using 3D CNN

This paper proposes a novel,efficient and affordable approach to detect the students’engagement levels in an e-learning environment by using webcams.Our method analyzes spatiotemporal features of e-learners’micro body gestures,which will be mapped to emotions and appropriate engagement states.The proposed engagement detection model uses a three-dimensional convolutional neural network to analyze both temporal and spatial information across video frames.We follow a transfer learning approach by using the C3D model that was trained on the Sports-1M dataset.The adopted C3D model was used based on two different approaches;as a feature extractor with linear classifiers and a classifier after applying fine-tuning to the pretrained model.Our model was tested and its performance was evaluated and compared to the existing models.It proved its effectiveness and superiority over the other existing methods with an accuracy of 94%.The results of this work will contribute to the development of smart and interactive e-learning systems with adaptive responses based on users’engagement levels.

Deep Reinforcement Learning Based Unmanned Aerial Vehicle (UAV) Control Using 3D Hand Gestures

The evident change in the design of the autopilot system produced massive help for the aviation industry and it required frequent upgrades.Reinforcement learning delivers appropriate outcomes when considering a continuous environment where the controlling Unmanned Aerial Vehicle(UAV)required maximum accuracy.In this paper,we designed a hybrid framework,which is based on Reinforcement Learning and Deep Learning where the traditional electronic flight controller is replaced by using 3D hand gestures.The algorithm is designed to take the input from 3D hand gestures and integrate with the Deep Deterministic Policy Gradient(DDPG)to receive the best reward and take actions according to 3D hand gestures input.The UAV consist of a Jetson Nano embedded testbed,Global Positioning System(GPS)sensor module,and Intel depth camera.The collision avoidance system based on the polar mask segmentation technique detects the obstacles and decides the best path according to the designed reward function.The analysis of the results has been observed providing best accuracy and computational time using novel design framework when compared with traditional Proportional Integral Derivatives(PID)flight controller.There are six reward functions estimated for 2500,5000,7500,and 10000 episodes of training,which have been normalized between 0 to−4000.The best observation has been captured on 2500 episodes where the rewards are calculated for maximum value.The achieved training accuracy of polar mask segmentation for collision avoidance is 86.36%.

Appearance Based Dynamic Hand Gesture Recognition Using 3D Separable Convolutional Neural Network

Appearance-based dynamic Hand Gesture Recognition(HGR)remains a prominent area of research in Human-Computer Interaction(HCI).Numerous environmental and computational constraints limit its real-time deployment.In addition,the performance of a model decreases as the subject’s distance from the camera increases.This study proposes a 3D separable Convolutional Neural Network(CNN),considering the model’s computa-tional complexity and recognition accuracy.The 20BN-Jester dataset was used to train the model for six gesture classes.After achieving the best offline recognition accuracy of 94.39%,the model was deployed in real-time while considering the subject’s attention,the instant of performing a gesture,and the subject’s distance from the camera.Despite being discussed in numerous research articles,the distance factor remains unresolved in real-time deployment,which leads to degraded recognition results.In the proposed approach,the distance calculation substantially improves the classification performance by reducing the impact of the subject’s distance from the camera.Additionally,the capability of feature extraction,degree of relevance,and statistical significance of the proposed model against other state-of-the-art models were validated using t-distributed Stochastic Neighbor Embedding(t-SNE),Mathew’s Correlation Coefficient(MCC),and the McNemar test,respectively.We observed that the proposed model exhibits state-of-the-art outcomes and a comparatively high significance level.

An Efficient and Robust Hand Gesture Recognition System of Sign Language Employing Finetuned Inception-V3 and Efficientnet-B0 Network

Hand Gesture Recognition(HGR)is a promising research area with an extensive range of applications,such as surgery,video game techniques,and sign language translation,where sign language is a complicated structured form of hand gestures.The fundamental building blocks of structured expressions in sign language are the arrangement of the fingers,the orientation of the hand,and the hand’s position concerning the body.The importance of HGR has increased due to the increasing number of touchless applications and the rapid growth of the hearing-impaired population.Therefore,real-time HGR is one of the most effective interaction methods between computers and humans.Developing a user-free interface with good recognition performance should be the goal of real-time HGR systems.Nowadays,Convolutional Neural Network(CNN)shows great recognition rates for different image-level classification tasks.It is challenging to train deep CNN networks like VGG-16,VGG-19,Inception-v3,and Efficientnet-B0 from scratch because only some significant labeled image datasets are available for static hand gesture images.However,an efficient and robust hand gesture recognition system of sign language employing finetuned Inception-v3 and Efficientnet-Bo network is proposed to identify hand gestures using a comparative small HGR dataset.Experiments show that Inception-v3 achieved 90%accuracy and 0.93%precision,0.91%recall,and 0.90%f1-score,respectively,while EfficientNet-B0 achieved 99%accuracy and 0.98%,0.97%,0.98%,precision,recall,and f1-score respectively.