Human Gait Recognition for Biometrics Application Based on Deep Learning Fusion Assisted Framework

The demand for a non-contact biometric approach for candidate identification has grown over the past ten years.Based on the most important biometric application,human gait analysis is a significant research topic in computer vision.Researchers have paid a lot of attention to gait recognition,specifically the identification of people based on their walking patterns,due to its potential to correctly identify people far away.Gait recognition systems have been used in a variety of applications,including security,medical examinations,identity management,and access control.These systems require a complex combination of technical,operational,and definitional considerations.The employment of gait recognition techniques and technologies has produced a number of beneficial and well-liked applications.Thiswork proposes a novel deep learning-based framework for human gait classification in video sequences.This framework’smain challenge is improving the accuracy of accuracy gait classification under varying conditions,such as carrying a bag and changing clothes.The proposed method’s first step is selecting two pre-trained deep learningmodels and training fromscratch using deep transfer learning.Next,deepmodels have been trained using static hyperparameters;however,the learning rate is calculated using the particle swarmoptimization(PSO)algorithm.Then,the best features are selected from both trained models using the Harris Hawks controlled Sine-Cosine optimization algorithm.This algorithm chooses the best features,combined in a novel correlation-based fusion technique.Finally,the fused best features are categorized using medium,bi-layer,and tri-layered neural networks.On the publicly accessible dataset known as the CASIA-B dataset,the experimental process of the suggested technique was carried out,and an improved accuracy of 94.14% was achieved.The achieved accuracy of the proposed method is improved by the recent state-of-the-art techniques that show the significance of this work.

GaitDONet: Gait Recognition Using Deep Features Optimization and Neural Network

Human gait recognition(HGR)is the process of identifying a sub-ject(human)based on their walking pattern.Each subject is a unique walking pattern and cannot be simulated by other subjects.But,gait recognition is not easy and makes the system difficult if any object is carried by a subject,such as a bag or coat.This article proposes an automated architecture based on deep features optimization for HGR.To our knowledge,it is the first architecture in which features are fused using multiset canonical correlation analysis(MCCA).In the proposed method,original video frames are processed for all 11 selected angles of the CASIA B dataset and utilized to train two fine-tuned deep learning models such as Squeezenet and Efficientnet.Deep transfer learning was used to train both fine-tuned models on selected angles,yielding two new targeted models that were later used for feature engineering.Features are extracted from the deep layer of both fine-tuned models and fused into one vector using MCCA.An improved manta ray foraging optimization algorithm is also proposed to select the best features from the fused feature matrix and classified using a narrow neural network classifier.The experimental process was conducted on all 11 angles of the large multi-view gait dataset(CASIA B)dataset and obtained improved accuracy than the state-of-the-art techniques.Moreover,a detailed confidence interval based analysis also shows the effectiveness of the proposed architecture for HGR.

Human Gait Recognition Based on Sequential Deep Learning and Best Features Selection

Gait recognition is an active research area that uses a walking theme to identify the subject correctly.Human Gait Recognition(HGR)is performed without any cooperation from the individual.However,in practice,it remains a challenging task under diverse walking sequences due to the covariant factors such as normal walking and walking with wearing a coat.Researchers,over the years,have worked on successfully identifying subjects using different techniques,but there is still room for improvement in accuracy due to these covariant factors.This paper proposes an automated model-free framework for human gait recognition in this article.There are a few critical steps in the proposed method.Firstly,optical flow-based motion region esti-mation and dynamic coordinates-based cropping are performed.The second step involves training a fine-tuned pre-trained MobileNetV2 model on both original and optical flow cropped frames;the training has been conducted using static hyperparameters.The third step proposed a fusion technique known as normal distribution serially fusion.In the fourth step,a better optimization algorithm is applied to select the best features,which are then classified using a Bi-Layered neural network.Three publicly available datasets,CASIA A,CASIA B,and CASIA C,were used in the experimental process and obtained average accuracies of 99.6%,91.6%,and 95.02%,respectively.The proposed framework has achieved improved accuracy compared to the other methods.

A Triplet-Branch Convolutional Neural Network for Part-Based Gait Recognition

Intelligent vision-based surveillance systems are designed to deal with the gigantic volume of videos captured in a particular environment to perform the interpretation of scenes in form of detection,tracking,monitoring,behavioral analysis,and retrievals.In addition to that,another evolving way of surveillance systems in a particular environment is human gait-based surveillance.In the existing research,several methodological frameworks are designed to use deep learning and traditional methods,nevertheless,the accuracies of these methods drop substantially when they are subjected to covariate conditions.These covariate variables disrupt the gait features and hence the recognition of subjects becomes difficult.To handle these issues,a region-based triplet-branch Convolutional Neural Network(CNN)is proposed in this research that is focused on different parts of the human Gait Energy Image(GEI)including the head,legs,and body separately to classify the subjects,and later on,the final identification of subjects is decided by probability-based majority voting criteria.Moreover,to enhance the feature extraction and draw the discriminative features,we have added soft attention layers on each branch to generate the soft attention maps.The proposed model is validated on the CASIA-B database and findings indicate that part-based learning through triplet-branch CNN shows good performance of 72.98%under covariate conditions as well as also outperforms single-branch CNN models.

Gait recognition based on Wasserstein generating adversarial image inpainting network

Aiming at the problem of small area human occlusion in gait recognition,a method based on generating adversarial image inpainting network was proposed which can generate a context consistent image for gait occlusion area.In order to reduce the effect of noise on feature extraction,the stacked automatic encoder with robustness was used.In order to improve the ability of gait classification,the sparse coding was used to express and classify the gait features.Experiments results showed the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases CASIA-B and TUM-GAID for gait recognition.

A Multi-View Gait Recognition Method Using Deep Convolutional Neural Network and Channel Attention Mechanism

In many existing multi-view gait recognition methods based on images or video sequences,gait sequences are usually used to superimpose and synthesize images and construct energy-like template.However,information may be lost during the process of compositing image and capture EMG signals.Errors and the recognition accuracy may be introduced and affected respectively by some factors such as period detection.To better solve the problems,a multi-view gait recognition method using deep convolutional neural network and channel attention mechanism is proposed.Firstly,the sliding time window method is used to capture EMG signals.Then,the back-propagation learning algorithm is used to train each layer of convolution,which improves the learning ability of the convolutional neural network.Finally,the channel attention mechanism is integrated into the neural network,which will improve the ability of expressing gait features.And a classifier is used to classify gait.As can be shown from experimental results on two public datasets,OULP and CASIA-B,the recognition rate of the proposed method can be achieved at 88.44%and 97.25%respectively.As can be shown from the comparative experimental results,the proposed method has better recognition effect than several other newer convolutional neural network methods.Therefore,the combination of convolutional neural network and channel attention mechanism is of great value for gait recognition.

Human Gait Recognition Using Deep Learning and Improved Ant Colony Optimization

Human gait recognition(HGR)has received a lot of attention in the last decade as an alternative biometric technique.The main challenges in gait recognition are the change in in-person view angle and covariant factors.The major covariant factors are walking while carrying a bag and walking while wearing a coat.Deep learning is a new machine learning technique that is gaining popularity.Many techniques for HGR based on deep learning are presented in the literature.The requirement of an efficient framework is always required for correct and quick gait recognition.We proposed a fully automated deep learning and improved ant colony optimization(IACO)framework for HGR using video sequences in this work.The proposed framework consists of four primary steps.In the first step,the database is normalized in a video frame.In the second step,two pre-trained models named ResNet101 and InceptionV3 are selected andmodified according to the dataset’s nature.After that,we trained both modified models using transfer learning and extracted the features.The IACO algorithm is used to improve the extracted features.IACO is used to select the best features,which are then passed to the Cubic SVM for final classification.The cubic SVM employs a multiclass method.The experiment was carried out on three angles(0,18,and 180)of the CASIA B dataset,and the accuracy was 95.2,93.9,and 98.2 percent,respectively.A comparison with existing techniques is also performed,and the proposed method outperforms in terms of accuracy and computational time.

Novel Walking Stability-Based Gait Recognition Method for Functional Electrical Stimulation System Control

Gait recognition is the key question of functional electrical stimulation (FES) system control for paraplegic walking. A new risk-tendency-graph (RTG) method was proposed to recognize the stability information in FES-assisted walking gait. The main instrument was a specialized walker dynamometer system based on a multi-channel strain-gauge bridge network fixed on the walker frame. During walking process, this system collected the reaction forces between patient's upper extremities and walker and converted them into RTG morphologic curves of dynamic gait stability in temporal and spatial domains. To demonstrate the potential usefulness of RTG, preliminary clinical trials were done with paraplegic patients. The gait stability levels of two walking cases with 4- and 12-week FES training from one subject were quantified (0.43 and 0.19) from the results of temporal and spatial RTG. Relevant instable phases in gait cycle and dangerous inclinations of patient's body during walking process were also brought forward. In conclusion, the new RTG method is practical for distinguishing more useful gait stability information for FES system control.

Human Gait Recognition:A Deep Learning and Best Feature Selection Framework

Background—Human Gait Recognition(HGR)is an approach based on biometric and is being widely used for surveillance.HGR is adopted by researchers for the past several decades.Several factors are there that affect the system performance such as the walking variation due to clothes,a person carrying some luggage,variations in the view angle.Proposed—In this work,a new method is introduced to overcome different problems of HGR.A hybrid method is proposed or efficient HGR using deep learning and selection of best features.Four major steps are involved in this work-preprocessing of the video frames,manipulation of the pre-trained CNN model VGG-16 for the computation of the features,removing redundant features extracted from the CNN model,and classification.In the reduction of irrelevant features Principal Score and Kurtosis based approach is proposed named PSbK.After that,the features of PSbK are fused in one materix.Finally,this fused vector is fed to the One against All Multi Support Vector Machine(OAMSVM)classifier for the final results.Results—The system is evaluated by utilizing the CASIA B database and six angles 00◦,18◦,36◦,54◦,72◦,and 90◦are used and attained the accuracy of 95.80%,96.0%,95.90%,96.20%,95.60%,and 95.50%,respectively.Conclusion—The comparison with recent methods show the proposed method work better.

Gait Recognition via Cross Walking Condition Constraint

Gait recognition is a biometric technique that captures human walking pattern using gait silhouettes as input and can be used for long-term recognition.Recently proposed video-based methods achieve high performance.However,gait covariates or walking conditions,i.e.,bag carrying and clothing,make the recognition of intra-class gait samples hard.Advanced methods simply use triplet loss for metric learning,which does not take the gait covariates into account.For alleviating the adverse influence of gait covariates,we propose cross walking condition constraint to explicitly consider the gait covariates.Specifically,this approach designs center-based and pair-wise loss functions to decrease discrepancy of intra-class gait samples under different walking conditions and enlarge the distance of inter-class gait samples under the same walking condition.Besides,we also propose a video-based strong baseline model of high performance by applying simple yet effective tricks,which have been validated in other individual recognition fields.With the proposed baseline model and loss functions,our method achieves the state-of-the-art performance.

Gait Recognition System in Thermal Infrared Night Imaging by Using Deep Convolutional Neural Networks

Gait is an essential biomedical feature that distinguishes individuals through walking.This feature automatically stimulates the need for remote human recognition in security-sensitive visual monitoring applications.However,there is still a lack of sufficient accuracy of gait recognition at night,in addition to taking some critical factors that affect the performances of the recognition algorithm.Therefore,a novel approach is proposed to automatically identify individuals from thermal infrared(TIR)images according to their gaits captured at night.This approach uses a new night gait network(NGaitNet)based on similarity deep convolutional neural networks(CNNs)method to enhance gait recognition at night.First,the TIR image is represented via personal movements and enhanced body skeleton segments.Then,the state-space method with a Hough transform is used to extract gait features to obtain skeletal joints′angles.These features are trained to identify the most discriminating gait patterns that indicate a change in human identity.To verify the proposed method,the experimental results are performed by using learning and validation curves via being connected by the Visdom website.The proposed thermal infrared imaging night gait recognition(TIRNGaitNet)approach has achieved the highest gait recognition accuracy rates(99.5%,97.0%),especially under normal walking conditions on the Chinese Academy of Sciences Institute of Automation infrared night gait dataset(CASIA C)and Donghua University thermal infrared night gait database(DHU night gait dataset).On the same dataset,the results of the TIRNGaitNet approach provide the record scores of(98.0%,87.0%)under the slow walking condition and(94.0%,86.0%)for the quick walking condition.

Interframe Variation Vector:A Novel Feature for Gait Recognition

Gait representation is an important issue in gait recognition. A simple yet efficient approach, called Interframe Variation Vector （IW）, is proposed. IW considers the spatiotemporal motion characteristic of gait, and uses the shape variation information between successive frames to represent gait signature. Different from other features, IVV rather than condenses a gait sequence into single image resulting in spatial sequence lost; it records the whole moving process in an IVV sequence. IVV can encode whole essential features of gait and preserve all the movements of limbs. Experimental results show that the proposed gait representation has a promising recognition performance.

Evaluating the Effect of Various Walking Conditions on KINECT-based Gait Recognition

Human gait is one of the unobtrusive behavioral biometrics that has been extensively studied for various commercial and government applications.Biometric security,medical rehabilitation,virtual reality,and autonomous driving cars are some of the fields of study that rely on accurate gait recognition.While majority of studies have been focused on achieving very high recognition performance on a specific dataset,different issues arise in the real-world applications of this technology.This research is one of the first to evaluate the effects of changing walking speeds and directions on gait recognition rates under various walking conditions.Dataset was collected using the KINECT sensor.To draw an overall conclusion about the effects of walking speed and di-rection to the sensor,we define distance features and angle features.Furthermore,we propose two feature fusion methods for person recognition.Results of the study provide insights into how walking speeds and walking di-rections to the KINECT sensor influence the accuracy of gait recognition.

Deep Learning Approach for Hand Gesture Recognition:Applications in Deaf Communication and Healthcare

Hand gestures have been used as a significant mode of communication since the advent of human civilization.By facilitating human-computer interaction(HCI),hand gesture recognition(HGRoc)technology is crucial for seamless and error-free HCI.HGRoc technology is pivotal in healthcare and communication for the deaf community.Despite significant advancements in computer vision-based gesture recognition for language understanding,two considerable challenges persist in this field:(a)limited and common gestures are considered,(b)processing multiple channels of information across a network takes huge computational time during discriminative feature extraction.Therefore,a novel hand vision-based convolutional neural network(CNN)model named(HVCNNM)offers several benefits,notably enhanced accuracy,robustness to variations,real-time performance,reduced channels,and scalability.Additionally,these models can be optimized for real-time performance,learn from large amounts of data,and are scalable to handle complex recognition tasks for efficient human-computer interaction.The proposed model was evaluated on two challenging datasets,namely the Massey University Dataset(MUD)and the American Sign Language(ASL)Alphabet Dataset(ASLAD).On the MUD and ASLAD datasets,HVCNNM achieved a score of 99.23% and 99.00%,respectively.These results demonstrate the effectiveness of CNN as a promising HGRoc approach.The findings suggest that the proposed model have potential roles in applications such as sign language recognition,human-computer interaction,and robotics.

A Novel 3D Gait Model for Subject Identification Robust against Carrying and Dressing Variations

Subject identification via the subject’s gait is challenging due to variations in the subject’s carrying and dressing conditions in real-life scenes.This paper proposes a novel targeted 3-dimensional(3D)gait model(3DGait)represented by a set of interpretable 3DGait descriptors based on a 3D parametric body model.The 3DGait descriptors are utilised as invariant gait features in the 3DGait recognition method to address object carrying and dressing.The 3DGait recognitionmethod involves 2-dimensional(2D)to 3DGaitdata learningbasedon3Dvirtual samples,a semantic gait parameter estimation Long Short Time Memory(LSTM)network(3D-SGPE-LSTM),a feature fusion deep model based on a multi-set canonical correlation analysis,and SoftMax recognition network.First,a sensory experiment based on 3D body shape and pose deformation with 3D virtual dressing is used to fit 3DGait onto the given 2D gait images.3Dinterpretable semantic parameters control the 3D morphing and dressing involved.Similarity degree measurement determines the semantic descriptors of 2D gait images of subjects with various shapes,poses and styles.Second,using the 2D gait images as input and the subjects’corresponding 3D semantic descriptors as output,an end-to-end 3D-SGPE-LSTM is constructed and trained.Third,body shape,pose and external gait factors(3D-eFactors)are estimated using the 3D-SGPE-LSTM model to create a set of interpretable gait descriptors to represent the 3DGait Model,i.e.,3D intrinsic semantic shape descriptor(3DShape);3D skeleton-based gait pose descriptor(3D-Pose)and 3D dressing with other 3D-eFators.Finally,the 3D-Shape and 3D-Pose descriptors are coupled to a unified pattern space by learning prior knowledge from the 3D-eFators.Practical research on CASIA B,CMU MoBo,TUM GAID and GPJATK databases shows that 3DGait is robust against object carrying and dressing variations,especially under multi-cross variations.

Human Gait Recognition Based on Kernel PCA Using Projections

This paper presents a novel approach for human identification at a distance using gait recognition. Recognition of a person from their gait is a biometric of increasing interest. The proposed work introduces a nonlinear machine learning method, kernel Principal Component Analysis （PCA）, to extract gait features from silhouettes for individual recognition. Binarized silhouette of a motion object is first represented by four 1-D signals which are the basic image features called the distance vectors. Fourier transform is performed to achieve translation invariant for the gait patterns accumulated from silhouette sequences which are extracted from different circumstances. Kernel PCA is then used to extract higher order relations among the gait patterns for future recognition. A fusion strategy is finally executed to produce a final decision. The experiments are carried out on the CMU and the USF gait databases and presented based on the different training gait cycles.

A partition approach for robust gait recognition based on gait template fusion

Gait recognition has significant potential for remote human identification,hut it is easily influenced by identity-unrelated factors such as clothing,carrying conditions,and view angles.Many gait templates have been presented that can effectively represent gait features.Each gait template has its advantages and can represent different prominent information.In this paper,gait template fusion is proposed to improve the classical representative gait template(such as a gait energy image)which represents incomplete information that is sensitive to changes in contour.We also present a partition method to reflect the different gait habits of different body parts of each pedestrian.The fused template is cropped into three parts(head,trunk,and leg regions)depending on the human body,and the three parts are then sent into the convolutional neural network to learn merged features.We present an extensive empirical evaluation of the CASIA-B dataset and compare the proposed method with existing ones.The results show good accuracy and robustness of the proposed method for gait recognition.

Probabilistic Model-Based Silhouette Refinement for Gait Recognition

An algorithm to refine and clean gait silhouette noises generated by imperfect motion detection techniques is developed,and a relatively complete and high quality silhouette is obtained.The silhouettes are sequentially refined in two levels according to two different probabilistic models.The first level is within-sequence refinement.Each silhouette in a particular sequence is refined by an individual model trained by the gait images from current sequence.The second level is between-sequence refinement.All the silhouettes that need further refinement are modified by a population model trained by the gait images chosen from a certain amount of pedestrians.The intention is to preserve the within-class similarity and to decrease the interaction between one class and others.Comparative experimental results indicate that the proposed algorithm is simple and quite effective,and it helps the existing recognition methods achieve a higher recognition performance.

Feature Fusion Based Deep Transfer Learning Based Human Gait Classification Model

Gait is a biological typical that defines the method by that people walk.Walking is the most significant performance which keeps our day-to-day life and physical condition.Surface electromyography(sEMG)is a weak bioelectric signal that portrays the functional state between the human muscles and nervous system to any extent.Gait classifiers dependent upon sEMG signals are extremely utilized in analysing muscle diseases and as a guide path for recovery treatment.Several approaches are established in the works for gait recognition utilizing conventional and deep learning(DL)approaches.This study designs an Enhanced Artificial Algae Algorithm with Hybrid Deep Learning based Human Gait Classification(EAAA-HDLGR)technique on sEMG signals.The EAAA-HDLGR technique extracts the time domain(TD)and frequency domain(FD)features from the sEMG signals and is fused.In addition,the EAAA-HDLGR technique exploits the hybrid deep learning(HDL)model for gait recognition.At last,an EAAA-based hyperparameter optimizer is applied for the HDL model,which is mainly derived from the quasi-oppositional based learning(QOBL)concept,showing the novelty of the work.A brief classifier outcome of the EAAA-HDLGR technique is examined under diverse aspects,and the results indicate improving the EAAA-HDLGR technique.The results imply that the EAAA-HDLGR technique accomplishes improved results with the inclusion of EAAA on gait recognition.

Gait Based Human Recognition with Various Classifiers Using Exhaustive Angle Calculations in Model Free Approach

Human Gait recognition is emerging as a supportive biometric technique in recent years that identifies the people through the way they walk. The gait recognition in model free approaches faces the challenges like speed variation, cloth variation, illumination changes and view angle variations which result in the reduced recognition rate. The proposed algorithm selected the exhaustive angles from head to toe of a person, and also height and width of the same subject. The experiments were conducted using silhouettes with view angle variation, and cloth variation. The recognition rate is improved to the extent of 91% using Support vector machine classifier. The proposed method is evaluated using CASIA Gait Dataset B (The institute of Automation, ChineseAcademy of Sciences), China. Experimental results demonstrate that the proposed technique shows promising results using state of the art classifiers.