Human Gait Recognition for Biometrics Application Based on Deep Learning Fusion Assisted Framework

The demand for a non-contact biometric approach for candidate identification has grown over the past ten years.Based on the most important biometric application,human gait analysis is a significant research topic in computer vision.Researchers have paid a lot of attention to gait recognition,specifically the identification of people based on their walking patterns,due to its potential to correctly identify people far away.Gait recognition systems have been used in a variety of applications,including security,medical examinations,identity management,and access control.These systems require a complex combination of technical,operational,and definitional considerations.The employment of gait recognition techniques and technologies has produced a number of beneficial and well-liked applications.Thiswork proposes a novel deep learning-based framework for human gait classification in video sequences.This framework’smain challenge is improving the accuracy of accuracy gait classification under varying conditions,such as carrying a bag and changing clothes.The proposed method’s first step is selecting two pre-trained deep learningmodels and training fromscratch using deep transfer learning.Next,deepmodels have been trained using static hyperparameters;however,the learning rate is calculated using the particle swarmoptimization(PSO)algorithm.Then,the best features are selected from both trained models using the Harris Hawks controlled Sine-Cosine optimization algorithm.This algorithm chooses the best features,combined in a novel correlation-based fusion technique.Finally,the fused best features are categorized using medium,bi-layer,and tri-layered neural networks.On the publicly accessible dataset known as the CASIA-B dataset,the experimental process of the suggested technique was carried out,and an improved accuracy of 94.14% was achieved.The achieved accuracy of the proposed method is improved by the recent state-of-the-art techniques that show the significance of this work.

GaitDONet: Gait Recognition Using Deep Features Optimization and Neural Network

Human gait recognition(HGR)is the process of identifying a sub-ject(human)based on their walking pattern.Each subject is a unique walking pattern and cannot be simulated by other subjects.But,gait recognition is not easy and makes the system difficult if any object is carried by a subject,such as a bag or coat.This article proposes an automated architecture based on deep features optimization for HGR.To our knowledge,it is the first architecture in which features are fused using multiset canonical correlation analysis(MCCA).In the proposed method,original video frames are processed for all 11 selected angles of the CASIA B dataset and utilized to train two fine-tuned deep learning models such as Squeezenet and Efficientnet.Deep transfer learning was used to train both fine-tuned models on selected angles,yielding two new targeted models that were later used for feature engineering.Features are extracted from the deep layer of both fine-tuned models and fused into one vector using MCCA.An improved manta ray foraging optimization algorithm is also proposed to select the best features from the fused feature matrix and classified using a narrow neural network classifier.The experimental process was conducted on all 11 angles of the large multi-view gait dataset(CASIA B)dataset and obtained improved accuracy than the state-of-the-art techniques.Moreover,a detailed confidence interval based analysis also shows the effectiveness of the proposed architecture for HGR.

Human Gait Recognition Based on Sequential Deep Learning and Best Features Selection

Gait recognition is an active research area that uses a walking theme to identify the subject correctly.Human Gait Recognition(HGR)is performed without any cooperation from the individual.However,in practice,it remains a challenging task under diverse walking sequences due to the covariant factors such as normal walking and walking with wearing a coat.Researchers,over the years,have worked on successfully identifying subjects using different techniques,but there is still room for improvement in accuracy due to these covariant factors.This paper proposes an automated model-free framework for human gait recognition in this article.There are a few critical steps in the proposed method.Firstly,optical flow-based motion region esti-mation and dynamic coordinates-based cropping are performed.The second step involves training a fine-tuned pre-trained MobileNetV2 model on both original and optical flow cropped frames;the training has been conducted using static hyperparameters.The third step proposed a fusion technique known as normal distribution serially fusion.In the fourth step,a better optimization algorithm is applied to select the best features,which are then classified using a Bi-Layered neural network.Three publicly available datasets,CASIA A,CASIA B,and CASIA C,were used in the experimental process and obtained average accuracies of 99.6%,91.6%,and 95.02%,respectively.The proposed framework has achieved improved accuracy compared to the other methods.

A Triplet-Branch Convolutional Neural Network for Part-Based Gait Recognition

Intelligent vision-based surveillance systems are designed to deal with the gigantic volume of videos captured in a particular environment to perform the interpretation of scenes in form of detection,tracking,monitoring,behavioral analysis,and retrievals.In addition to that,another evolving way of surveillance systems in a particular environment is human gait-based surveillance.In the existing research,several methodological frameworks are designed to use deep learning and traditional methods,nevertheless,the accuracies of these methods drop substantially when they are subjected to covariate conditions.These covariate variables disrupt the gait features and hence the recognition of subjects becomes difficult.To handle these issues,a region-based triplet-branch Convolutional Neural Network(CNN)is proposed in this research that is focused on different parts of the human Gait Energy Image(GEI)including the head,legs,and body separately to classify the subjects,and later on,the final identification of subjects is decided by probability-based majority voting criteria.Moreover,to enhance the feature extraction and draw the discriminative features,we have added soft attention layers on each branch to generate the soft attention maps.The proposed model is validated on the CASIA-B database and findings indicate that part-based learning through triplet-branch CNN shows good performance of 72.98%under covariate conditions as well as also outperforms single-branch CNN models.

A Multi-View Gait Recognition Method Using Deep Convolutional Neural Network and Channel Attention Mechanism

In many existing multi-view gait recognition methods based on images or video sequences,gait sequences are usually used to superimpose and synthesize images and construct energy-like template.However,information may be lost during the process of compositing image and capture EMG signals.Errors and the recognition accuracy may be introduced and affected respectively by some factors such as period detection.To better solve the problems,a multi-view gait recognition method using deep convolutional neural network and channel attention mechanism is proposed.Firstly,the sliding time window method is used to capture EMG signals.Then,the back-propagation learning algorithm is used to train each layer of convolution,which improves the learning ability of the convolutional neural network.Finally,the channel attention mechanism is integrated into the neural network,which will improve the ability of expressing gait features.And a classifier is used to classify gait.As can be shown from experimental results on two public datasets,OULP and CASIA-B,the recognition rate of the proposed method can be achieved at 88.44%and 97.25%respectively.As can be shown from the comparative experimental results,the proposed method has better recognition effect than several other newer convolutional neural network methods.Therefore,the combination of convolutional neural network and channel attention mechanism is of great value for gait recognition.

Human Gait Recognition Using Deep Learning and Improved Ant Colony Optimization

Human gait recognition(HGR)has received a lot of attention in the last decade as an alternative biometric technique.The main challenges in gait recognition are the change in in-person view angle and covariant factors.The major covariant factors are walking while carrying a bag and walking while wearing a coat.Deep learning is a new machine learning technique that is gaining popularity.Many techniques for HGR based on deep learning are presented in the literature.The requirement of an efficient framework is always required for correct and quick gait recognition.We proposed a fully automated deep learning and improved ant colony optimization(IACO)framework for HGR using video sequences in this work.The proposed framework consists of four primary steps.In the first step,the database is normalized in a video frame.In the second step,two pre-trained models named ResNet101 and InceptionV3 are selected andmodified according to the dataset’s nature.After that,we trained both modified models using transfer learning and extracted the features.The IACO algorithm is used to improve the extracted features.IACO is used to select the best features,which are then passed to the Cubic SVM for final classification.The cubic SVM employs a multiclass method.The experiment was carried out on three angles(0,18,and 180)of the CASIA B dataset,and the accuracy was 95.2,93.9,and 98.2 percent,respectively.A comparison with existing techniques is also performed,and the proposed method outperforms in terms of accuracy and computational time.

Human Gait Recognition:A Deep Learning and Best Feature Selection Framework

Background—Human Gait Recognition(HGR)is an approach based on biometric and is being widely used for surveillance.HGR is adopted by researchers for the past several decades.Several factors are there that affect the system performance such as the walking variation due to clothes,a person carrying some luggage,variations in the view angle.Proposed—In this work,a new method is introduced to overcome different problems of HGR.A hybrid method is proposed or efficient HGR using deep learning and selection of best features.Four major steps are involved in this work-preprocessing of the video frames,manipulation of the pre-trained CNN model VGG-16 for the computation of the features,removing redundant features extracted from the CNN model,and classification.In the reduction of irrelevant features Principal Score and Kurtosis based approach is proposed named PSbK.After that,the features of PSbK are fused in one materix.Finally,this fused vector is fed to the One against All Multi Support Vector Machine(OAMSVM)classifier for the final results.Results—The system is evaluated by utilizing the CASIA B database and six angles 00◦,18◦,36◦,54◦,72◦,and 90◦are used and attained the accuracy of 95.80%,96.0%,95.90%,96.20%,95.60%,and 95.50%,respectively.Conclusion—The comparison with recent methods show the proposed method work better.

Gait Recognition via Cross Walking Condition Constraint

Gait recognition is a biometric technique that captures human walking pattern using gait silhouettes as input and can be used for long-term recognition.Recently proposed video-based methods achieve high performance.However,gait covariates or walking conditions,i.e.,bag carrying and clothing,make the recognition of intra-class gait samples hard.Advanced methods simply use triplet loss for metric learning,which does not take the gait covariates into account.For alleviating the adverse influence of gait covariates,we propose cross walking condition constraint to explicitly consider the gait covariates.Specifically,this approach designs center-based and pair-wise loss functions to decrease discrepancy of intra-class gait samples under different walking conditions and enlarge the distance of inter-class gait samples under the same walking condition.Besides,we also propose a video-based strong baseline model of high performance by applying simple yet effective tricks,which have been validated in other individual recognition fields.With the proposed baseline model and loss functions,our method achieves the state-of-the-art performance.

Gait Recognition System in Thermal Infrared Night Imaging by Using Deep Convolutional Neural Networks

Gait is an essential biomedical feature that distinguishes individuals through walking.This feature automatically stimulates the need for remote human recognition in security-sensitive visual monitoring applications.However,there is still a lack of sufficient accuracy of gait recognition at night,in addition to taking some critical factors that affect the performances of the recognition algorithm.Therefore,a novel approach is proposed to automatically identify individuals from thermal infrared(TIR)images according to their gaits captured at night.This approach uses a new night gait network(NGaitNet)based on similarity deep convolutional neural networks(CNNs)method to enhance gait recognition at night.First,the TIR image is represented via personal movements and enhanced body skeleton segments.Then,the state-space method with a Hough transform is used to extract gait features to obtain skeletal joints′angles.These features are trained to identify the most discriminating gait patterns that indicate a change in human identity.To verify the proposed method,the experimental results are performed by using learning and validation curves via being connected by the Visdom website.The proposed thermal infrared imaging night gait recognition(TIRNGaitNet)approach has achieved the highest gait recognition accuracy rates(99.5%,97.0%),especially under normal walking conditions on the Chinese Academy of Sciences Institute of Automation infrared night gait dataset(CASIA C)and Donghua University thermal infrared night gait database(DHU night gait dataset).On the same dataset,the results of the TIRNGaitNet approach provide the record scores of(98.0%,87.0%)under the slow walking condition and(94.0%,86.0%)for the quick walking condition.

Evaluating the Effect of Various Walking Conditions on KINECT-based Gait Recognition

Human gait is one of the unobtrusive behavioral biometrics that has been extensively studied for various commercial and government applications.Biometric security,medical rehabilitation,virtual reality,and autonomous driving cars are some of the fields of study that rely on accurate gait recognition.While majority of studies have been focused on achieving very high recognition performance on a specific dataset,different issues arise in the real-world applications of this technology.This research is one of the first to evaluate the effects of changing walking speeds and directions on gait recognition rates under various walking conditions.Dataset was collected using the KINECT sensor.To draw an overall conclusion about the effects of walking speed and di-rection to the sensor,we define distance features and angle features.Furthermore,we propose two feature fusion methods for person recognition.Results of the study provide insights into how walking speeds and walking di-rections to the KINECT sensor influence the accuracy of gait recognition.

Deep Learning Approach for Hand Gesture Recognition:Applications in Deaf Communication and Healthcare

Hand gestures have been used as a significant mode of communication since the advent of human civilization.By facilitating human-computer interaction(HCI),hand gesture recognition(HGRoc)technology is crucial for seamless and error-free HCI.HGRoc technology is pivotal in healthcare and communication for the deaf community.Despite significant advancements in computer vision-based gesture recognition for language understanding,two considerable challenges persist in this field:(a)limited and common gestures are considered,(b)processing multiple channels of information across a network takes huge computational time during discriminative feature extraction.Therefore,a novel hand vision-based convolutional neural network(CNN)model named(HVCNNM)offers several benefits,notably enhanced accuracy,robustness to variations,real-time performance,reduced channels,and scalability.Additionally,these models can be optimized for real-time performance,learn from large amounts of data,and are scalable to handle complex recognition tasks for efficient human-computer interaction.The proposed model was evaluated on two challenging datasets,namely the Massey University Dataset(MUD)and the American Sign Language(ASL)Alphabet Dataset(ASLAD).On the MUD and ASLAD datasets,HVCNNM achieved a score of 99.23% and 99.00%,respectively.These results demonstrate the effectiveness of CNN as a promising HGRoc approach.The findings suggest that the proposed model have potential roles in applications such as sign language recognition,human-computer interaction,and robotics.

Human Gait Recognition Based on Kernel PCA Using Projections

This paper presents a novel approach for human identification at a distance using gait recognition. Recognition of a person from their gait is a biometric of increasing interest. The proposed work introduces a nonlinear machine learning method, kernel Principal Component Analysis （PCA）, to extract gait features from silhouettes for individual recognition. Binarized silhouette of a motion object is first represented by four 1-D signals which are the basic image features called the distance vectors. Fourier transform is performed to achieve translation invariant for the gait patterns accumulated from silhouette sequences which are extracted from different circumstances. Kernel PCA is then used to extract higher order relations among the gait patterns for future recognition. A fusion strategy is finally executed to produce a final decision. The experiments are carried out on the CMU and the USF gait databases and presented based on the different training gait cycles.

A partition approach for robust gait recognition based on gait template fusion

Gait recognition has significant potential for remote human identification,hut it is easily influenced by identity-unrelated factors such as clothing,carrying conditions,and view angles.Many gait templates have been presented that can effectively represent gait features.Each gait template has its advantages and can represent different prominent information.In this paper,gait template fusion is proposed to improve the classical representative gait template(such as a gait energy image)which represents incomplete information that is sensitive to changes in contour.We also present a partition method to reflect the different gait habits of different body parts of each pedestrian.The fused template is cropped into three parts(head,trunk,and leg regions)depending on the human body,and the three parts are then sent into the convolutional neural network to learn merged features.We present an extensive empirical evaluation of the CASIA-B dataset and compare the proposed method with existing ones.The results show good accuracy and robustness of the proposed method for gait recognition.

Feature Fusion Based Deep Transfer Learning Based Human Gait Classification Model

Gait is a biological typical that defines the method by that people walk.Walking is the most significant performance which keeps our day-to-day life and physical condition.Surface electromyography(sEMG)is a weak bioelectric signal that portrays the functional state between the human muscles and nervous system to any extent.Gait classifiers dependent upon sEMG signals are extremely utilized in analysing muscle diseases and as a guide path for recovery treatment.Several approaches are established in the works for gait recognition utilizing conventional and deep learning(DL)approaches.This study designs an Enhanced Artificial Algae Algorithm with Hybrid Deep Learning based Human Gait Classification(EAAA-HDLGR)technique on sEMG signals.The EAAA-HDLGR technique extracts the time domain(TD)and frequency domain(FD)features from the sEMG signals and is fused.In addition,the EAAA-HDLGR technique exploits the hybrid deep learning(HDL)model for gait recognition.At last,an EAAA-based hyperparameter optimizer is applied for the HDL model,which is mainly derived from the quasi-oppositional based learning(QOBL)concept,showing the novelty of the work.A brief classifier outcome of the EAAA-HDLGR technique is examined under diverse aspects,and the results indicate improving the EAAA-HDLGR technique.The results imply that the EAAA-HDLGR technique accomplishes improved results with the inclusion of EAAA on gait recognition.

Gait Based Human Recognition with Various Classifiers Using Exhaustive Angle Calculations in Model Free Approach

Human Gait recognition is emerging as a supportive biometric technique in recent years that identifies the people through the way they walk. The gait recognition in model free approaches faces the challenges like speed variation, cloth variation, illumination changes and view angle variations which result in the reduced recognition rate. The proposed algorithm selected the exhaustive angles from head to toe of a person, and also height and width of the same subject. The experiments were conducted using silhouettes with view angle variation, and cloth variation. The recognition rate is improved to the extent of 91% using Support vector machine classifier. The proposed method is evaluated using CASIA Gait Dataset B (The institute of Automation, ChineseAcademy of Sciences), China. Experimental results demonstrate that the proposed technique shows promising results using state of the art classifiers.

View-invariant Gait Authentication Based on Silhouette Contours Analysis and View Estimation

In this paper, we propose a novel view-invariant gait authentication method based on silhouette contours analysis and view estimation. The approach extracts Lucas-Kanade based gait flow image and head and shoulder mean shape(LKGFI-HSMS)of a human by using the Lucas-Kanade s method and procrustes shape analysis(PSA). LKGFI-HSMS can preserve the dynamic and static features of a gait sequence. The view between a person and a camera is identified for selecting the target s gait feature to overcome view variations. The similarity scores of LKGFI and HSMS are calculated. The product rule combines the two similarity scores to further improve the discrimination power of extracted features. Experimental results demonstrate that the proposed approach is robust to view variations and has a high authentication rate.

Multimodal Adaptive Identity-Recognition Algorithm Fused with Gait Perception

Identity-recognition technologies require assistive equipment,whereas they are poor in recognition accuracy and expensive.To overcome this deficiency,this paper proposes several gait feature identification algorithms.First,in combination with the collected gait information of individuals from triaxial accelerometers on smartphones,the collected information is preprocessed,and multimodal fusion is used with the existing standard datasets to yield a multimodal synthetic dataset;then,with the multimodal characteristics of the collected biological gait information,a Convolutional Neural Network based Gait Recognition(CNN-GR)model and the related scheme for the multimodal features are developed;at last,regarding the proposed CNN-GR model and scheme,a unimodal gait feature identity single-gait feature identification algorithm and a multimodal gait feature fusion identity multimodal gait information algorithm are proposed.Experimental results show that the proposed algorithms perform well in recognition accuracy,the confusion matrix,and the kappa statistic,and they have better recognition scores and robustness than the compared algorithms;thus,the proposed algorithm has prominent promise in practice.

Intelligent recognition system for viewpoint variations on gait and speech using CNN-CapsNet

Purpose-The paper aims to introduce an intelligent recognition system for viewpoint variations of gait and speech.It proposes a convolutional neural network-based capsule network(CNN-CapsNet)model and outlining the performance of the system in recognition of gait and speech variations.The proposed intelligent system mainly focuses on relative spatial hierarchies between gait features in the entities of the image due to translational invariances in sub-sampling and speech variations.Design/methodology/approach-This proposed work CNN-CapsNet is mainly used for automatic learning of feature representations based on CNNand used capsule vectors as neurons to encode all the spatial information of an image by adapting equal variances to change in viewpoint.The proposed study will resolve the discrepancies caused by cofactors and gait recognition between opinions based on a model of CNN-CapsNet.Findings-This research work provides recognition of signal,biometric-based gait recognition and sound/speech analysis.Empirical evaluations are conducted on three aspects of scenarios,namely fixed-view,cross-view and multi-view conditions.The main parameters for recognition of gait are speed,change in clothes,subjects walking with carrying object and intensity of light.Research limitations/implications-The proposed CNN-CapsNet has some limitations when considering for detecting the walking targets from surveillance videos considering multimodal fusion approaches using hardware sensor devices.It can also act as a pre-requisite tool to analyze,identify,detect and verify the malware practices.Practical implications-This research work includes for detecting the walking targets from surveillance videos considering multimodal fusion approaches using hardware sensor devices.It can also act as a pre-requisite tool to analyze,identify,detect and verify the malware practices.Originality/value-This proposed research work proves to be performing better for the recognition of gait and speech when compared with other techniques.

A Novel Gait Pattern Recognition Method Based on LSTM-CNN for Lower Limb Exoskeleton

This paper describes a novel gait pattern recognition method based on Long Short-Term Memory(LSTM)and Convolutional Neural Network(CNN)for lower limb exoskeleton.The Inertial Measurement Unit(IMU)installed on the exoskeleton to collect motion information,which is used for LSTM-CNN input.This article considers five common gait patterns,including walking,going up stairs,going down stairs,sitting down,and standing up.In the LSTM-CNN model,the LSTM layer is used to process temporal sequences and the CNN layer is used to extract features.To optimize the deep neural network structure proposed in this paper,some hyperparameter selection experiments were carried out.In addition,to verify the superiority of the proposed recognition method,the method is compared with several common methods such as LSTM,CNN and SVM.The results show that the average recognition accuracy can reach 97.78%,which has a good recognition eff ect.Finally,according to the experimental results of gait pattern switching,the proposed method can identify the switching gait pattern in time,which shows that it has good real-time performance.

Critical review of the use and scientific basis of forensic gait analysis

This review summarizes the scientific basis of forensic gait analysis and evaluates its use in the Netherlands,United Kingdom and Denmark,following recent critique on the admission of gait evidence in Canada.A useful forensic feature is(1)measurable,(2)consistent within and(3)different between individuals.Reviewing the academic literature,this article found that(1)forensic gait features can be quantified or observed from surveillance video,but research into accuracy,validity and reliability of these methods is needed;(2)gait is variable within individuals under differing and constant circumstances,with speed having major influence;(3)the discriminative strength of gait features needs more research,although clearly variation exists between individuals.Nevertheless,forensic gait analysis has contributed to several criminal trials in Europe in the past 15 years.The admission of gait evidence differs between courts.The methods are mainly observer-based:multiple gait analysts(independently)assess gait features on video footage of a perpetrator and suspect.Using gait feature databases,likelihood ratios of the hypotheses that the observed individuals have the same or another identity can be calculated.Automated gait recognition algorithms calculate a difference measure between video clips,which is compared with a threshold value derived from a video gait recognition database to indicate likelihood.However,only partly automated algorithms have been used in practice.We argue that the scientific basis of forensic gait analysis is limited.However,gait feature databases enable its use in court for supportive evidence with relatively low evidential value.The recommendations made in this review are(1)to expand knowledge on inter-and intra-subject gait variabilities,discriminative strength and interdependency of gait features,method accuracies,gait feature databases and likelihood ratio estimations;(2)to compare automated and observer-based gait recognition methods;to design(3)an international standard method with known validity,reliability and proficiency tests for analysts;(4)an international standard gait feature data collection method resulting in database(s);(5)(inter)national guidelines for the admission of gait evidence in court;and(6)to decrease the risk for cognitive and contextual bias in forensic gait analysis.This is expected to improve admission of gait evidence in court and judgment of its evidential value.Several ongoing research projects focus on parts of these recommendations.