The demand for a non-contact biometric approach for candidate identification has grown over the past ten years.Based on the most important biometric application,human gait analysis is a significant research topic in c...The demand for a non-contact biometric approach for candidate identification has grown over the past ten years.Based on the most important biometric application,human gait analysis is a significant research topic in computer vision.Researchers have paid a lot of attention to gait recognition,specifically the identification of people based on their walking patterns,due to its potential to correctly identify people far away.Gait recognition systems have been used in a variety of applications,including security,medical examinations,identity management,and access control.These systems require a complex combination of technical,operational,and definitional considerations.The employment of gait recognition techniques and technologies has produced a number of beneficial and well-liked applications.Thiswork proposes a novel deep learning-based framework for human gait classification in video sequences.This framework’smain challenge is improving the accuracy of accuracy gait classification under varying conditions,such as carrying a bag and changing clothes.The proposed method’s first step is selecting two pre-trained deep learningmodels and training fromscratch using deep transfer learning.Next,deepmodels have been trained using static hyperparameters;however,the learning rate is calculated using the particle swarmoptimization(PSO)algorithm.Then,the best features are selected from both trained models using the Harris Hawks controlled Sine-Cosine optimization algorithm.This algorithm chooses the best features,combined in a novel correlation-based fusion technique.Finally,the fused best features are categorized using medium,bi-layer,and tri-layered neural networks.On the publicly accessible dataset known as the CASIA-B dataset,the experimental process of the suggested technique was carried out,and an improved accuracy of 94.14% was achieved.The achieved accuracy of the proposed method is improved by the recent state-of-the-art techniques that show the significance of this work.展开更多
Aiming at the problem of small area human occlusion in gait recognition,a method based on generating adversarial image inpainting network was proposed which can generate a context consistent image for gait occlusion a...Aiming at the problem of small area human occlusion in gait recognition,a method based on generating adversarial image inpainting network was proposed which can generate a context consistent image for gait occlusion area.In order to reduce the effect of noise on feature extraction,the stacked automatic encoder with robustness was used.In order to improve the ability of gait classification,the sparse coding was used to express and classify the gait features.Experiments results showed the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases CASIA-B and TUM-GAID for gait recognition.展开更多
Hand gestures have been used as a significant mode of communication since the advent of human civilization.By facilitating human-computer interaction(HCI),hand gesture recognition(HGRoc)technology is crucial for seaml...Hand gestures have been used as a significant mode of communication since the advent of human civilization.By facilitating human-computer interaction(HCI),hand gesture recognition(HGRoc)technology is crucial for seamless and error-free HCI.HGRoc technology is pivotal in healthcare and communication for the deaf community.Despite significant advancements in computer vision-based gesture recognition for language understanding,two considerable challenges persist in this field:(a)limited and common gestures are considered,(b)processing multiple channels of information across a network takes huge computational time during discriminative feature extraction.Therefore,a novel hand vision-based convolutional neural network(CNN)model named(HVCNNM)offers several benefits,notably enhanced accuracy,robustness to variations,real-time performance,reduced channels,and scalability.Additionally,these models can be optimized for real-time performance,learn from large amounts of data,and are scalable to handle complex recognition tasks for efficient human-computer interaction.The proposed model was evaluated on two challenging datasets,namely the Massey University Dataset(MUD)and the American Sign Language(ASL)Alphabet Dataset(ASLAD).On the MUD and ASLAD datasets,HVCNNM achieved a score of 99.23% and 99.00%,respectively.These results demonstrate the effectiveness of CNN as a promising HGRoc approach.The findings suggest that the proposed model have potential roles in applications such as sign language recognition,human-computer interaction,and robotics.展开更多
A facial expression emotion recognition based human-robot interaction(FEER-HRI) system is proposed, for which a four-layer system framework is designed. The FEERHRI system enables the robots not only to recognize huma...A facial expression emotion recognition based human-robot interaction(FEER-HRI) system is proposed, for which a four-layer system framework is designed. The FEERHRI system enables the robots not only to recognize human emotions, but also to generate facial expression for adapting to human emotions. A facial emotion recognition method based on2D-Gabor, uniform local binary pattern(LBP) operator, and multiclass extreme learning machine(ELM) classifier is presented,which is applied to real-time facial expression recognition for robots. Facial expressions of robots are represented by simple cartoon symbols and displayed by a LED screen equipped in the robots, which can be easily understood by human. Four scenarios,i.e., guiding, entertainment, home service and scene simulation are performed in the human-robot interaction experiment, in which smooth communication is realized by facial expression recognition of humans and facial expression generation of robots within 2 seconds. As a few prospective applications, the FEERHRI system can be applied in home service, smart home, safe driving, and so on.展开更多
In many existing multi-view gait recognition methods based on images or video sequences,gait sequences are usually used to superimpose and synthesize images and construct energy-like template.However,information may b...In many existing multi-view gait recognition methods based on images or video sequences,gait sequences are usually used to superimpose and synthesize images and construct energy-like template.However,information may be lost during the process of compositing image and capture EMG signals.Errors and the recognition accuracy may be introduced and affected respectively by some factors such as period detection.To better solve the problems,a multi-view gait recognition method using deep convolutional neural network and channel attention mechanism is proposed.Firstly,the sliding time window method is used to capture EMG signals.Then,the back-propagation learning algorithm is used to train each layer of convolution,which improves the learning ability of the convolutional neural network.Finally,the channel attention mechanism is integrated into the neural network,which will improve the ability of expressing gait features.And a classifier is used to classify gait.As can be shown from experimental results on two public datasets,OULP and CASIA-B,the recognition rate of the proposed method can be achieved at 88.44%and 97.25%respectively.As can be shown from the comparative experimental results,the proposed method has better recognition effect than several other newer convolutional neural network methods.Therefore,the combination of convolutional neural network and channel attention mechanism is of great value for gait recognition.展开更多
Human gait recognition(HGR)has received a lot of attention in the last decade as an alternative biometric technique.The main challenges in gait recognition are the change in in-person view angle and covariant factors....Human gait recognition(HGR)has received a lot of attention in the last decade as an alternative biometric technique.The main challenges in gait recognition are the change in in-person view angle and covariant factors.The major covariant factors are walking while carrying a bag and walking while wearing a coat.Deep learning is a new machine learning technique that is gaining popularity.Many techniques for HGR based on deep learning are presented in the literature.The requirement of an efficient framework is always required for correct and quick gait recognition.We proposed a fully automated deep learning and improved ant colony optimization(IACO)framework for HGR using video sequences in this work.The proposed framework consists of four primary steps.In the first step,the database is normalized in a video frame.In the second step,two pre-trained models named ResNet101 and InceptionV3 are selected andmodified according to the dataset’s nature.After that,we trained both modified models using transfer learning and extracted the features.The IACO algorithm is used to improve the extracted features.IACO is used to select the best features,which are then passed to the Cubic SVM for final classification.The cubic SVM employs a multiclass method.The experiment was carried out on three angles(0,18,and 180)of the CASIA B dataset,and the accuracy was 95.2,93.9,and 98.2 percent,respectively.A comparison with existing techniques is also performed,and the proposed method outperforms in terms of accuracy and computational time.展开更多
Gait recognition is a biometric technique that captures human walking pattern using gait silhouettes as input and can be used for long-term recognition.Recently proposed video-based methods achieve high performance.Ho...Gait recognition is a biometric technique that captures human walking pattern using gait silhouettes as input and can be used for long-term recognition.Recently proposed video-based methods achieve high performance.However,gait covariates or walking conditions,i.e.,bag carrying and clothing,make the recognition of intra-class gait samples hard.Advanced methods simply use triplet loss for metric learning,which does not take the gait covariates into account.For alleviating the adverse influence of gait covariates,we propose cross walking condition constraint to explicitly consider the gait covariates.Specifically,this approach designs center-based and pair-wise loss functions to decrease discrepancy of intra-class gait samples under different walking conditions and enlarge the distance of inter-class gait samples under the same walking condition.Besides,we also propose a video-based strong baseline model of high performance by applying simple yet effective tricks,which have been validated in other individual recognition fields.With the proposed baseline model and loss functions,our method achieves the state-of-the-art performance.展开更多
Gait recognition is the key question of functional electrical stimulation (FES) system control for paraplegic walking. A new risk-tendency-graph (RTG) method was proposed to recognize the stability information in FES-...Gait recognition is the key question of functional electrical stimulation (FES) system control for paraplegic walking. A new risk-tendency-graph (RTG) method was proposed to recognize the stability information in FES-assisted walking gait. The main instrument was a specialized walker dynamometer system based on a multi-channel strain-gauge bridge network fixed on the walker frame. During walking process, this system collected the reaction forces between patient's upper extremities and walker and converted them into RTG morphologic curves of dynamic gait stability in temporal and spatial domains. To demonstrate the potential usefulness of RTG, preliminary clinical trials were done with paraplegic patients. The gait stability levels of two walking cases with 4- and 12-week FES training from one subject were quantified (0.43 and 0.19) from the results of temporal and spatial RTG. Relevant instable phases in gait cycle and dangerous inclinations of patient's body during walking process were also brought forward. In conclusion, the new RTG method is practical for distinguishing more useful gait stability information for FES system control.展开更多
Human gait recognition(HGR)is the process of identifying a sub-ject(human)based on their walking pattern.Each subject is a unique walking pattern and cannot be simulated by other subjects.But,gait recognition is not e...Human gait recognition(HGR)is the process of identifying a sub-ject(human)based on their walking pattern.Each subject is a unique walking pattern and cannot be simulated by other subjects.But,gait recognition is not easy and makes the system difficult if any object is carried by a subject,such as a bag or coat.This article proposes an automated architecture based on deep features optimization for HGR.To our knowledge,it is the first architecture in which features are fused using multiset canonical correlation analysis(MCCA).In the proposed method,original video frames are processed for all 11 selected angles of the CASIA B dataset and utilized to train two fine-tuned deep learning models such as Squeezenet and Efficientnet.Deep transfer learning was used to train both fine-tuned models on selected angles,yielding two new targeted models that were later used for feature engineering.Features are extracted from the deep layer of both fine-tuned models and fused into one vector using MCCA.An improved manta ray foraging optimization algorithm is also proposed to select the best features from the fused feature matrix and classified using a narrow neural network classifier.The experimental process was conducted on all 11 angles of the large multi-view gait dataset(CASIA B)dataset and obtained improved accuracy than the state-of-the-art techniques.Moreover,a detailed confidence interval based analysis also shows the effectiveness of the proposed architecture for HGR.展开更多
Gait recognition is an active research area that uses a walking theme to identify the subject correctly.Human Gait Recognition(HGR)is performed without any cooperation from the individual.However,in practice,it remain...Gait recognition is an active research area that uses a walking theme to identify the subject correctly.Human Gait Recognition(HGR)is performed without any cooperation from the individual.However,in practice,it remains a challenging task under diverse walking sequences due to the covariant factors such as normal walking and walking with wearing a coat.Researchers,over the years,have worked on successfully identifying subjects using different techniques,but there is still room for improvement in accuracy due to these covariant factors.This paper proposes an automated model-free framework for human gait recognition in this article.There are a few critical steps in the proposed method.Firstly,optical flow-based motion region esti-mation and dynamic coordinates-based cropping are performed.The second step involves training a fine-tuned pre-trained MobileNetV2 model on both original and optical flow cropped frames;the training has been conducted using static hyperparameters.The third step proposed a fusion technique known as normal distribution serially fusion.In the fourth step,a better optimization algorithm is applied to select the best features,which are then classified using a Bi-Layered neural network.Three publicly available datasets,CASIA A,CASIA B,and CASIA C,were used in the experimental process and obtained average accuracies of 99.6%,91.6%,and 95.02%,respectively.The proposed framework has achieved improved accuracy compared to the other methods.展开更多
Background—Human Gait Recognition(HGR)is an approach based on biometric and is being widely used for surveillance.HGR is adopted by researchers for the past several decades.Several factors are there that affect the s...Background—Human Gait Recognition(HGR)is an approach based on biometric and is being widely used for surveillance.HGR is adopted by researchers for the past several decades.Several factors are there that affect the system performance such as the walking variation due to clothes,a person carrying some luggage,variations in the view angle.Proposed—In this work,a new method is introduced to overcome different problems of HGR.A hybrid method is proposed or efficient HGR using deep learning and selection of best features.Four major steps are involved in this work-preprocessing of the video frames,manipulation of the pre-trained CNN model VGG-16 for the computation of the features,removing redundant features extracted from the CNN model,and classification.In the reduction of irrelevant features Principal Score and Kurtosis based approach is proposed named PSbK.After that,the features of PSbK are fused in one materix.Finally,this fused vector is fed to the One against All Multi Support Vector Machine(OAMSVM)classifier for the final results.Results—The system is evaluated by utilizing the CASIA B database and six angles 00◦,18◦,36◦,54◦,72◦,and 90◦are used and attained the accuracy of 95.80%,96.0%,95.90%,96.20%,95.60%,and 95.50%,respectively.Conclusion—The comparison with recent methods show the proposed method work better.展开更多
Human personality assessment using gait pattern recognition is one of the most recent and exciting research domains.Gait is a person’s identity that can reflect reliable information about his mood,emotions,and substa...Human personality assessment using gait pattern recognition is one of the most recent and exciting research domains.Gait is a person’s identity that can reflect reliable information about his mood,emotions,and substantial personality traits under scrutiny.This research focuses on recognizing key personality traits,including neuroticism,extraversion,openness to experience,agreeableness,and conscientiousness,in line with the bigfive model of personality.We inferred personality traits based on the gait pattern recognition of individuals utilizing built-in smartphone sensors.For experimentation,we collected a novel dataset of 22 participants using an android application and further segmented it into six data chunks for a critical evaluation.After data pre-processing,we extracted selected features from each data segment and then applied four multiclass machine learning algorithms for training and classifying the dataset corresponding to the users’Big-Five Personality Traits Profiles(BFPT).Experimental results and performance evaluation of the classifiers revealed the efficacy of the proposed scheme for all big-five traits.展开更多
Gait representation is an important issue in gait recognition. A simple yet efficient approach, called Interframe Variation Vector (IW), is proposed. IW considers the spatiotemporal motion characteristic of gait, an...Gait representation is an important issue in gait recognition. A simple yet efficient approach, called Interframe Variation Vector (IW), is proposed. IW considers the spatiotemporal motion characteristic of gait, and uses the shape variation information between successive frames to represent gait signature. Different from other features, IVV rather than condenses a gait sequence into single image resulting in spatial sequence lost; it records the whole moving process in an IVV sequence. IVV can encode whole essential features of gait and preserve all the movements of limbs. Experimental results show that the proposed gait representation has a promising recognition performance.展开更多
Intelligent vision-based surveillance systems are designed to deal with the gigantic volume of videos captured in a particular environment to perform the interpretation of scenes in form of detection,tracking,monitori...Intelligent vision-based surveillance systems are designed to deal with the gigantic volume of videos captured in a particular environment to perform the interpretation of scenes in form of detection,tracking,monitoring,behavioral analysis,and retrievals.In addition to that,another evolving way of surveillance systems in a particular environment is human gait-based surveillance.In the existing research,several methodological frameworks are designed to use deep learning and traditional methods,nevertheless,the accuracies of these methods drop substantially when they are subjected to covariate conditions.These covariate variables disrupt the gait features and hence the recognition of subjects becomes difficult.To handle these issues,a region-based triplet-branch Convolutional Neural Network(CNN)is proposed in this research that is focused on different parts of the human Gait Energy Image(GEI)including the head,legs,and body separately to classify the subjects,and later on,the final identification of subjects is decided by probability-based majority voting criteria.Moreover,to enhance the feature extraction and draw the discriminative features,we have added soft attention layers on each branch to generate the soft attention maps.The proposed model is validated on the CASIA-B database and findings indicate that part-based learning through triplet-branch CNN shows good performance of 72.98%under covariate conditions as well as also outperforms single-branch CNN models.展开更多
Human gait is a complex phenomenon involving the motion of various parts of the body simultaneously in a 3 dimensional space. Dynamics of different parts of the body translate its center of gravity from one point to a...Human gait is a complex phenomenon involving the motion of various parts of the body simultaneously in a 3 dimensional space. Dynamics of different parts of the body translate its center of gravity from one point to another in the most efficient way. Body dynamics as well as static parameters of different body parts contribute to gait recognition. Studies have been performed to assess the discriminatory power of static and dynamic features. The current research literature, however, lacks the work on the comparative significance of dynamic features from different parts of the body. This paper sheds some light on the recognition performance of dynamic features extracted from different parts of human body in an appearance based set up.展开更多
Gait is an essential biomedical feature that distinguishes individuals through walking.This feature automatically stimulates the need for remote human recognition in security-sensitive visual monitoring applications.H...Gait is an essential biomedical feature that distinguishes individuals through walking.This feature automatically stimulates the need for remote human recognition in security-sensitive visual monitoring applications.However,there is still a lack of sufficient accuracy of gait recognition at night,in addition to taking some critical factors that affect the performances of the recognition algorithm.Therefore,a novel approach is proposed to automatically identify individuals from thermal infrared(TIR)images according to their gaits captured at night.This approach uses a new night gait network(NGaitNet)based on similarity deep convolutional neural networks(CNNs)method to enhance gait recognition at night.First,the TIR image is represented via personal movements and enhanced body skeleton segments.Then,the state-space method with a Hough transform is used to extract gait features to obtain skeletal joints′angles.These features are trained to identify the most discriminating gait patterns that indicate a change in human identity.To verify the proposed method,the experimental results are performed by using learning and validation curves via being connected by the Visdom website.The proposed thermal infrared imaging night gait recognition(TIRNGaitNet)approach has achieved the highest gait recognition accuracy rates(99.5%,97.0%),especially under normal walking conditions on the Chinese Academy of Sciences Institute of Automation infrared night gait dataset(CASIA C)and Donghua University thermal infrared night gait database(DHU night gait dataset).On the same dataset,the results of the TIRNGaitNet approach provide the record scores of(98.0%,87.0%)under the slow walking condition and(94.0%,86.0%)for the quick walking condition.展开更多
Gait energy image(GEI)is composed of static body silhouette and dynamic frequency information of human gait.To achieve fast and efficient gait recognition,combined with the accurate description of the information of d...Gait energy image(GEI)is composed of static body silhouette and dynamic frequency information of human gait.To achieve fast and efficient gait recognition,combined with the accurate description of the information of details and directions in image by Curvelet transform,a gait recognition method using GEI and Curvelet(GEIC)is presented.Firstly,to gain the gait energy images,the gait cycle is selected according to the aspect ratio.Secondly,Curvelet energy coefficients of the GEI,which are used as gait feature vector,are extracted by Curvelet transform in different scales and different directions.Finally,the gait recognition is accomplished by the K nearest neighbor(KNN)classifier.The experimental results demonstrate that GEIC performs well on CASIA(B)database,with the average accuracy of 86.83%.Compared with GEI+KPCA,GEI+W(2D)2PCA and GEI+(2D)~2PCA,the algorithm GEIC achieves better robustness in the condition of the person wearing or packaging.展开更多
Human Gait recognition is emerging as a supportive biometric technique in recent years that identifies the people through the way they walk. The gait recognition in model free approaches faces the challenges like spee...Human Gait recognition is emerging as a supportive biometric technique in recent years that identifies the people through the way they walk. The gait recognition in model free approaches faces the challenges like speed variation, cloth variation, illumination changes and view angle variations which result in the reduced recognition rate. The proposed algorithm selected the exhaustive angles from head to toe of a person, and also height and width of the same subject. The experiments were conducted using silhouettes with view angle variation, and cloth variation. The recognition rate is improved to the extent of 91% using Support vector machine classifier. The proposed method is evaluated using CASIA Gait Dataset B (The institute of Automation, ChineseAcademy of Sciences), China. Experimental results demonstrate that the proposed technique shows promising results using state of the art classifiers.展开更多
The results of image preprocessing may directly affect gait feature extraction in the area of gait recognition. Due to the influence of light, shelter and other external factors of gait image, some problems such as lo...The results of image preprocessing may directly affect gait feature extraction in the area of gait recognition. Due to the influence of light, shelter and other external factors of gait image, some problems such as loss of information,image shadows, and improper threshold of image preprocessing may occur. In order to solve these problems, an image preprocessing method of gait recognition is proposed. Firstly, background image is extracted by background modeling, secondly, the target profile is extracted by the direct difference method; thirdly, the shadow elimination based on the HSV color model is carried out on the target profile map; Finally, the complete target profile is obtained by threshold segmentation. Experimental results on CASIA_A database demonstrate that this proposed method is quite effective on both target profile extraction and proportion comparison with the real area.展开更多
Human gait is one of the unobtrusive behavioral biometrics that has been extensively studied for various commercial and government applications.Biometric security,medical rehabilitation,virtual reality,and autonomous ...Human gait is one of the unobtrusive behavioral biometrics that has been extensively studied for various commercial and government applications.Biometric security,medical rehabilitation,virtual reality,and autonomous driving cars are some of the fields of study that rely on accurate gait recognition.While majority of studies have been focused on achieving very high recognition performance on a specific dataset,different issues arise in the real-world applications of this technology.This research is one of the first to evaluate the effects of changing walking speeds and directions on gait recognition rates under various walking conditions.Dataset was collected using the KINECT sensor.To draw an overall conclusion about the effects of walking speed and di-rection to the sensor,we define distance features and angle features.Furthermore,we propose two feature fusion methods for person recognition.Results of the study provide insights into how walking speeds and walking di-rections to the KINECT sensor influence the accuracy of gait recognition.展开更多
基金supported by the“Human Resources Program in Energy Technol-ogy”of the Korea Institute of Energy Technology Evaluation and Planning(KETEP)and Granted Financial Resources from the Ministry of Trade,Industry,and Energy,Republic of Korea(No.20204010600090)The funding of this work was provided by Princess Nourah bint Abdulrahman University Researchers Supporting Project Number(PNURSP2023R410),Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia.
文摘The demand for a non-contact biometric approach for candidate identification has grown over the past ten years.Based on the most important biometric application,human gait analysis is a significant research topic in computer vision.Researchers have paid a lot of attention to gait recognition,specifically the identification of people based on their walking patterns,due to its potential to correctly identify people far away.Gait recognition systems have been used in a variety of applications,including security,medical examinations,identity management,and access control.These systems require a complex combination of technical,operational,and definitional considerations.The employment of gait recognition techniques and technologies has produced a number of beneficial and well-liked applications.Thiswork proposes a novel deep learning-based framework for human gait classification in video sequences.This framework’smain challenge is improving the accuracy of accuracy gait classification under varying conditions,such as carrying a bag and changing clothes.The proposed method’s first step is selecting two pre-trained deep learningmodels and training fromscratch using deep transfer learning.Next,deepmodels have been trained using static hyperparameters;however,the learning rate is calculated using the particle swarmoptimization(PSO)algorithm.Then,the best features are selected from both trained models using the Harris Hawks controlled Sine-Cosine optimization algorithm.This algorithm chooses the best features,combined in a novel correlation-based fusion technique.Finally,the fused best features are categorized using medium,bi-layer,and tri-layered neural networks.On the publicly accessible dataset known as the CASIA-B dataset,the experimental process of the suggested technique was carried out,and an improved accuracy of 94.14% was achieved.The achieved accuracy of the proposed method is improved by the recent state-of-the-art techniques that show the significance of this work.
基金Project(51678075) supported by the National Natural Science Foundation of ChinaProject(2017GK2271) supported by Hunan Provincial Science and Technology Department,China
文摘Aiming at the problem of small area human occlusion in gait recognition,a method based on generating adversarial image inpainting network was proposed which can generate a context consistent image for gait occlusion area.In order to reduce the effect of noise on feature extraction,the stacked automatic encoder with robustness was used.In order to improve the ability of gait classification,the sparse coding was used to express and classify the gait features.Experiments results showed the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases CASIA-B and TUM-GAID for gait recognition.
基金funded by Researchers Supporting Project Number(RSPD2024 R947),King Saud University,Riyadh,Saudi Arabia.
文摘Hand gestures have been used as a significant mode of communication since the advent of human civilization.By facilitating human-computer interaction(HCI),hand gesture recognition(HGRoc)technology is crucial for seamless and error-free HCI.HGRoc technology is pivotal in healthcare and communication for the deaf community.Despite significant advancements in computer vision-based gesture recognition for language understanding,two considerable challenges persist in this field:(a)limited and common gestures are considered,(b)processing multiple channels of information across a network takes huge computational time during discriminative feature extraction.Therefore,a novel hand vision-based convolutional neural network(CNN)model named(HVCNNM)offers several benefits,notably enhanced accuracy,robustness to variations,real-time performance,reduced channels,and scalability.Additionally,these models can be optimized for real-time performance,learn from large amounts of data,and are scalable to handle complex recognition tasks for efficient human-computer interaction.The proposed model was evaluated on two challenging datasets,namely the Massey University Dataset(MUD)and the American Sign Language(ASL)Alphabet Dataset(ASLAD).On the MUD and ASLAD datasets,HVCNNM achieved a score of 99.23% and 99.00%,respectively.These results demonstrate the effectiveness of CNN as a promising HGRoc approach.The findings suggest that the proposed model have potential roles in applications such as sign language recognition,human-computer interaction,and robotics.
基金supported by the National Natural Science Foundation of China(61403422,61273102)the Hubei Provincial Natural Science Foundation of China(2015CFA010)+1 种基金the Ⅲ Project(B17040)the Fundamental Research Funds for National University,China University of Geosciences(Wuhan)
文摘A facial expression emotion recognition based human-robot interaction(FEER-HRI) system is proposed, for which a four-layer system framework is designed. The FEERHRI system enables the robots not only to recognize human emotions, but also to generate facial expression for adapting to human emotions. A facial emotion recognition method based on2D-Gabor, uniform local binary pattern(LBP) operator, and multiclass extreme learning machine(ELM) classifier is presented,which is applied to real-time facial expression recognition for robots. Facial expressions of robots are represented by simple cartoon symbols and displayed by a LED screen equipped in the robots, which can be easily understood by human. Four scenarios,i.e., guiding, entertainment, home service and scene simulation are performed in the human-robot interaction experiment, in which smooth communication is realized by facial expression recognition of humans and facial expression generation of robots within 2 seconds. As a few prospective applications, the FEERHRI system can be applied in home service, smart home, safe driving, and so on.
基金This work was supported by the Natural Science Foundation of China(No.61902133)Fujian natural science foundation project(No.2018J05106)Xiamen Collaborative Innovation projects of Produces study grinds(3502Z20173046)。
文摘In many existing multi-view gait recognition methods based on images or video sequences,gait sequences are usually used to superimpose and synthesize images and construct energy-like template.However,information may be lost during the process of compositing image and capture EMG signals.Errors and the recognition accuracy may be introduced and affected respectively by some factors such as period detection.To better solve the problems,a multi-view gait recognition method using deep convolutional neural network and channel attention mechanism is proposed.Firstly,the sliding time window method is used to capture EMG signals.Then,the back-propagation learning algorithm is used to train each layer of convolution,which improves the learning ability of the convolutional neural network.Finally,the channel attention mechanism is integrated into the neural network,which will improve the ability of expressing gait features.And a classifier is used to classify gait.As can be shown from experimental results on two public datasets,OULP and CASIA-B,the recognition rate of the proposed method can be achieved at 88.44%and 97.25%respectively.As can be shown from the comparative experimental results,the proposed method has better recognition effect than several other newer convolutional neural network methods.Therefore,the combination of convolutional neural network and channel attention mechanism is of great value for gait recognition.
基金This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Ministry of Education(2018R1D1A1B07042967)and the Soonchunhyang University Research Fund.
文摘Human gait recognition(HGR)has received a lot of attention in the last decade as an alternative biometric technique.The main challenges in gait recognition are the change in in-person view angle and covariant factors.The major covariant factors are walking while carrying a bag and walking while wearing a coat.Deep learning is a new machine learning technique that is gaining popularity.Many techniques for HGR based on deep learning are presented in the literature.The requirement of an efficient framework is always required for correct and quick gait recognition.We proposed a fully automated deep learning and improved ant colony optimization(IACO)framework for HGR using video sequences in this work.The proposed framework consists of four primary steps.In the first step,the database is normalized in a video frame.In the second step,two pre-trained models named ResNet101 and InceptionV3 are selected andmodified according to the dataset’s nature.After that,we trained both modified models using transfer learning and extracted the features.The IACO algorithm is used to improve the extracted features.IACO is used to select the best features,which are then passed to the Cubic SVM for final classification.The cubic SVM employs a multiclass method.The experiment was carried out on three angles(0,18,and 180)of the CASIA B dataset,and the accuracy was 95.2,93.9,and 98.2 percent,respectively.A comparison with existing techniques is also performed,and the proposed method outperforms in terms of accuracy and computational time.
基金This work was supported in part by the Natural Science Foundation of China under Grant 61972169 and U1536203in part by the National key research and development program of China(2016QY01W0200)in part by the Major Scientific and Technological Project of Hubei Province(2018AAA068 and 2019AAA051).
文摘Gait recognition is a biometric technique that captures human walking pattern using gait silhouettes as input and can be used for long-term recognition.Recently proposed video-based methods achieve high performance.However,gait covariates or walking conditions,i.e.,bag carrying and clothing,make the recognition of intra-class gait samples hard.Advanced methods simply use triplet loss for metric learning,which does not take the gait covariates into account.For alleviating the adverse influence of gait covariates,we propose cross walking condition constraint to explicitly consider the gait covariates.Specifically,this approach designs center-based and pair-wise loss functions to decrease discrepancy of intra-class gait samples under different walking conditions and enlarge the distance of inter-class gait samples under the same walking condition.Besides,we also propose a video-based strong baseline model of high performance by applying simple yet effective tricks,which have been validated in other individual recognition fields.With the proposed baseline model and loss functions,our method achieves the state-of-the-art performance.
基金Supported by National Natural Science Foundation of China (No.60501005)Key Programof Tianjin Science Technology Support Plan(No.2007-68)
文摘Gait recognition is the key question of functional electrical stimulation (FES) system control for paraplegic walking. A new risk-tendency-graph (RTG) method was proposed to recognize the stability information in FES-assisted walking gait. The main instrument was a specialized walker dynamometer system based on a multi-channel strain-gauge bridge network fixed on the walker frame. During walking process, this system collected the reaction forces between patient's upper extremities and walker and converted them into RTG morphologic curves of dynamic gait stability in temporal and spatial domains. To demonstrate the potential usefulness of RTG, preliminary clinical trials were done with paraplegic patients. The gait stability levels of two walking cases with 4- and 12-week FES training from one subject were quantified (0.43 and 0.19) from the results of temporal and spatial RTG. Relevant instable phases in gait cycle and dangerous inclinations of patient's body during walking process were also brought forward. In conclusion, the new RTG method is practical for distinguishing more useful gait stability information for FES system control.
基金supported by the MSIT(Ministry of Science and ICT),Korea,under the ICAN(ICT Challenge and Advanced Network of HRD)program(IITP-2022-2020-0-01832)supervised by the IITP(Institute of Information&Communications Technology Planning&Evaluation)and the Soonchunhyang University Research Fund.
文摘Human gait recognition(HGR)is the process of identifying a sub-ject(human)based on their walking pattern.Each subject is a unique walking pattern and cannot be simulated by other subjects.But,gait recognition is not easy and makes the system difficult if any object is carried by a subject,such as a bag or coat.This article proposes an automated architecture based on deep features optimization for HGR.To our knowledge,it is the first architecture in which features are fused using multiset canonical correlation analysis(MCCA).In the proposed method,original video frames are processed for all 11 selected angles of the CASIA B dataset and utilized to train two fine-tuned deep learning models such as Squeezenet and Efficientnet.Deep transfer learning was used to train both fine-tuned models on selected angles,yielding two new targeted models that were later used for feature engineering.Features are extracted from the deep layer of both fine-tuned models and fused into one vector using MCCA.An improved manta ray foraging optimization algorithm is also proposed to select the best features from the fused feature matrix and classified using a narrow neural network classifier.The experimental process was conducted on all 11 angles of the large multi-view gait dataset(CASIA B)dataset and obtained improved accuracy than the state-of-the-art techniques.Moreover,a detailed confidence interval based analysis also shows the effectiveness of the proposed architecture for HGR.
基金supported by“Human Resources Program in Energy Technology”of the Korea Institute of Energy Technology Evaluation and Planning(KETEP)granted financial resources from the Ministry of Trade,Industry&Energy,Republic of Korea.(No.20204010600090).
文摘Gait recognition is an active research area that uses a walking theme to identify the subject correctly.Human Gait Recognition(HGR)is performed without any cooperation from the individual.However,in practice,it remains a challenging task under diverse walking sequences due to the covariant factors such as normal walking and walking with wearing a coat.Researchers,over the years,have worked on successfully identifying subjects using different techniques,but there is still room for improvement in accuracy due to these covariant factors.This paper proposes an automated model-free framework for human gait recognition in this article.There are a few critical steps in the proposed method.Firstly,optical flow-based motion region esti-mation and dynamic coordinates-based cropping are performed.The second step involves training a fine-tuned pre-trained MobileNetV2 model on both original and optical flow cropped frames;the training has been conducted using static hyperparameters.The third step proposed a fusion technique known as normal distribution serially fusion.In the fourth step,a better optimization algorithm is applied to select the best features,which are then classified using a Bi-Layered neural network.Three publicly available datasets,CASIA A,CASIA B,and CASIA C,were used in the experimental process and obtained average accuracies of 99.6%,91.6%,and 95.02%,respectively.The proposed framework has achieved improved accuracy compared to the other methods.
基金This study was supported by the grants of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute(KHIDI),funded by the Ministry of Health&Welfare(HI18C1216)and the Soonchunhyang University Research Fund.
文摘Background—Human Gait Recognition(HGR)is an approach based on biometric and is being widely used for surveillance.HGR is adopted by researchers for the past several decades.Several factors are there that affect the system performance such as the walking variation due to clothes,a person carrying some luggage,variations in the view angle.Proposed—In this work,a new method is introduced to overcome different problems of HGR.A hybrid method is proposed or efficient HGR using deep learning and selection of best features.Four major steps are involved in this work-preprocessing of the video frames,manipulation of the pre-trained CNN model VGG-16 for the computation of the features,removing redundant features extracted from the CNN model,and classification.In the reduction of irrelevant features Principal Score and Kurtosis based approach is proposed named PSbK.After that,the features of PSbK are fused in one materix.Finally,this fused vector is fed to the One against All Multi Support Vector Machine(OAMSVM)classifier for the final results.Results—The system is evaluated by utilizing the CASIA B database and six angles 00◦,18◦,36◦,54◦,72◦,and 90◦are used and attained the accuracy of 95.80%,96.0%,95.90%,96.20%,95.60%,and 95.50%,respectively.Conclusion—The comparison with recent methods show the proposed method work better.
基金This research was supported by a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute(KHIDI),funded by the Ministry of Health&Welfare,Republic of Korea(grant number:HI21C1831)the Soonchunhyang University Research Fund.
文摘Human personality assessment using gait pattern recognition is one of the most recent and exciting research domains.Gait is a person’s identity that can reflect reliable information about his mood,emotions,and substantial personality traits under scrutiny.This research focuses on recognizing key personality traits,including neuroticism,extraversion,openness to experience,agreeableness,and conscientiousness,in line with the bigfive model of personality.We inferred personality traits based on the gait pattern recognition of individuals utilizing built-in smartphone sensors.For experimentation,we collected a novel dataset of 22 participants using an android application and further segmented it into six data chunks for a critical evaluation.After data pre-processing,we extracted selected features from each data segment and then applied four multiclass machine learning algorithms for training and classifying the dataset corresponding to the users’Big-Five Personality Traits Profiles(BFPT).Experimental results and performance evaluation of the classifiers revealed the efficacy of the proposed scheme for all big-five traits.
基金National Natural Science Foundation of China ( No.60873179)Shenzhen Technology Fundamental Research Project, China ( No.JC200903180630A)Doctoral Program Foundation of Institutions of Higher Education of China ( No.20090121110032)
文摘Gait representation is an important issue in gait recognition. A simple yet efficient approach, called Interframe Variation Vector (IW), is proposed. IW considers the spatiotemporal motion characteristic of gait, and uses the shape variation information between successive frames to represent gait signature. Different from other features, IVV rather than condenses a gait sequence into single image resulting in spatial sequence lost; it records the whole moving process in an IVV sequence. IVV can encode whole essential features of gait and preserve all the movements of limbs. Experimental results show that the proposed gait representation has a promising recognition performance.
基金supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.2022R1F1A1063134)the MSIT (Ministry of Science and ICT),Korea,under the ITRC (Information Technology Research Center)Support Program (IITP-2022-2018-0-01799)supervised by the IITP (Institute for Information&communications Technology Planning&Evaluation).
文摘Intelligent vision-based surveillance systems are designed to deal with the gigantic volume of videos captured in a particular environment to perform the interpretation of scenes in form of detection,tracking,monitoring,behavioral analysis,and retrievals.In addition to that,another evolving way of surveillance systems in a particular environment is human gait-based surveillance.In the existing research,several methodological frameworks are designed to use deep learning and traditional methods,nevertheless,the accuracies of these methods drop substantially when they are subjected to covariate conditions.These covariate variables disrupt the gait features and hence the recognition of subjects becomes difficult.To handle these issues,a region-based triplet-branch Convolutional Neural Network(CNN)is proposed in this research that is focused on different parts of the human Gait Energy Image(GEI)including the head,legs,and body separately to classify the subjects,and later on,the final identification of subjects is decided by probability-based majority voting criteria.Moreover,to enhance the feature extraction and draw the discriminative features,we have added soft attention layers on each branch to generate the soft attention maps.The proposed model is validated on the CASIA-B database and findings indicate that part-based learning through triplet-branch CNN shows good performance of 72.98%under covariate conditions as well as also outperforms single-branch CNN models.
文摘Human gait is a complex phenomenon involving the motion of various parts of the body simultaneously in a 3 dimensional space. Dynamics of different parts of the body translate its center of gravity from one point to another in the most efficient way. Body dynamics as well as static parameters of different body parts contribute to gait recognition. Studies have been performed to assess the discriminatory power of static and dynamic features. The current research literature, however, lacks the work on the comparative significance of dynamic features from different parts of the body. This paper sheds some light on the recognition performance of dynamic features extracted from different parts of human body in an appearance based set up.
文摘Gait is an essential biomedical feature that distinguishes individuals through walking.This feature automatically stimulates the need for remote human recognition in security-sensitive visual monitoring applications.However,there is still a lack of sufficient accuracy of gait recognition at night,in addition to taking some critical factors that affect the performances of the recognition algorithm.Therefore,a novel approach is proposed to automatically identify individuals from thermal infrared(TIR)images according to their gaits captured at night.This approach uses a new night gait network(NGaitNet)based on similarity deep convolutional neural networks(CNNs)method to enhance gait recognition at night.First,the TIR image is represented via personal movements and enhanced body skeleton segments.Then,the state-space method with a Hough transform is used to extract gait features to obtain skeletal joints′angles.These features are trained to identify the most discriminating gait patterns that indicate a change in human identity.To verify the proposed method,the experimental results are performed by using learning and validation curves via being connected by the Visdom website.The proposed thermal infrared imaging night gait recognition(TIRNGaitNet)approach has achieved the highest gait recognition accuracy rates(99.5%,97.0%),especially under normal walking conditions on the Chinese Academy of Sciences Institute of Automation infrared night gait dataset(CASIA C)and Donghua University thermal infrared night gait database(DHU night gait dataset).On the same dataset,the results of the TIRNGaitNet approach provide the record scores of(98.0%,87.0%)under the slow walking condition and(94.0%,86.0%)for the quick walking condition.
基金The Graduate Education Steering Committee of National Engineering Professional Degree (2016-ZX-064)Natural Science Foundation of Tianjin of China (16JCYBJC 15400)
文摘Gait energy image(GEI)is composed of static body silhouette and dynamic frequency information of human gait.To achieve fast and efficient gait recognition,combined with the accurate description of the information of details and directions in image by Curvelet transform,a gait recognition method using GEI and Curvelet(GEIC)is presented.Firstly,to gain the gait energy images,the gait cycle is selected according to the aspect ratio.Secondly,Curvelet energy coefficients of the GEI,which are used as gait feature vector,are extracted by Curvelet transform in different scales and different directions.Finally,the gait recognition is accomplished by the K nearest neighbor(KNN)classifier.The experimental results demonstrate that GEIC performs well on CASIA(B)database,with the average accuracy of 86.83%.Compared with GEI+KPCA,GEI+W(2D)2PCA and GEI+(2D)~2PCA,the algorithm GEIC achieves better robustness in the condition of the person wearing or packaging.
文摘Human Gait recognition is emerging as a supportive biometric technique in recent years that identifies the people through the way they walk. The gait recognition in model free approaches faces the challenges like speed variation, cloth variation, illumination changes and view angle variations which result in the reduced recognition rate. The proposed algorithm selected the exhaustive angles from head to toe of a person, and also height and width of the same subject. The experiments were conducted using silhouettes with view angle variation, and cloth variation. The recognition rate is improved to the extent of 91% using Support vector machine classifier. The proposed method is evaluated using CASIA Gait Dataset B (The institute of Automation, ChineseAcademy of Sciences), China. Experimental results demonstrate that the proposed technique shows promising results using state of the art classifiers.
文摘The results of image preprocessing may directly affect gait feature extraction in the area of gait recognition. Due to the influence of light, shelter and other external factors of gait image, some problems such as loss of information,image shadows, and improper threshold of image preprocessing may occur. In order to solve these problems, an image preprocessing method of gait recognition is proposed. Firstly, background image is extracted by background modeling, secondly, the target profile is extracted by the direct difference method; thirdly, the shadow elimination based on the HSV color model is carried out on the target profile map; Finally, the complete target profile is obtained by threshold segmentation. Experimental results on CASIA_A database demonstrate that this proposed method is quite effective on both target profile extraction and proportion comparison with the real area.
文摘Human gait is one of the unobtrusive behavioral biometrics that has been extensively studied for various commercial and government applications.Biometric security,medical rehabilitation,virtual reality,and autonomous driving cars are some of the fields of study that rely on accurate gait recognition.While majority of studies have been focused on achieving very high recognition performance on a specific dataset,different issues arise in the real-world applications of this technology.This research is one of the first to evaluate the effects of changing walking speeds and directions on gait recognition rates under various walking conditions.Dataset was collected using the KINECT sensor.To draw an overall conclusion about the effects of walking speed and di-rection to the sensor,we define distance features and angle features.Furthermore,we propose two feature fusion methods for person recognition.Results of the study provide insights into how walking speeds and walking di-rections to the KINECT sensor influence the accuracy of gait recognition.