Two-Stream Deep Learning Architecture-Based Human Action Recognition

Human action recognition(HAR)based on Artificial intelligence reasoning is the most important research area in computer vision.Big breakthroughs in this field have been observed in the last few years;additionally,the interest in research in this field is evolving,such as understanding of actions and scenes,studying human joints,and human posture recognition.Many HAR techniques are introduced in the literature.Nonetheless,the challenge of redundant and irrelevant features reduces recognition accuracy.They also faced a few other challenges,such as differing perspectives,environmental conditions,and temporal variations,among others.In this work,a deep learning and improved whale optimization algorithm based framework is proposed for HAR.The proposed framework consists of a few core stages i.e.,frames initial preprocessing,fine-tuned pre-trained deep learning models through transfer learning(TL),features fusion using modified serial based approach,and improved whale optimization based best features selection for final classification.Two pre-trained deep learning models such as InceptionV3 and Resnet101 are fine-tuned and TL is employed to train on action recognition datasets.The fusion process increases the length of feature vectors;therefore,improved whale optimization algorithm is proposed and selects the best features.The best selected features are finally classified usingmachine learning(ML)classifiers.Four publicly accessible datasets such as Ut-interaction,Hollywood,Free Viewpoint Action Recognition usingMotion History Volumes(IXMAS),and centre of computer vision(UCF)Sports,are employed and achieved the testing accuracy of 100%,99.9%,99.1%,and 100%respectively.Comparison with state of the art techniques(SOTA),the proposed method showed the improved accuracy.

Human Action Recognition Using Difference of Gaussian and Difference of Wavelet

Human Action Recognition(HAR)attempts to recognize the human action from images and videos.The major challenge in HAR is the design of an action descriptor that makes the HAR system robust for different environments.A novel action descriptor is proposed in this study,based on two independent spatial and spectral filters.The proposed descriptor uses a Difference of Gaussian(DoG)filter to extract scale-invariant features and a Difference of Wavelet(DoW)filter to extract spectral information.To create a composite feature vector for a particular test action picture,the Discriminant of Guassian(DoG)and Difference of Wavelet(DoW)features are combined.Linear Discriminant Analysis(LDA),a widely used dimensionality reduction technique,is also used to eliminate duplicate data.Finally,a closest neighbor method is used to classify the dataset.Weizmann and UCF 11 datasets were used to run extensive simulations of the suggested strategy,and the accuracy assessed after the simulations were run on Weizmann datasets for five-fold cross validation is shown to perform well.The average accuracy of DoG+DoW is observed as 83.6635%while the average accuracy of Discrinanat of Guassian(DoG)and Difference of Wavelet(DoW)is observed as 80.2312%and 77.4215%,respectively.The average accuracy measured after the simulation of proposed methods over UCF 11 action dataset for five-fold cross validation DoG+DoW is observed as 62.5231%while the average accuracy of Difference of Guassian(DoG)and Difference of Wavelet(DoW)is observed as 60.3214%and 58.1247%,respectively.From the above accuracy observations,the accuracy of Weizmann is high compared to the accuracy of UCF 11,hence verifying the effectiveness in the improvisation of recognition accuracy.

Recognition of Human Actions through Speech or Voice Using Machine Learning Techniques

The development of artificial intelligence(AI)and smart home technologies has driven the need for speech recognition-based solutions.This demand stems from the quest for more intuitive and natural interaction between users and smart devices in their homes.Speech recognition allows users to control devices and perform everyday actions through spoken commands,eliminating the need for physical interfaces or touch screens and enabling specific tasks such as turning on or off the light,heating,or lowering the blinds.The purpose of this study is to develop a speech-based classification model for recognizing human actions in the smart home.It seeks to demonstrate the effectiveness and feasibility of using machine learning techniques in predicting categories,subcategories,and actions from sentences.A dataset labeled with relevant information about categories,subcategories,and actions related to human actions in the smart home is used.The methodology uses machine learning techniques implemented in Python,extracting features using CountVectorizer to convert sentences into numerical representations.The results show that the classification model is able to accurately predict categories,subcategories,and actions based on sentences,with 82.99%accuracy for category,76.19%accuracy for subcategory,and 90.28%accuracy for action.The study concludes that using machine learning techniques is effective for recognizing and classifying human actions in the smart home,supporting its feasibility in various scenarios and opening new possibilities for advanced natural language processing systems in the field of AI and smart homes.

SlowFast Based Real-Time Human Motion Recognition with Action Localization

Artificial intelligence is increasingly being applied in the field of video analysis,particularly in the area of public safety where video surveillance equipment such as closed-circuit television(CCTV)is used and automated analysis of video information is required.However,various issues such as data size limitations and low processing speeds make real-time extraction of video data challenging.Video analysis technology applies object classification,detection,and relationship analysis to continuous 2D frame data,and the various meanings within the video are thus analyzed based on the extracted basic data.Motion recognition is key in this analysis.Motion recognition is a challenging field that analyzes human body movements,requiring the interpretation of complex movements of human joints and the relationships between various objects.The deep learning-based human skeleton detection algorithm is a representative motion recognition algorithm.Recently,motion analysis models such as the SlowFast network algorithm,have also been developed with excellent performance.However,these models do not operate properly in most wide-angle video environments outdoors,displaying low response speed,as expected from motion classification extraction in environments associated with high-resolution images.The proposed method achieves high level of extraction and accuracy by improving SlowFast’s input data preprocessing and data structure methods.The input data are preprocessed through object tracking and background removal using YOLO and DeepSORT.A higher performance than that of a single model is achieved by improving the existing SlowFast’s data structure into a frame unit structure.Based on the confusion matrix,accuracies of 70.16%and 70.74%were obtained for the existing SlowFast and proposed model,respectively,indicating a 0.58%increase in accuracy.Comparing detection,based on behavioral classification,the existing SlowFast detected 2,341,164 cases,whereas the proposed model detected 3,119,323 cases,which is an increase of 33.23%.

Distribution of action movements （DAM）： a descriptor for human action recognition

Human action recognition from skeletal data is an important and active area of research in which the state of the art has not yet achieved near-perfect accuracy on many well- known datasets. In this paper, we introduce the Distribution of Action Movements Descriptor, a novel action descriptor based on the distribution of the directions of the motions of the joints between frames, over the set of all possible mo- tions in the dataset. The descriptor is computed as a normal- ized histogram over a set of representative directions of the joints, which are in turn obtained via clustering. While the descriptor is global in the sense that it represents the overall distribution of movement directions of an action, it is able to partially retain its temporal structure by applying a window- ing scheme. The descriptor, together with performs several state-of-the-art known datasets. a standard classifier, out- techniques on many well-

A Survey of Human Action Recognition and Posture Prediction

Human action recognition and posture prediction aim to recognize and predict respectively the action and postures of persons in videos.They are both active research topics in computer vision community,which have attracted considerable attention from academia and industry.They are also the precondition for intelligent interaction and human-computer cooperation,and they help the machine perceive the external environment.In the past decade,tremendous progress has been made in the field,especially after the emergence of deep learning technologies.Hence,it is necessary to make a comprehensive review of recent developments.In this paper,firstly,we attempt to present the background,and then discuss research progresses.Secondly,we introduce datasets,various typical feature representation methods,and explore advanced human action recognition and posture prediction algorithms.Finally,facing the challenges in the field,this paper puts forward the research focus,and introduces the importance of action recognition and posture prediction by taking interactive cognition in self-driving vehicle as an example.

Intelligent 3D garment system of the human body based on deep spiking neural network

Background Intelligent garments,a burgeoning class of wearable devices,have extensive applications in domains such as sports training and medical rehabilitation.Nonetheless,existing research in the smart wearables domain predominantly emphasizes sensor functionality and quantity,often skipping crucial aspects related to user experience and interaction.Methods To address this gap,this study introduces a novel real-time 3D interactive system based on intelligent garments.The system utilizes lightweight sensor modules to collect human motion data and introduces a dual-stream fusion network based on pulsed neural units to classify and recognize human movements,thereby achieving real-time interaction between users and sensors.Additionally,the system incorporates 3D human visualization functionality,which visualizes sensor data and recognizes human actions as 3D models in real time,providing accurate and comprehensive visual feedback to help users better understand and analyze the details and features of human motion.This system has significant potential for applications in motion detection,medical monitoring,virtual reality,and other fields.The accurate classification of human actions contributes to the development of personalized training plans and injury prevention strategies.Conclusions This study has substantial implications in the domains of intelligent garments,human motion monitoring,and digital twin visualization.The advancement of this system is expected to propel the progress of wearable technology and foster a deeper comprehension of human motion.

Human action recognition using a convolutional neural network based on skeleton heatmaps from two-stage pose estimation

Human action recognition based on skeleton information has been extensively used in various areas,such as human-computer interaction.In this paper,we extracted human skeleton data by constructing a two-stage human pose estimation model,which combined the improved single shot detector(SSD)algorithm with convolutional pose machines(CPM)to obtain human skeleton heatmaps.The backbone of the SSD algorithm was replaced with ResNet,which can characterize images effectively.In addition,we designed multiscale transformation rules for CPM to fuse the information of different scales and a convolutional neural network for the classification of the skeleton keypoints heatmaps to complete action recognition.Indoor and outdoor experiments were conducted on the Caster Moma mobile robot platform,and without an external remote control,the real-time movement of the robot was controlled by the leader through command actions.

Smart Deep Learning Based Human Behaviour Classification for Video Surveillance

Real-time video surveillance system is commonly employed to aid security professionals in preventing crimes.The use of deep learning(DL)technologies has transformed real-time video surveillance into smart video surveillance systems that automate human behavior classification.The recognition of events in the surveillance videos is considered a hot research topic in the field of computer science and it is gaining significant attention.Human action recognition(HAR)is treated as a crucial issue in several applications areas and smart video surveillance to improve the security level.The advancements of the DL models help to accomplish improved recognition performance.In this view,this paper presents a smart deep-based human behavior classification(SDL-HBC)model for real-time video surveillance.The proposed SDL-HBC model majorly aims to employ an adaptive median filtering(AMF)based pre-processing to reduce the noise content.Also,the capsule network(CapsNet)model is utilized for the extraction of feature vectors and the hyperparameter tuning of the CapsNet model takes place utilizing the Adam optimizer.Finally,the differential evolution(DE)with stacked autoencoder(SAE)model is applied for the classification of human activities in the intelligent video surveillance system.The performance validation of the SDL-HBC technique takes place using two benchmark datasets such as the KTH dataset.The experimental outcomes reported the enhanced recognition performance of the SDL-HBC technique over the recent state of art approaches with maximum accuracy of 0.9922.

Effective Frameworks Based on Infinite Mixture Model for Real-World Applications

Interest in automated data classification and identification systems has increased over the past years in conjunction with the high demand for artificial intelligence and security applications.In particular,recognizing human activities with accurate results have become a topic of high interest.Although the current tools have reached remarkable successes,it is still a challenging problem due to various uncontrolled environments and conditions.In this paper two statistical frameworks based on nonparametric hierarchical Bayesian models and Gamma distribution are proposed to solve some realworld applications.In particular,two nonparametric hierarchical Bayesian models based on Dirichlet process and Pitman-Yor process are developed.These models are then applied to address the problem of modelling grouped data where observations are organized into groups and these groups are statistically linked by sharing mixture components.The choice of the Gamma mixtures is motivated by its flexibility for modelling heavy-tailed distributions.In addition,deploying the Dirichlet process prior is justified by its advantage of automatically finding the right number of components and providing nice properties.Moreover,a learning step via variational Bayesian setting is presented in a flexible way.The priors over the parameters are selected appropriately and the posteriors are approximated effectively in a closed form.Experimental results based on a real-life applications that concerns texture classification and human actions recognition show the capabilities and effectiveness of the proposed framework.

Sensors-Based Ambient Assistant Living via E-Monitoring Technology

Independent human living systems require smart,intelligent,and sustainable online monitoring so that an individual can be assisted timely.Apart from ambient assisted living,the task of monitoring human activities plays an important role in different fields including virtual reality,surveillance security,and human interaction with robots.Such systems have been developed in the past with the use of various wearable inertial sensors and depth cameras to capture the human actions.In this paper,we propose multiple methods such as random occupancy pattern,spatio temporal cloud,waypoint trajectory,Hilbert transform,Walsh Hadamard transform and bone pair descriptors to extract optimal features corresponding to different human actions.These features sets are then normalized using min-max normalization and optimized using the Fuzzy optimization method.Finally,the Masi entropy classifier is applied for action recognition and classification.Experiments have been performed on three challenging datasets,namely,UTDMHAD,50 Salad,and CMU-MMAC.During experimental evaluation,the proposed novel approach of recognizing human actions has achieved an accuracy rate of 90.1%with UTD-MHAD dataset,90.6%with 50 Salad dataset,and 89.5%with CMU-MMAC dataset.Hence experimental results validated the proposed system.