Enhancing Human Action Recognition with Adaptive Hybrid Deep Attentive Networks and Archerfish Optimization

In recent years,wearable devices-based Human Activity Recognition(HAR)models have received significant attention.Previously developed HAR models use hand-crafted features to recognize human activities,leading to the extraction of basic features.The images captured by wearable sensors contain advanced features,allowing them to be analyzed by deep learning algorithms to enhance the detection and recognition of human actions.Poor lighting and limited sensor capabilities can impact data quality,making the recognition of human actions a challenging task.The unimodal-based HAR approaches are not suitable in a real-time environment.Therefore,an updated HAR model is developed using multiple types of data and an advanced deep-learning approach.Firstly,the required signals and sensor data are accumulated from the standard databases.From these signals,the wave features are retrieved.Then the extracted wave features and sensor data are given as the input to recognize the human activity.An Adaptive Hybrid Deep Attentive Network(AHDAN)is developed by incorporating a“1D Convolutional Neural Network(1DCNN)”with a“Gated Recurrent Unit(GRU)”for the human activity recognition process.Additionally,the Enhanced Archerfish Hunting Optimizer(EAHO)is suggested to fine-tune the network parameters for enhancing the recognition process.An experimental evaluation is performed on various deep learning networks and heuristic algorithms to confirm the effectiveness of the proposed HAR model.The EAHO-based HAR model outperforms traditional deep learning networks with an accuracy of 95.36,95.25 for recall,95.48 for specificity,and 95.47 for precision,respectively.The result proved that the developed model is effective in recognizing human action by taking less time.Additionally,it reduces the computation complexity and overfitting issue through using an optimization approach.

A Novel Human Action Recognition Algorithm Based on Decision Level Multi-Feature Fusion

In order to take advantage of the logical structure of video sequences and improve the recognition accuracy of the human action, a novel hybrid human action detection method based on three descriptors and decision level fusion is proposed. Firstly, the minimal 3D space region of human action region is detected by combining frame difference method and Vi BE algorithm, and the three-dimensional histogram of oriented gradient(HOG3D) is extracted. At the same time, the characteristics of global descriptors based on frequency domain filtering(FDF) and the local descriptors based on spatial-temporal interest points(STIP) are extracted. Principal component analysis(PCA) is implemented to reduce the dimension of the gradient histogram and the global descriptor, and bag of words(BoW) model is applied to describe the local descriptors based on STIP. Finally, a linear support vector machine(SVM) is used to create a new decision level fusion classifier. Some experiments are done to verify the performance of the multi-features, and the results show that they have good representation ability and generalization ability. Otherwise, the proposed scheme obtains very competitive results on the well-known datasets in terms of mean average precision.

Silhouettes Based Human Action Recognition in Video via Procrustes Analysis and Fisher Vector Coding

This paper proposes a framework for human action recognition based on procrustes analysis and Fisher vector coding(FVC).Firstly,we applied a pose feature extracted from silhouette image by employing Procrustes analysis and local preserving projection(LPP).Secondly,the extracted feature can preserve the discriminative shape information and local manifold structure of human pose and is invariant to translation,rotation and scaling.Finally,after the pose feature was extracted,a recognition framework based on FVC and multi-class supporting vector machine was employed to classify the human action.Experimental results on benchmarks demonstrate the effectiveness of the proposed method.

Human-Object Interaction Recognition Based on Modeling Context

This paper proposes a method to recognize human-object interactions by modeling context between human actions and interacted objects.Human-object interaction recognition is a challenging task due to severe occlusion between human and objects during the interacting process.Since that human actions and interacted objects provide strong context information,i.e.some actions are usually related to some specific objects,the accuracy of recognition is significantly improved for both of them.Through the proposed method,both global and local temporal features from skeleton sequences are extracted to model human actions.In the meantime,kernel features are utilized to describe interacted objects.Finally,all possible solutions from actions and objects are optimized by modeling the context between them.The results of experiments demonstrate the effectiveness of our method.

Combining Multi-scale Directed Depth Motion Maps and Log-Gabor Filters for Human Action Recognition

Recognition of the human actions by computer vision has become an active research area in recent years. Due to the speed and the high similarity of the actions, the current algorithms cannot get high recognition rate. A new recognition method of the human action is proposed with the multi-scale directed depth motion maps(MsdDMMs) and Log-Gabor filters. According to the difference between the speed and time order of an action, MsdDMMs is proposed under the energy framework. Meanwhile, Log-Gabor is utilized to describe the texture details of MsdDMMs for the motion characteristics. It can easily satisfy both the texture characterization and the visual features of human eye. Furthermore, the collaborative representation is employed as action recognition by the classification. Experimental results show that the proposed algorithm, which is applied in the MSRAction3 D dataset and MSRGesture3 D dataset, can achieve the accuracy of 95.79% and 96.43% respectively. It also has higher accuracy than the existing algorithms, such as super normal vector(SNV), hierarchical recurrent neural network(Hierarchical RNN).

Recognition of Human Actions through Speech or Voice Using Machine Learning Techniques

The development of artificial intelligence(AI)and smart home technologies has driven the need for speech recognition-based solutions.This demand stems from the quest for more intuitive and natural interaction between users and smart devices in their homes.Speech recognition allows users to control devices and perform everyday actions through spoken commands,eliminating the need for physical interfaces or touch screens and enabling specific tasks such as turning on or off the light,heating,or lowering the blinds.The purpose of this study is to develop a speech-based classification model for recognizing human actions in the smart home.It seeks to demonstrate the effectiveness and feasibility of using machine learning techniques in predicting categories,subcategories,and actions from sentences.A dataset labeled with relevant information about categories,subcategories,and actions related to human actions in the smart home is used.The methodology uses machine learning techniques implemented in Python,extracting features using CountVectorizer to convert sentences into numerical representations.The results show that the classification model is able to accurately predict categories,subcategories,and actions based on sentences,with 82.99%accuracy for category,76.19%accuracy for subcategory,and 90.28%accuracy for action.The study concludes that using machine learning techniques is effective for recognizing and classifying human actions in the smart home,supporting its feasibility in various scenarios and opening new possibilities for advanced natural language processing systems in the field of AI and smart homes.

Two-Stream Deep Learning Architecture-Based Human Action Recognition

Human action recognition(HAR)based on Artificial intelligence reasoning is the most important research area in computer vision.Big breakthroughs in this field have been observed in the last few years;additionally,the interest in research in this field is evolving,such as understanding of actions and scenes,studying human joints,and human posture recognition.Many HAR techniques are introduced in the literature.Nonetheless,the challenge of redundant and irrelevant features reduces recognition accuracy.They also faced a few other challenges,such as differing perspectives,environmental conditions,and temporal variations,among others.In this work,a deep learning and improved whale optimization algorithm based framework is proposed for HAR.The proposed framework consists of a few core stages i.e.,frames initial preprocessing,fine-tuned pre-trained deep learning models through transfer learning(TL),features fusion using modified serial based approach,and improved whale optimization based best features selection for final classification.Two pre-trained deep learning models such as InceptionV3 and Resnet101 are fine-tuned and TL is employed to train on action recognition datasets.The fusion process increases the length of feature vectors;therefore,improved whale optimization algorithm is proposed and selects the best features.The best selected features are finally classified usingmachine learning(ML)classifiers.Four publicly accessible datasets such as Ut-interaction,Hollywood,Free Viewpoint Action Recognition usingMotion History Volumes(IXMAS),and centre of computer vision(UCF)Sports,are employed and achieved the testing accuracy of 100%,99.9%,99.1%,and 100%respectively.Comparison with state of the art techniques(SOTA),the proposed method showed the improved accuracy.

SlowFast Based Real-Time Human Motion Recognition with Action Localization

Artificial intelligence is increasingly being applied in the field of video analysis,particularly in the area of public safety where video surveillance equipment such as closed-circuit television(CCTV)is used and automated analysis of video information is required.However,various issues such as data size limitations and low processing speeds make real-time extraction of video data challenging.Video analysis technology applies object classification,detection,and relationship analysis to continuous 2D frame data,and the various meanings within the video are thus analyzed based on the extracted basic data.Motion recognition is key in this analysis.Motion recognition is a challenging field that analyzes human body movements,requiring the interpretation of complex movements of human joints and the relationships between various objects.The deep learning-based human skeleton detection algorithm is a representative motion recognition algorithm.Recently,motion analysis models such as the SlowFast network algorithm,have also been developed with excellent performance.However,these models do not operate properly in most wide-angle video environments outdoors,displaying low response speed,as expected from motion classification extraction in environments associated with high-resolution images.The proposed method achieves high level of extraction and accuracy by improving SlowFast’s input data preprocessing and data structure methods.The input data are preprocessed through object tracking and background removal using YOLO and DeepSORT.A higher performance than that of a single model is achieved by improving the existing SlowFast’s data structure into a frame unit structure.Based on the confusion matrix,accuracies of 70.16%and 70.74%were obtained for the existing SlowFast and proposed model,respectively,indicating a 0.58%increase in accuracy.Comparing detection,based on behavioral classification,the existing SlowFast detected 2,341,164 cases,whereas the proposed model detected 3,119,323 cases,which is an increase of 33.23%.

基于毫米波雷达三维点云的人体动作识别数据集与方法

毫米波雷达凭借其出色的环境适应性、高分辨率和隐私保护等优势,在智能家居、智慧养老和安防监控等领域具有广泛的应用前景。毫米波雷达三维点云是一种重要的空间数据表达形式,对于人体行为姿态识别具有极大的价值。然而,由于毫米波雷达点云具有强稀疏性,给精准快速识别人体动作带来了巨大的挑战。针对这一问题,该文公开了一个毫米波雷达人体动作三维点云数据集mmWave-3DPCHM-1.0,并提出了相应的数据处理方法和人体动作识别模型。该数据集由TI公司的IWR1443-ISK和Vayyar公司的vBlu射频成像模组分别采集,包括常见的12种人体动作,如走路、挥手、站立和跌倒等。在网络模型方面,该文将边缘卷积(EdgeConv)与Transformer相结合,提出了一种处理长时序三维点云的网络模型,即Point EdgeConv and Transformer(PETer)网络。该网络通过边缘卷积对三维点云逐帧创建局部有向邻域图,以提取单帧点云的空间几何特征,并通过堆叠多个编码器的Transformer模块,提取多帧点云之间的时序关系。实验结果表明,所提出的PETer网络在所构建的TI数据集和Vayyar数据集上的平均识别准确率分别达到98.77%和99.51%,比传统最优的基线网络模型提高了大约5%,且网络规模仅为1.09 M,适于在存储受限的边缘设备上部署。

基于多尺度滑动窗口自注意力网络的交互动作识别

近年来交互式动作识别逐渐应用于人机交互领域,具有很强的经济价值。在之前的工作中,主要考虑了实体之间固定的物理连接,这种基于先验知识的方法忽略了实体的多样性。在此基础上,推出了一种基于滑动窗口分割考虑不同实体间交互的方法,虽然弥补了之前的实体多样性问题,却忽视了多尺度时空和实体的多尺度交互。提出了一种基于多尺度滑动窗口的自注意力Transformer网络方法,将输入数据按照多尺度窗口划分,并将划分好的数据进行融合,在自注意力网络中得到最终的分类结果。实验证明,按照多尺度窗口的划分方法,与其他方法相比具有更好的识别效果,并且在两种数据集上都达到了更好的识别效果。

HARTIV:Human Activity Recognition Using Temporal Information in Videos

Nowadays,the most challenging and important problem of computer vision is to detect human activities and recognize the same with temporal information from video data.The video datasets are generated using cameras available in various devices that can be in a static or dynamic position and are referred to as untrimmed videos.Smarter monitoring is a historical necessity in which commonly occurring,regular,and out-of-the-ordinary activities can be automatically identified using intelligence systems and computer vision technology.In a long video,human activity may be present anywhere in the video.There can be a single ormultiple human activities present in such videos.This paper presents a deep learning-based methodology to identify the locally present human activities in the video sequences captured by a single wide-view camera in a sports environment.The recognition process is split into four parts:firstly,the video is divided into different set of frames,then the human body part in a sequence of frames is identified,next process is to identify the human activity using a convolutional neural network and finally the time information of the observed postures for each activity is determined with the help of a deep learning algorithm.The proposed approach has been tested on two different sports datasets including ActivityNet and THUMOS.Three sports activities like swimming,cricket bowling and high jump have been considered in this paper and classified with the temporal information i.e.,the start and end time for every activity present in the video.The convolutional neural network and long short-term memory are used for feature extraction of temporal action recognition from video data of sports activity.The outcomes show that the proposed method for activity recognition in the sports domain outperforms the existing methods.

Deep learning and transfer learning for device-free human activity recognition:A survey

Device-free activity recognition plays a crucial role in smart building,security,and human–computer interaction,which shows its strength in its convenience and cost-efficiency.Traditional machine learning has made significant progress by heuristic hand-crafted features and statistical models,but it suffers from the limitation of manual feature design.Deep learning overcomes such issues by automatic high-level feature extraction,but its performance degrades due to the requirement of massive annotated data and cross-site issues.To deal with these problems,transfer learning helps to transfer knowledge from existing datasets while dealing with the negative effect of background dynamics.This paper surveys the recent progress of deep learning and transfer learning for device-free activity recognition.We begin with the motivation of deep learning and transfer learning,and then introduce the major sensor modalities.Then the deep and transfer learning techniques for device-free human activity recognition are introduced.Eventually,insights on existing works and grand challenges are summarized and presented to promote future research.

Research on Human Body Behavior Recognition Based on Vision

This paper proposes the research on human body behavior recognition based on vision. Behavior based on high-level human structure can describe behavior more accurately, but it is dif？ cult to extract the behavioral characteristics while often relying on the accuracy of the human pose estimation. Moving object extraction of the moving targets in video analysis as the main content, research based on the image sequence robust, fast moving target extraction, motion estimation and target description algorithm, and the correlation between motion detection is to use frame, frame by comparing the difference between for change and not change area. The model is proposed based on the probability theory, and the future research will be focused on the simulation.

基于双流CNN-BiLSTM的毫米波雷达人体动作识别方法

目前基于雷达的人体动作识别方法,大多是先对人体动作的回波信号进行多维快速傅里叶变换(FFT)得到距离、多普勒和角度等信息,构造各种数据谱图后再输入到神经网络中进行分类识别,数据预处理过程较为复杂。提出了一种双流卷积神经网络(CNN)与双向长短时记忆网络(BiLSTM)串联的毫米波雷达人体动作识别方法。首先对原始的雷达回波信号复数采样数据(I/Q)进行帧差处理,以消除静态干扰,并将其转换为幅度/相位(A/P)的数据格式;然后将帧差后的I/Q和A/P数据分别输入单流的CNN-BiLSTM网络,提取人体动作的空间和时间特征,最后进行双流网络的融合以增强特征的交互性,提高识别准确率。实验结果表明,该方法数据预处理简单,并充分利用了动作数据的帧间相关性,模型收敛快,识别准确率可以达到99%,是一种快速有效的人体动作识别方法。

面向铁路行车调度员疲劳识别的动作检测

行车指挥调度是铁路运输的核心监控岗位,检测调度员的疲劳动作对保障铁路运营安全具有重要意义。为了能识别铁路行车调度员疲劳动作,降低安全生产风险,提出一种基于双向长短时记忆神经网络和支持向量机的自适应增强算法对调度员疲劳状态下的动作进行识别。首先,通过高分辨率网络(High-Resolution Net, HRNet)人体关键点检测模型,提取多个人体关键点及人体动作行为角度特征与长度比例特征。其次,搭建基于双向长短时记忆神经网络和支持向量机(Bi-directional Long Short-Term Memory-Support Vector Machine, BiLSTM-SVM)的动作识别模型,使用正交试验法对模型参数进行优化,并采用自适应增强算法(Adaboost, Adaptive Boosting)进一步提升疲劳动作识别。最后,基于调度仿真疲劳动作数据,对该模型的有效性进行验证。结果显示,该模型的精确率为0.97、准确率为0.96、召回率为0.96、F1分数为0.96。该模型提高了人体疲劳动作分类的准确率,为调度员疲劳检测提供了判断依据。

融合人体骨架和姿势信息特征的轻量级人体动作识别方法

针对人体动作识别任务中特征值选取不当导致识别率低、使用多模态数据导致训练成本高等问题,提出一种轻量级人体动作识别方法。首先使用OpenPose、PoseNet提取出人体骨架信息,使用BWT69CL传感器提取姿势信息;其次对数据进行预处理、特征融合,对人体动作进行深度学习分类识别;最后,为验证此方法的有效性,在公开数据集WISDM、UCIHAR、HASC和自建的人体动作数据集上进行实验验证,并使用改进的目标引导注意力机制(target-guided attention,TGA)–长短期记忆(long short term memory,LSTM)网络输出最终的分类结果。实验结果表明,在自建数据集下融合姿势和骨架特征达到99.87%准确率,相比于只使用姿势信息特征,识别准确率提高了约5.31个百分点;相比于只使用人体骨架特征,识别准确率提高了约1.87个百分点;在识别时间上相比于只使用姿势信息,识别时间降低了约29.73 s;相比于只使用人体骨架数据,识别时间降低了约9 s。使用该方法能及时有效地反映人体的运动意图,有助于提高人体动作和行为的识别准确率和训练效率。

一种融合注意力机制与3D双流卷积网络的动态头势识别方法

头势能够传递丰富的情绪和意图信息,属于重要的人机交互方式之一。然而,目前基于穿戴式传感器的头势识别方法虽然具有较高的识别率,但缺乏经济性和便捷性,而基于机器视觉的方法普遍存在准确率低、泛化性差、算力成本较高的问题,因此目前的头势识别方法仍难以部署于移动机器人。针对以上问题,提出了一种融合注意力机制与3D双流卷积的动态头势识别方法。该方法从动态头势视频帧中提取RGB信号和光流特征,在注意力机制的启发下,从通道域和空间域进行动作特征提取和增强,从而对关键特征进行准确提取,然后对特征进行融合与分类。实验结果表明,所提方法能够有效提取头势中关键的通道域和空间域信息,可显著提高头势识别的准确率及泛化能力,可在有限算力下实现较高的准确率与实时性。其后,将所提方法应用于助老机器人,在实际示范应用中进行了验证,结果表明本方法适于移动机器人等算力受限的移动机载计算平台。

基于度量学习的毫米波雷达少样本人体动作识别

针对目前基于毫米波雷达的人体动作识别方法普遍需要大量的样本数据且计算复杂度较高的问题,提出了一种基于度量学习的毫米波雷达少样本人体动作识别方法。该方法首先对采集到的人体动作回波信号进行背景帧差处理得到校准后的帧数据,然后对其进行二维傅里叶变换(2D-FFT)获得距离-多普勒图,再对距离-多普勒图基于速度维投影法进行逐帧拼接来构造微多普勒时频谱图(DTM),最后使用基于残差的度量学习原型网络对8类人体动作的微多普勒时频谱图进行训练验证,实现了人体不同动作的识别。实验结果表明,所提方法在只有30个训练样本的情况下,8类动作的平均识别准确率可达到99.05%。

基于多模态数据的人体行为识别方法研究综述

人体行为识别广泛应用于智能安防、自动驾驶和人机交互等领域。随着拍摄设备和传感器技术的发展,可获取用于人体行为识别的数据不再局限于RGB数据,还有深度、骨骼和红外等多模态数据。详细介绍了基于RGB和骨骼数据模态的人体行为识别任务中特征提取方法,包括基于手工标注和基于深度学习的方法。对于RGB数据模态,重点分析了基于双流卷积神经网络、3D卷积神经网络和混合网络的特征提取算法。对于骨骼数据模态,介绍了目前流行的单人和多人姿态评估算法;重点分析了基于卷积神经网络、循环神经网络和图卷积神经网络的分类算法;进一步全面展示了两种数据模态的通用数据集。此外,基于RGB和骨骼各自的数据结构特征,探讨了目前面临的挑战,最后对未来基于深度学习的人体行为识别方法的研究方向进行了展望。

基于显著性特征的多视角人体动作图像识别研究

为了应对多角度、多姿态特点的人体动作图像识别难的问题,研究一种基于显著性特征的多视角人体动作图像识别方法。通过显著性区域检测模块处理多视角人体动作图像,获取显著性区域序列并拼接成多视角显著性区域拼接图;利用VGG-Net网络从中提取其卷积特征图,输入到CA(上下注意力)模块中,将有利于人体动作识别的区域予以突出显示,并输出人体动作类别标签的概率值,实现多视角人体动作识别。实验结果表明,所提方法能够有效识别多视角人体动作,同时通过整合显著性区域检测和CA模块,可以显著提升其在多视角人体动作识别方面的应用效果。