A Survey on Deep Learning-Based 2D Human Pose Estimation Models

In this article,a comprehensive survey of deep learning-based(DLbased)human pose estimation(HPE)that can help researchers in the domain of computer vision is presented.HPE is among the fastest-growing research domains of computer vision and is used in solving several problems for human endeavours.After the detailed introduction,three different human body modes followed by the main stages of HPE and two pipelines of twodimensional(2D)HPE are presented.The details of the four components of HPE are also presented.The keypoints output format of two popular 2D HPE datasets and the most cited DL-based HPE articles from the year of breakthrough are both shown in tabular form.This study intends to highlight the limitations of published reviews and surveys respecting presenting a systematic review of the current DL-based solution to the 2D HPE model.Furthermore,a detailed and meaningful survey that will guide new and existing researchers on DL-based 2D HPE models is achieved.Finally,some future research directions in the field of HPE,such as limited data on disabled persons and multi-training DL-based models,are revealed to encourage researchers and promote the growth of HPE research.

Squirrel Search Optimization with Deep Convolutional Neural Network for Human Pose Estimation

Human pose estimation(HPE)is a procedure for determining the structure of the body pose and it is considered a challenging issue in the computer vision(CV)communities.HPE finds its applications in several fields namely activity recognition and human-computer interface.Despite the benefits of HPE,it is still a challenging process due to the variations in visual appearances,lighting,occlusions,dimensionality,etc.To resolve these issues,this paper presents a squirrel search optimization with a deep convolutional neural network for HPE(SSDCNN-HPE)technique.The major intention of the SSDCNN-HPE technique is to identify the human pose accurately and efficiently.Primarily,the video frame conversion process is performed and pre-processing takes place via bilateral filtering-based noise removal process.Then,the EfficientNet model is applied to identify the body points of a person with no problem constraints.Besides,the hyperparameter tuning of the EfficientNet model takes place by the use of the squirrel search algorithm(SSA).In the final stage,the multiclass support vector machine(M-SVM)technique was utilized for the identification and classification of human poses.The design of bilateral filtering followed by SSA based EfficientNetmodel for HPE depicts the novelty of the work.To demonstrate the enhanced outcomes of the SSDCNN-HPE approach,a series of simulations are executed.The experimental results reported the betterment of the SSDCNN-HPE system over the recent existing techniques in terms of different measures.

Overview of 3D Human Pose Estimation

3D human pose estimation is a major focus area in the field of computer vision,which plays an important role in practical applications.This article summarizes the framework and research progress related to the estimation of monocular RGB images and videos.An overall perspective ofmethods integrated with deep learning is introduced.Novel image-based and video-based inputs are proposed as the analysis framework.From this viewpoint,common problems are discussed.The diversity of human postures usually leads to problems such as occlusion and ambiguity,and the lack of training datasets often results in poor generalization ability of the model.Regression methods are crucial for solving such problems.Considering image-based input,the multi-view method is commonly used to solve occlusion problems.Here,the multi-view method is analyzed comprehensively.By referring to video-based input,the human prior knowledge of restricted motion is used to predict human postures.In addition,structural constraints are widely used as prior knowledge.Furthermore,weakly supervised learningmethods are studied and discussed for these two types of inputs to improve the model generalization ability.The problem of insufficient training datasets must also be considered,especially because 3D datasets are usually biased and limited.Finally,emerging and popular datasets and evaluation indicators are discussed.The characteristics of the datasets and the relationships of the indicators are explained and highlighted.Thus,this article can be useful and instructive for researchers who are lacking in experience and find this field confusing.In addition,by providing an overview of 3D human pose estimation,this article sorts and refines recent studies on 3D human pose estimation.It describes kernel problems and common useful methods,and discusses the scope for further research.

Optimal Deep Convolutional Neural Network with Pose Estimation for Human Activity Recognition

Human Action Recognition(HAR)and pose estimation from videos have gained significant attention among research communities due to its applica-tion in several areas namely intelligent surveillance,human robot interaction,robot vision,etc.Though considerable improvements have been made in recent days,design of an effective and accurate action recognition model is yet a difficult process owing to the existence of different obstacles such as variations in camera angle,occlusion,background,movement speed,and so on.From the literature,it is observed that hard to deal with the temporal dimension in the action recognition process.Convolutional neural network(CNN)models could be used widely to solve this.With this motivation,this study designs a novel key point extraction with deep convolutional neural networks based pose estimation(KPE-DCNN)model for activity recognition.The KPE-DCNN technique initially converts the input video into a sequence of frames followed by a three stage process namely key point extraction,hyperparameter tuning,and pose estimation.In the keypoint extraction process an OpenPose model is designed to compute the accurate key-points in the human pose.Then,an optimal DCNN model is developed to classify the human activities label based on the extracted key points.For improving the training process of the DCNN technique,RMSProp optimizer is used to optimally adjust the hyperparameters such as learning rate,batch size,and epoch count.The experimental results tested using benchmark dataset like UCF sports dataset showed that KPE-DCNN technique is able to achieve good results compared with benchmark algorithms like CNN,DBN,SVM,STAL,T-CNN and so on.

Deep Learning Based 2D Human Pose Estimation:A Survey

Human pose estimation has received significant attention recently due to its various applications in the real world. As the performance of the state-of-the-art human pose estimation methods can be improved by deep learning, this paper presents a comprehensive survey of deep learning based human pose estimation methods and analyzes the methodologies employed. We summarize and discuss recent works with a methodologybased taxonomy. Single-person and multi-person pipelines are first reviewed separately. Then, the deep learning techniques applied in these pipelines are compared and analyzed. The datasets and metrics used in this task are also discussed and compared. The aim of this survey is to make every step in the estimation pipelines interpretable and to provide readers a readily comprehensible explanation. Moreover, the unsolved problems and challenges for future research are discussed.

RFPose-OT:RF-based 3D human pose estimation via optimal transport theory

This paper introduces a novel framework,i.e.,RFPose-OT,to enable three-dimensional(3D)human pose estimation from radio frequency(RF)signals.Different from existing methods that predict human poses from RF signals at the signal level directly,we consider the structure difference between the RF signals and the human poses,propose a transformation of the RF signals to the pose domain at the feature level based on the optimal transport(OT)theory,and generate human poses from the transformed features.To evaluate RFPose-OT,we build a radio system and a multi-view camera system to acquire the RF signal data and the ground-truth human poses.The experimental results in a basic indoor environment,an occlusion indoor environment,and an outdoor environment demonstrate that RFPose-OT can predict 3D human poses with higher precision than state-of-the-art methods.

视觉SLAM方法综述

实时定位与建图(SLAM)技术搭载特定传感器,使移动机器人在无任何环境先验条件下,在运动过程中自主建立环境模型来计算自身位姿,大幅提高其自主导航能力,以及对不同应用环境的适应性。视觉SLAM方法以相机作为外部传感器,通过采集周围环境信息来创建地图并实时估计机器人自身位姿。为此,介绍了具有代表性的经典视觉SLAM方法及与深度学习相结合的视觉SLAM方法,分析了视觉SLAM方法中采用的不同特征检测方法、后端优化、闭环检测,以及动态环境下视觉SLAM方法的应用,总结了视觉SLAM方法的问题,并探讨了视觉SLAM方法在未来的热点研究方向和发展前景。

基于深度学习的二维人体姿态估计综述

人体姿态估计是近年来计算机视觉问题中的一个热门话题,它在改善人类生活方面具有巨大的益处和潜在的应用。近年来深度神经网络得到快速发展,相较于传统方法而言,采用深度学习的方法更能提取图像表征信息。综合分析近年来人体姿态估计的进展,根据检测人数分为单人和多人人体姿态估计。针对单人姿态估计,介绍了基于直接预测人体坐标点的坐标回归方法及基于预测人体关键点高斯分布的热图检测方法;针对多人姿态估计,采用解决多人到解决单人过程的自顶向下方法和直接处理多人关键点的自底向上方法。总结了各方法网络结构的特点和优缺点,并阐述当前面临的问题及未来发展趋势。

基于Transformer的三维人体姿态估计及其动作达成度评估

针对人机交互、医疗康复等领域存在的人体姿态分析与评估问题,本文提出了一种基于Transformer的三维人体姿态估计及其动作达成度评估方法。首先,本文定义了人体姿态的关键点及关节角,并在深度位姿估计网络(DPEN)的基础上,提出并构建了一个基于Transformer的三维人体姿态估计模型(TPEM),Transformer的引入能够更好的提取人体姿态的长时序特征;其次,利用TPEM模型对三维人体姿态估计结果,设计了基于加权3D关节角的动态时间规整算法,在时序上对不同人物同一动作的姿态进行姿态关键帧的规整匹配,并据此提出了动作达成度评估方法,用于给出动作的达成度分数;最后,通过在不同数据集上进行实验验证,TPEM在Human3.6 M数据集上实现了平均关节点误差为37.3 mm,而基于加权3D关节角的动态时间规整算法在Fit3D数据集上的平均误差帧数为5.08,展现了本文所提方法在三维人体姿态估计与动作达成度评估方面的可行性和有效性。

基于深度学习的二维人体姿态估计研究进展

人体姿态估计在人体行为识别、人机交互、体育运动分析等方面有着广泛的应用前景,是计算机视觉领域的一个研究热点。在最近的十年中,得益于深度学习技术,大量的研究工作极大地推动了人体姿态估计技术的发展,但由于受训练样本不足、人体姿态的多变性、遮挡、环境的复杂性等因素影响,人体姿态估计仍然面临着诸多的挑战。文中对近年来基于深度学习的2D人体姿态估计方法进行归纳和总结,着重分析一些有代表性的人体姿态估计方法的思路及工作原理,以便研究人员了解当前的研究现状、面临的挑战以及今后的研究方向,拓展研究思路。

基于多模态数据的人体行为识别方法研究综述

人体行为识别广泛应用于智能安防、自动驾驶和人机交互等领域。随着拍摄设备和传感器技术的发展,可获取用于人体行为识别的数据不再局限于RGB数据,还有深度、骨骼和红外等多模态数据。详细介绍了基于RGB和骨骼数据模态的人体行为识别任务中特征提取方法,包括基于手工标注和基于深度学习的方法。对于RGB数据模态,重点分析了基于双流卷积神经网络、3D卷积神经网络和混合网络的特征提取算法。对于骨骼数据模态,介绍了目前流行的单人和多人姿态评估算法;重点分析了基于卷积神经网络、循环神经网络和图卷积神经网络的分类算法;进一步全面展示了两种数据模态的通用数据集。此外,基于RGB和骨骼各自的数据结构特征,探讨了目前面临的挑战,最后对未来基于深度学习的人体行为识别方法的研究方向进行了展望。

动态环境下基于深度学习的视觉SLAM

传统的视觉同时定位与建图(SLAM)技术都是根据对静态环境条件的假设而设计,在动态环境中,运动目标的移动会导致特征匹配失败,进而影响位姿的估计。基于此提出了一种结合卷积神经网络的视觉SLAM算法,通过对ORB-SLAM2算法RGB-D模式前端添加结合注意力机制的卷积神经网络动态目标检测线程,在提取图像特征点时剔除动态目标区域,使用静态特征点完成对相机位姿精确的估计。仿真实验在TUM动态数据集下测试,通过多次测试后结果显示改进后的算法的位姿精度比原始算法提高90%以上,并且算法能满足实时性要求。

融合自我知识蒸馏和卷积压缩的轻量化人体姿态估计方法

为追求更准确的关键点检测结果,现有许多有关人体姿态估计研究多采用复杂的深度网络架构构建模型,忽略了模型的实际部署成本,导致模型在资源受限的边缘设备上很难实际部署,缺乏实用性.为了解决上述问题,本文设计了一种融合自我知识蒸馏和卷积压缩的轻量化人体姿态估计模型.该模型首先使用改进的EfficientNet网络构建一个编码器,提取图像的多尺度特征;其次,基于深度可分离转置卷积,设计一种轻量化上采样解码器,估计人体姿态;最后,采用轻量化多尺度双向融合与知识自我蒸馏方法,进一步提高人体姿态估计的准确性.在COCO和MPII标准数据集上进行了广泛的定性、定量和消融实验,实验结果表明所提出的模型不仅能获得准确的人体姿态估计,而且能显著降低模型的计算复杂性.

基于神经网络的VSLAM综述

传统的基于视觉的SLAM技术成果颇丰,但在具有挑战性的环境中难以取得想要的效果.深度学习推动了计算机视觉领域的快速发展,并在图像处理中展现出愈加突出的优势.将深度学习与基于视觉的SLAM结合是一个热门话题,诸多研究人员的努力使二者的广泛结合成为可能.本文从深度学习经典的神经网络入手,介绍了深度学习与传统基于视觉的SLAM算法的结合,概述了卷积神经网络(CNN)与循环神经网络(RNN)在深度估计、位姿估计、闭环检测等方面的成就,分析了神经网络在语义信息提取方面的优点,以期为未来自主移动机器人真正自主化提供帮助.最后,对未来VSLAM发展进行了展望.

对深度学习中目标定位不确定度评定的讨论

得益于深度学习技术的快速发展,高准确率的目标自动定位得以实现,这为各领域的智能化转型提供了极大的助力。然而,尽管大多数情况下基于深度学习的目标定位方法都能够获得较为精确的输出,但一些误差较大的定位结果还是难以避免。正因为缺少规范的不确定度评定,所以那些误差较大的定位结果难以被有效地消除,进而影响了基于深度学习的目标定位方法的工程化应用。从符合计量规范的测量不确定度入手,讨论了当前深度学习中目标定位技术评价指标的意义和不足,并提出了对规范化目标定位技术不确定度评定的建议。

室内动态场景下基于语义关联的视觉SLAM方法

针对视觉SLAM在动态场景下鲁棒性不足的问题,提出一种适用于动态场景下的视觉SLAM算法——SAD-SLAM。该算法首先使用GCNv2网络进行特征提取,以获取分布均匀的特征点集合,并加快提取速度。然后使用YOLOv8-seg语义分割网络完成场景内物体的检测,并对推理得到的物体按照是否具备自主运动能力进行划分。同时提出一种语义关联方法,通过对潜在动态物体进行2D和深度层面过滤,以确定潜在动态物体运动的可能性。最后,构建了含有语义信息的稠密3D点云地图,并避免了动态物体的干扰。算法使用TUM数据集及真实场景进行实验验证,结果表明,相较于ORB-SLAM3及其他相关的动态SLAM算法,SAD-SLAM在动态场景下具有更好的定位精度。

强化先验骨架结构的轻量型高效人体姿态估计

为了更好地利用人体姿态关键点特有的分布属性,提出强化先验骨架结构的轻量型高效人体姿态估计方法.利用高分辨率网络较好地保留空间位置信息,为了进一步降低模型参数量,提出轻量倒残差模块.设计体位强化模块,利用全局空间特征和上下文信息强化躯干位置的先验信息及关键点之间的联系.针对多分辨率特征图像融合时,像素位置模糊、卷积核优化方向偏移导致关键点空间特征信息遗失的问题,提出方向强化卷积模块,利用躯干上关键点分布的水平和垂直方向特性,高效融合关键点先验分布.实验结果表明,利用该网络,可以高效地估计人体姿态.与基准网络相比,该模型在COCO测试集上的平均精度达到78.4,参数量减少了17.4×10^(6),兼顾精度与效率.

改进的三维人体姿态估计算法

针对目前三维人体姿态由于遮挡、姿态复杂等预测不准确的问题,提出了一种改进的三维人体姿态估计算法以获得准确的三维人体姿态,提高人体姿态估计性能.本文采用时空图注意力卷积网络中的图注意力块来构建整个网络,在此基础上对全局多头图注意力部分的网络结构进行改进,使节点间更好传播和融合信息,捕获图中没有显式表示的语义信息.同时引入运动学约束,在MPJPE损失的基础上,加上骨骼长度损失.通过对局部和全局的空间节点信息建模,实现对局部运动学连接、对称性和全局姿态的人体骨骼运动学约束的学习.通过实验证明,本文改进后的模型有效地提高了人体姿态估计性能,在Human3.6M数据集上相较于原始模型,实现了1.8%的平均关节位置误差(MPJPE)提升和1.3%的预测关节与真值关节刚性对齐后的平均关节位置误差(P-MPJPE)提升.

基于HRNet的轻量化人体姿态估计网络

由于算力和内存的限制,目前的人体姿态估计网络难以广泛应用于移动设备、嵌入式平台.针对这个问题,本文以HRNet为基础框架,提出了一种轻量化的人体姿态估计网络X-HRNet,使用ResNeXt模块替换普通的Basic模块以减少网络的参数和计算复杂度.实验结果表明,所提出模型在COCO验证集上取得了78.2%的精度,比HRNet高1.9%,参数量下降了22.2M,计算量下降了27.3GFLOPs.与以往的轻量化人体姿态估计方法不同,所提出的XHRNet是一种兼顾精度和轻量化的方法,在保持精度的同时有效减少了计算量和参数量,为嵌入式平台提出了一种新的轻量化人体姿态估计网络.

运动场景下的多目标人体姿态估计

为提高群体活动场景下细粒度人体姿态估计的准确率,优化网路中人体识别及姿态估计算法,在现有研究的基础上,提出一种结合多尺度预测以及改进并行注意力模块的多目标人体姿态估计算法。在充分利用不同尺度特征信息的基础上,实现高质量的人体姿态估计;针对运动场景下多目标人体姿态数据集较少,提出一种数据集CUPB Sport Dataset。实验结果表明,该算法在公开基准数据集和自制数据集上分别达到了81.4 mAP和79.7 mAP,验证了该算法在运动场景下针对多目标的高效性。