Investigation of Inside-Out Tracking Methods for Six Degrees of Freedom Pose Estimation of a Smartphone in Augmented Reality

Six degrees of freedom(6DoF)input interfaces are essential formanipulating virtual objects through translation or rotation in three-dimensional(3D)space.A traditional outside-in tracking controller requires the installation of expensive hardware in advance.While inside-out tracking controllers have been proposed,they often suffer from limitations such as interaction limited to the tracking range of the sensor(e.g.,a sensor on the head-mounted display(HMD))or the need for pose value modification to function as an input interface(e.g.,a sensor on the controller).This study investigates 6DoF pose estimation methods without restricting the tracking range,using a smartphone as a controller in augmented reality(AR)environments.Our approach involves proposing methods for estimating the initial pose of the controller and correcting the pose using an inside-out tracking approach.In addition,seven pose estimation algorithms were presented as candidates depending on the tracking range of the device sensor,the tracking method(e.g.,marker recognition,visual-inertial odometry(VIO)),and whether modification of the initial pose is necessary.Through two experiments(discrete and continuous data),the performance of the algorithms was evaluated.The results demonstrate enhanced final pose accuracy achieved by correcting the initial pose.Furthermore,the importance of selecting the tracking algorithm based on the tracking range of the devices and the actual input value of the 3D interaction was emphasized.

An Iterative Pose Estimation Algorithm Based on Epipolar Geometry With Application to Multi-Target Tracking

This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images use matching features to estimate the essential matrix. The essential matrix is then decomposed into the relative rotation and normalized translation between frames. To be robust to noise and feature match outliers, these methods generate a large number of essential matrix hypotheses from randomly selected minimal subsets of feature pairs, and then score these hypotheses on all feature pairs. Alternatively, the algorithm introduced in this paper calculates relative pose hypotheses by directly optimizing the rotation and normalized translation between frames, rather than calculating the essential matrix and then performing the decomposition. The resulting algorithm improves computation time by an order of magnitude. If an inertial measurement unit(IMU) is available, it is used to seed the optimizer, and in addition, we reuse the best hypothesis at each iteration to seed the optimizer thereby reducing the number of relative pose hypotheses that must be generated and scored. These advantages greatly speed up performance and enable the algorithm to run in real-time on low cost embedded hardware. We show application of our algorithm to visual multi-target tracking(MTT) in the presence of parallax and demonstrate its real-time performance on a 640 × 480 video sequence captured on a UAV. Video results are available at https://youtu.be/Hh K-p2 h XNn U.

Toward Coordination Control of Multiple Fish-Like Robots:Real-Time Vision-Based Pose Estimation and Tracking via Deep Neural Networks

Controlling multiple multi-joint fish-like robots has long captivated the attention of engineers and biologists,for which a fundamental but challenging topic is to robustly track the postures of the individuals in real time.This requires detecting multiple robots,estimating multi-joint postures,and tracking identities,as well as processing fast in real time.To the best of our knowledge,this challenge has not been tackled in the previous studies.In this paper,to precisely track the planar postures of multiple swimming multi-joint fish-like robots in real time,we propose a novel deep neural network-based method,named TAB-IOL.Its TAB part fuses the top-down and bottom-up approaches for vision-based pose estimation,while the IOL part with long short-term memory considers the motion constraints among joints for precise pose tracking.The satisfying performance of our TAB-IOL is verified by testing on a group of freely swimming fish-like robots in various scenarios with strong disturbances and by a deed comparison of accuracy,speed,and robustness with most state-of-the-art algorithms.Further,based on the precise pose estimation and tracking realized by our TAB-IOL,several formation control experiments are conducted for the group of fish-like robots.The results clearly demonstrate that our TAB-IOL lays a solid foundation for the coordination control of multiple fish-like robots in a real working environment.We believe our proposed method will facilitate the growth and development of related fields.

Research on 3D Target Pose Tracking and Modeling

This paper tackles pose tracking and model refinement, one of the fundamental work for 3D photogrammetry. The researches belong to the videometrics, an interdisciplinewhich combines computer vision, digital image processing, photogrammetry and optical measurement. Related works are summarized briefly in this paper. This paper studies the problem of pose tracking for target with 3D model. For the target with accurate 3D model, line model based pose tracking methods are proposed for target which is rich in line features. Experimental results indicate that the proposed methods track the target pose accurately. Normal distance iterative reweighted least squares and distance image iterative least squares methods are proposed to process more general targets. This paper adopts bundle adjustment to tackle pose tracking in image sequence for target with inaccurate 3D line model. The proposed method optimizes the model line parameters and the pose parameters simultaneously. The model line orientation, position and mean angle error, mean position error of the pose are 0.3°,3.5 mm and 0.12°,20.1 mm in simulation experiments of satellite pose tracking. Line features are used to track target pose with unknown 3D model through image sequence. The model line parameters and pose parameters are optimized under the framework of SFM. In simulation experiments, the reconstructed line orientation, position error and mean angle error, mean position error of pose are 0.4°,7.5 mm and 0.16°,23.5 mm.

Tracking Human Poses with Head Orientation Estimation

Lots of progress has been made recently on 2 D human pose tracking with tracking-by-detection approaches. However,several challenges still remain in this area which is due to self-occlusions and the confusion between the left and right limbs during tracking. In this work,a head orientation detection step is introduced into the tracking framework to serve as a complementary tool to assist human pose estimation. With the face orientation determined,the system can decide whether the left or right side of the human body is exactly visible and infer the state of the symmetric counterpart. By granting a higher priority for the completely visible side,the system can avoid double counting to a great extent when inferring body poses. The proposed framework is evaluated on the HumanEva dataset. The results show that it largely reduces the occurrence of double counting and distinguishes the left and right sides consistently.

RFID-based 3D human pose tracking: A subject generalization approach

Three-dimensional (3D) human pose tracking has recently attracted more and more attention in the computer vision field. Real-time pose tracking is highly useful in various domains such as video surveillance, somatosensory games, and human-computer interaction. However, vision-based pose tracking techniques usually raise privacy concerns, making human pose tracking without vision data usage an important problem. Thus, we propose using Radio Frequency Identification (RFID) as a pose tracking technique via a low-cost wearable sensing device. Although our prior work illustrated how deep learning could transfer RFID data into real-time human poses, generalization for different subjects remains challenging. This paper proposes a subject-adaptive technique to address this generalization problem. In the proposed system, termed Cycle-Pose, we leverage a cross-skeleton learning structure to improve the adaptability of the deep learning model to different human skeletons. Moreover, our novel cycle kinematic network is proposed for unpaired RFID and labeled pose data from different subjects. The Cycle-Pose system is implemented and evaluated by comparing its prototype with a traditional RFID pose tracking system. The experimental results demonstrate that Cycle-Pose can achieve lower estimation error and better subject generalization than the traditional system.

Human Body Tracking and Pose Estimation Using Modified Camshift Algorithm

In this paper, we propose multiple CAMShift Algorithm based on Kalman filter and weighted search windows that extracts skin color area and tracks several human body parts for real-time human tracking system. The CAMShift Algorithm we propose searches the skin color region by detecting the skin color area from background model. Kalman filter stabilizes the floated search area of CAMShift Algorithm. Each occlusion areas are avoided by using weighted window of non-search areas and main-search area. And shadows are eliminated from background model and intensity of shadow. The proposed modified Camshaft algorithm can estimate human pose in real-time and achieves 96.82% accuracy even in the case of occlusions.

Drone for Dynamic Monitoring and Tracking with Intelligent Image Analysis

Traditional monitoring systems that are used in shopping malls or com-munity management,mostly use a remote control to monitor and track specific objects;therefore,it is often impossible to effectively monitor the entire environ-ment.Whenfinding a suspicious person,the tracked object cannot be locked in time for tracking.This research replaces the traditionalfixed-point monitor with the intelligent drone and combines the image processing technology and automatic judgment for the movements of the monitored person.This intelligent system can effectively improve the shortcomings of low efficiency and high cost of the traditional monitor system.In this article,we proposed a TIMT(The Intel-ligent Monitoring and Tracking)algorithm which can make the drone have smart surveillance and tracking capabilities.It combined with Artificial Intelligent(AI)face recognition technology and the OpenPose which is able to monitor the phy-sical movements of multiple people in real time to analyze the meaning of human body movements and to track the monitored intelligently through the remote con-trol interface of the drone.This system is highly agile and could be adjusted immediately to any angle and screen that we monitor.Therefore,the system couldfind abnormal conditions immediately and track and monitor them automatically.That is the system can immediately detect when someone invades the home or community,and the drone can automatically track the intruder to achieve that the two significant shortcomings of the traditional monitor will be improved.Experimental results show that the intelligent monitoring and tracking drone sys-tem has an excellent performance,which not only dramatically reduces the num-ber of monitors and the required equipment but also achieves perfect monitoring and tracking.

基于计算机视觉的电力作业人员行为分析研究现状与展望

电力作业人员的有效监管是保障电力安全生产的基础。该文对电力视频中作业人员的行为识别研究进行了归类总结,涵盖静态行为分析(穿戴分析、动作分析和组合分析)和动态行为分析(复杂动作、时序行为和行为预测等);详细综述了电力作业行为分析中的核心算法模块,包括目标检测、姿态估计和视频跟踪等;论述了电力作业行为识别在算法高效性、鲁棒性、灵活性等方面所面临的应用难点和挑战,并展望了电力作业行为智能监控领域的未来发展方向,特别强调了在软硬件结合、通用大模型、生成式人工智能方面进行技术创新和改进所蕴含的潜在机会。

煤矿掘进机自定位截割控制方法及试验研究

开展了掘进机自定位截割控制方法及试验研究,提出了掘进机自定位截割控制策略,使用基于单目视觉与深度学习的掘进机机体六自由度位姿检测方法,以及具有规定性能的双层模糊自适应反步控制方法实现了掘进机自定位截割,并将其用于机体任意位姿下的截割头循迹跟踪控制。在掘进机两种不同的机体位姿下进行了截割头循迹跟踪控制试验,试验结果显示最大轮廓误差分别在48 mm和52 mm(约2.14%与2.32%)以内,验证了提出的掘进机自定位截割控制方法的有效性。

一种用于中国空间站增强现实维护应用的自动跟踪框架

跟踪注册是增强现实(Augmented reality,AR)中一项至关重要的技术。基于人工标识和基于图像的注册方法已广泛应用于增强装配系统中。然而,由于空间产品的弱纹理、结构对称、数据有限以及难以张贴标识等特点,空间站的跟踪注册仍面临挑战。本文提出了一种专门为中国空间站诱导维修应用设计的三维物体跟踪方法,旨在用AR技术取代传统的纸质手册,为宇航员提供更直观的操作指导。本文提出了一种无标记的中国空间站智能维护方法。采用点对特征方法来估计初始帧姿态,不需要手动调整来获得相应的姿态,这对宇航员来说至关重要,因为在微重力环境中精确运动是一项重大挑战,而且结合曲率信息可以提高采用点对特征方法估计初始帧姿态的效率。通过结合纹理和区域信息,消除了对宇航员位置的精确限制,进一步增强了空间对称产品的跟踪鲁棒性。该方法利用点云匹配来估计第一帧的初始姿态,并在失去跟踪后重新计算姿态。一旦获得了前一帧的姿态,则仅根据二维图像的区域和纹理信息来计算跟踪,以获得实时跟踪。实验结果表明:本文方法与基于标识的方法具有相同的姿态趋势;基于电子游标卡尺测量之间的误差在毫米级;使用专门为航天产品设计的专用工具成功更换过滤器,证明了在空间站上实施诱导维修程序的实用性和潜力。

人体上肢运动姿态多点视觉自动跟踪仿真

由于视觉传感器本身的噪声以及运动模糊等原因,使得采集的运动上肢姿态数据存在噪声和误差,导致多点视觉自动跟踪精度较低。为此,提出人体运动上肢姿态多点视觉自动跟踪方法仿真。布设多维传感器对姿态多点采集,建立传感器加速度误差模型,获取误差修正目标函数,利用蚁群算法(Ant Colony Optimization, ACO)寻优,修正传感器采集过程中的误差;采用卡尔曼滤波算法(Kalman Filtering Algorithm, Kalman),实现人体运动上肢姿态的多点视觉自动跟踪。实验结果表明,所提方法能够在保障跟踪稳定性的用时,提高姿态跟踪的精度和效率,跟踪耗时仅为10ms左右。

基于单目RGB数据的三维模板物体跟踪算法综述

文中介绍了三维物体跟踪技术及其最新研究进展,包括理论基础与评估指标、基于视觉特征的传统位姿求解和基于学习的位姿估计.分析了传统算法中各种特征约束、统计分割模型、能量函数以及加速策略.其中,区域及边缘约束成为跟踪的主流方向,多种特征可以融合产生更强的约束.统计分割模型由全局模型逐步发展为局部模型.非线性最小二乘形式的能量函数易于优化而被广泛采用.预计算、预渲染的加速策略使得跟踪的实时性极大提高.基于学习的算法现阶段在精度和速度上不及传统算法,但展现了更好的特征提取能力和处理更复杂场景的潜力.梳理了多种算法的跟踪精度及取得高精度的原因.总结了三维物体跟踪在复杂情境下跟踪精度下降和失败的问题,及其潜在的解决方案与发展方向.三维物体跟踪正朝向多特征融合、预计算、多任务的方向发展.

农业无人机动态降落方法研究

为实现农业无人机高速、准确降落于地面移动机械设备(自走式施肥机构、农业无人车等)上,基于Aruco码设计了一种复合视觉识别标志,并基于该视觉识别标志提出了农业无人机动态降落控制策略。降落策略主要分为两个阶段:阶段一为标识识别阶段,使用Aruco标识检测算法进行标识识别;阶段二为对带标识地面移动机械设备的稳定跟踪。工作时,利用PnP类算法求解无人机当前位姿,使用PD轨迹追踪算法实现农业无人机对地面移动平台的追踪。为了验证该控制策略的有效性与准确性,利用开源仿真平台Prometheus进行仿真验证,并使用MatLab进行数据分析,结果表明:该控制策略可实现农业无人机的自主降落。

基于光学定位系统反馈的机器人末端位姿误差在线补偿方法

为了提高手术辅助机器人的定位精度,提出了一种基于改进L-M迭代的机器人末端工具位姿误差在线补偿方法。该方法无须标定机器人运动学模型参数,将机器人的几何误差、非几何误差和坐标系配准误差都归结于机器人关节误差,利用光学定位系统反馈机器人末端工具位姿,通过雅可比矩阵把机器人末端工具位姿误差映射到机器人关节空间,迭代改变机器人关节位置,使机器人末端工具位姿逐渐逼近目标位姿。实验结果表明,所提方法可将机器人末端工具位置误差降至0.1mm,末端工具姿态误差降至0.05°,显著提高了机器人末端工具位姿精度。

姿势自适应权重更新策略及尺度回归的煤矿井下目标跟踪算法

随着计算机技术的飞速发展,煤矿井下智能工作面的建设取得了巨大进步。目标跟踪作为井下智能工作面建设的一个重要环节,在保证作业人员安全,提升开采效率等方面发挥着重要作用。但是煤矿井下光照条件较差,当目标有较大外观变化和遮挡时,算法的跟踪预测框将不能准确跟踪目标。为了解决上述问题,设计出姿势自适应权重更新策略,该策略能计算出不同帧与模板帧之间目标外观姿势的变化程度,该变化程度作为目标估计网络调制向量更新的依据,动态地更新调制向量。通过该更新策略,使得算法对于目标尺度变化较大物体的跟踪精度得到提升。同时设计的中心点尺度回归损失函数,将中心点距离和高宽损失引入损失函数,使得目标框回归更加精准。

A unified multi-view multi-person tracking framework

Despite significant developments in 3D multi-view multi-person (3D MM) tracking, current frameworks separately target footprint tracking, or pose tracking. Frameworks designed for the former cannot be used for the latter, because they directly obtain 3D positions on the ground plane via a homography projection, which is inapplicable to 3D poses above the ground. In contrast, frameworks designed for pose tracking generally isolate multi-view and multi-frame associations and may not be sufficiently robust for footprint tracking, which utilizes fewer key points than pose tracking, weakening multi-view association cues in a single frame. This study presents a unified multi-view multi-person tracking framework to bridge the gap between footprint tracking and pose tracking. Without additional modifications, the framework can adopt monocular 2D bounding boxes and 2D poses as its input to produce robust 3D trajectories for multiple persons. Importantly, multi-frame and multi-view information are jointly employed to improve association and triangulation. Our framework is shown to provide state-of-the-art performance on the Campus and Shelf datasets for 3D pose tracking, with comparable results on the WILDTRACK and MMPTRACK datasets for 3D footprint tracking.

基于自控测距的液压启闭机油缸清洁机器人位姿跟踪研究

针对当前机器人开展清洁工作时只考虑方向信息而未考虑异常情况,导致位姿末端跟踪结果不精准的问题,研究基于自控测距的液压启闭机油缸清洁机器人位姿跟踪技术。充分考虑到清洁机器人清洁过程中位姿不断变化,使用激光跟踪仪进行跟踪布局。提取机器人动态特征,计算俯仰轴姿态角、机器人前进位移,以此为依据构建清洁机器人位姿跟踪数学模型。使用自控测距法进行位姿跟踪,解决无相应幅值标志物条件下的非系统误差问题。通过对机器人各个角度的补偿,实现了机器人末端位姿误差的最小化。由仿真结果可知:该方法运动轨迹与实际轨迹基本一致,所跟踪的末端位姿坐标与实际坐标位姿一致。

基于模板匹配的法兰盘位姿估计研究

针对工业机器人自动分拣中当工件有较大的旋转角度时,普通的模板匹配法无法准确地得到抓取和放置工件正确位置的问题,提出一种基于内边界追踪粗定位和基于模板匹配精定位的两步视觉定位方法。针对谐波减速器输出法兰盘,采用八邻域内边界追踪法对工件旋转角度进行粗定位,通过霍夫圆检测技术得到工件的中心位置,并通过一种改进的模板匹配法得到工件精确的旋转角度。实验结果表明,位姿检测的平均误差不大于0.1°,该定位方法在合适条件下可以满足精度要求。

Real-Time Facial Pose Estimation and Tracking by Coarse-to-Fine Iterative Optimization

We present a novel and efficient method for real-time multiple facial poses estimation and tracking in a single frame or video.First,we combine two standard convolutional neural network models for face detection and mean shape learning to generate initial estimations of alignment and pose.Then,we design a bi-objective optimization strategy to iteratively refine the obtained estimations.This strategy achieves faster speed and more accurate outputs.Finally,we further apply algebraic filtering processing,including Gaussian filter for background removal and extended Kalman filter for target prediction,to maintain real-time tracking superiority.Only general RGB photos or videos are required,which are captured by a commodity monocular camera without any priori or label.We demonstrate the advantages of our approach by comparing it with the most recent work in terms of performance and accuracy.