RFID-based 3D human pose tracking: A subject generalization approach

Three-dimensional (3D) human pose tracking has recently attracted more and more attention in the computer vision field. Real-time pose tracking is highly useful in various domains such as video surveillance, somatosensory games, and human-computer interaction. However, vision-based pose tracking techniques usually raise privacy concerns, making human pose tracking without vision data usage an important problem. Thus, we propose using Radio Frequency Identification (RFID) as a pose tracking technique via a low-cost wearable sensing device. Although our prior work illustrated how deep learning could transfer RFID data into real-time human poses, generalization for different subjects remains challenging. This paper proposes a subject-adaptive technique to address this generalization problem. In the proposed system, termed Cycle-Pose, we leverage a cross-skeleton learning structure to improve the adaptability of the deep learning model to different human skeletons. Moreover, our novel cycle kinematic network is proposed for unpaired RFID and labeled pose data from different subjects. The Cycle-Pose system is implemented and evaluated by comparing its prototype with a traditional RFID pose tracking system. The experimental results demonstrate that Cycle-Pose can achieve lower estimation error and better subject generalization than the traditional system.

Toward Coordination Control of Multiple Fish-Like Robots:Real-Time Vision-Based Pose Estimation and Tracking via Deep Neural Networks

Controlling multiple multi-joint fish-like robots has long captivated the attention of engineers and biologists,for which a fundamental but challenging topic is to robustly track the postures of the individuals in real time.This requires detecting multiple robots,estimating multi-joint postures,and tracking identities,as well as processing fast in real time.To the best of our knowledge,this challenge has not been tackled in the previous studies.In this paper,to precisely track the planar postures of multiple swimming multi-joint fish-like robots in real time,we propose a novel deep neural network-based method,named TAB-IOL.Its TAB part fuses the top-down and bottom-up approaches for vision-based pose estimation,while the IOL part with long short-term memory considers the motion constraints among joints for precise pose tracking.The satisfying performance of our TAB-IOL is verified by testing on a group of freely swimming fish-like robots in various scenarios with strong disturbances and by a deed comparison of accuracy,speed,and robustness with most state-of-the-art algorithms.Further,based on the precise pose estimation and tracking realized by our TAB-IOL,several formation control experiments are conducted for the group of fish-like robots.The results clearly demonstrate that our TAB-IOL lays a solid foundation for the coordination control of multiple fish-like robots in a real working environment.We believe our proposed method will facilitate the growth and development of related fields.

Three-dimensional CityGML building models in mobile augmented reality: a smartphone-based pose tracking system

Many augmented reality(AR)systems are developed for entertainment,but AR and particularly mobile AR potentially have more application possibilities in other fields.For example,in civil engineering or city planning,AR could be used in combination with CityGML building models to enhance some typical workflows in planning,execution and operation processes.A concrete example is the geo-referenced on-site visualization of planned buildings or building parts,to simplify planning processes and optimize the communication between the participating decision-makers.One of the main challenges for the visualization lies in the pose tracking,i.e.the real-time estimation of the translation and rotation of the mobile device to align the virtual objects with reality.In this paper,we introduce a proof-of-concept fine-grained mobile AR CityGML-based pose tracking system aimed at the mentioned applications.The system estimates poses by combining 3D CityGML data with information derived from 2D camera images and an inertial measurement unit and is fully self-sufficient and operates without external infrastructure.The results of our evaluation show that CityGML and low-cost off-the-shelf mobile devices,such as smartphones,already provide performant and accurate mobile pose tracking for AR in civil engineering and city planning.

Tracking Human Poses with Head Orientation Estimation

Lots of progress has been made recently on 2 D human pose tracking with tracking-by-detection approaches. However,several challenges still remain in this area which is due to self-occlusions and the confusion between the left and right limbs during tracking. In this work,a head orientation detection step is introduced into the tracking framework to serve as a complementary tool to assist human pose estimation. With the face orientation determined,the system can decide whether the left or right side of the human body is exactly visible and infer the state of the symmetric counterpart. By granting a higher priority for the completely visible side,the system can avoid double counting to a great extent when inferring body poses. The proposed framework is evaluated on the HumanEva dataset. The results show that it largely reduces the occurrence of double counting and distinguishes the left and right sides consistently.

A unified multi-view multi-person tracking framework

Despite significant developments in 3D multi-view multi-person (3D MM) tracking, current frameworks separately target footprint tracking, or pose tracking. Frameworks designed for the former cannot be used for the latter, because they directly obtain 3D positions on the ground plane via a homography projection, which is inapplicable to 3D poses above the ground. In contrast, frameworks designed for pose tracking generally isolate multi-view and multi-frame associations and may not be sufficiently robust for footprint tracking, which utilizes fewer key points than pose tracking, weakening multi-view association cues in a single frame. This study presents a unified multi-view multi-person tracking framework to bridge the gap between footprint tracking and pose tracking. Without additional modifications, the framework can adopt monocular 2D bounding boxes and 2D poses as its input to produce robust 3D trajectories for multiple persons. Importantly, multi-frame and multi-view information are jointly employed to improve association and triangulation. Our framework is shown to provide state-of-the-art performance on the Campus and Shelf datasets for 3D pose tracking, with comparable results on the WILDTRACK and MMPTRACK datasets for 3D footprint tracking.

SPATIAL TRAJECTORY PREDICTION OF VISUAL SERVOING

Target tracking is one typical application of visual servoing technology. It is still a difficult task to track high speed target with current visual servo system. The improvement of visual servoing scheme is strongly required. A position-based visual servo parallel system is presented for tracking target with high speed. A local Frenet frame is assigned to the sampling point of spatial trajectory. Position estimation is formed by the differential features of intrinsic geometry, and orientation estimation is formed by homogenous transformation. The time spent for searching and processing can be greatly reduced by shifting the window according to features location prediction. The simulation results have demonstrated the ability of the system to track spatial moving object.

Input-Constrained Hybrid Control of a Hyper-Redundant Mobile Medical Manipulator

To reduce the risk of infection in medical personnel working in infectious-disease areas, we proposed ahyper-redundant mobile medical manipulator (HRMMM) to perform contact tasks in place of healthcare workers.A kinematics-based tracking algorithm was designed to obtain highly accurate pose tracking. A kinematic modelof the HRMMM was established and its global Jacobian matrix was deduced. An expression of the trackingerror based on the Rodrigues rotation formula was designed, and the relationship between tracking errors andgripper velocities was derived to ensure accurate object tracking. Considering the input constraints of the physicalsystem, a joint-constraint model of the HRMMM was established, and the variable-substitution method was usedto transform asymmetric constraints to symmetric constraints. All constraints were normalized by dividing bytheir maximum values. A hybrid controller based on pseudo-inverse (PI) and quadratic programming (QP) wasdesigned to satisfy the real-time motion-control requirements in medical events. The PI method was used whenthere was no input saturation, and the QP method was used when saturation occurred. A quadratic performanceindex was designed to ensure smooth switching between PI and QP. The simulation results showed that theHRMMM could approach the target pose with a smooth motion trajectory, while meeting different types of inputconstraints.