Movement Function Assessment Based on Human Pose Estimation from Multi-View

Human pose estimation is a basic and critical task in the field of computer vision that involves determining the position(or spatial coordinates)of the joints of the human body in a given image or video.It is widely used in motion analysis,medical evaluation,and behavior monitoring.In this paper,the authors propose a method for multi-view human pose estimation.Two image sensors were placed orthogonally with respect to each other to capture the pose of the subject as they moved,and this yielded accurate and comprehensive results of three-dimensional(3D)motion reconstruction that helped capture their multi-directional poses.Following this,we propose a method based on 3D pose estimation to assess the similarity of the features of motion of patients with motor dysfunction by comparing differences between their range of motion and that of normal subjects.We converted these differences into Fugl–Meyer assessment(FMA)scores in order to quantify them.Finally,we implemented the proposed method in the Unity framework,and built a Virtual Reality platform that provides users with human–computer interaction to make the task more enjoyable for them and ensure their active participation in the assessment process.The goal is to provide a suitable means of assessing movement disorders without requiring the immediate supervision of a physician.

CPG Human Motion Phase Recognition Algorithm for a Hip Exoskeleton with VSA Actuator

Due to the dynamic stiffness characteristics of human joints, it is easy to cause impact and disturbance on normal movements during exoskeleton assistance. This not only brings strict requirements for exoskeleton control design, but also makes it difficult to improve assistive level. The Variable Stiffness Actuator (VSA), as a physical variable stiffness mechanism, has the characteristics of dynamic stiffness adjustment and high stiffness control bandwidth, which is in line with the stiffness matching experiment. However, there are still few works exploring the assistive human stiffness matching experiment based on VSA. Therefore, this paper designs a hip exoskeleton based on VSA actuator and studies CPG human motion phase recognition algorithm. Firstly, this paper puts forward the requirements of variable stiffness experimental design and the output torque and variable stiffness dynamic response standards based on human lower limb motion parameters. Plate springs are used as elastic elements to establish the mechanical principle of variable stiffness, and a small variable stiffness actuator is designed based on the plate spring. Then the corresponding theoretical dynamic model is established and analyzed. Starting from the CPG phase recognition algorithm, this paper uses perturbation theory to expand the first-order CPG unit, obtains the phase convergence equation and verifies the phase convergence when using hip joint angle as the input signal with the same frequency, and then expands the second-order CPG unit under the premise of circular limit cycle and analyzes the frequency convergence criterion. Afterwards, this paper extracts the plate spring modal from Abaqus and generates the neutral file of the flexible body model to import into Adams, and conducts torque-stiffness one-way loading and reciprocating loading experiments on the variable stiffness mechanism. After that, Simulink is used to verify the validity of the criterion. Finally, based on the above criterions, the signal mean value is removed using feedback structure to complete the phase recognition algorithm for the human hip joint angle signal, and the convergence is verified using actual human walking data on flat ground.

Robotic Etiquette:Socially Acceptable Navigation of Service Robots with Human Motion Pattern Learning and Prediction

Nonverbal and noncontact behaviors play a significant role in allowing service robots to structure their interactions withhumans.In this paper, a novel human-mimic mechanism of robot’s navigational skills was proposed for developing sociallyacceptable robotic etiquette.Based on the sociological and physiological concerns of interpersonal interactions in movement,several criteria in navigation were represented by constraints and incorporated into a unified probabilistic cost grid for safemotion planning and control, followed by an emphasis on the prediction of the human’s movement for adjusting the robot’spre-collision navigational strategy.The human motion prediction utilizes a clustering-based algorithm for modeling humans’indoor motion patterns as well as the combination of the long-term and short-term tendency prediction that takes into accountthe uncertainties of both velocity and heading direction.Both simulation and real-world experiments verified the effectivenessand reliability of the method to ensure human’s safety and comfort in navigation.A statistical user trials study was also given tovalidate the users’favorable views of the human-friendly navigational behavior.

Superflexible and Lead-Free Piezoelectric Nanogenerator as a Highly Sensitive Self-Powered Sensor for Human Motion Monitoring

For traditional piezoelectric sensors based on poled ceramics,a low curie tem-perature(T_(c))is a fatal flaw due to the depolarization phenomenon.However,in this study,we find the low T_(c) would be a benefit for flex-ible piezoelectric sensors because small alterations of force trig-ger large changes in polarization.BaTi_(0.88)Sn_(0.12)O_(3)(BTS)with high piezoelectric coefficient and low T_(c) close to human body temperature is taken as an example for materials of this kind.Continuous piezo-electric BTS films were deposited on the flexible glass fiber fabrics(GFF),self-powered sensors based on the ultra-thin,superflexible,and polarization-free BTS-GFF/PVDF composite piezoelectric films are used for human motion sensing.In the low force region(1-9 N),the sensors have the outstanding performance with voltage sensitivity of 1.23 V N^(−1) and current sensitivity of 41.0 nA N^(−1).The BTS-GFF/PVDF sensors can be used to detect the tiny forces of falling water drops,finger joint motion,tiny surface deformation,and fatigue driving with high sensitivity.This work provides a new paradigm for the preparation of superflexible,highly sensitive and wearable self-powered piezoelectric sensors,and this kind of sensors will have a broad application prospect in the fields of medical rehabilitation,human motion monitoring,and intelligent robot.

Dataset of human motion status using IR-UWB through-wall radar

Ultra-wideband(UWB)through-wall radar has a wide range of applications in non-contact human information detection and monitoring.With the integration of machine learning technology,its potential prospects include the physiological monitoring of patients in the hospital environment and the daily monitoring at home.Although many target detection methods of UWB through-wall radar based on machine learning have been proposed,there is a lack of an opensource dataset to evaluate the performance of the algorithm.This published dataset is measured by impulse radio UWB(IR-UWB)through-wall radar system.Three test subjects are measured in different environments and several defined motion status.Using the presented dataset,we propose a human-motion-status recognition method using a convolutional neural network(CNN),and the detailed dataset partition method and the recognition process flow are given.On the well-trained network,the recognition accuracy of testing data for three kinds of motion status is higher than 99.7%.The dataset presented in this paper considers a simple environment.Therefore,we call on all organizations in the UWB radar field to cooperate to build opensource datasets to further promote the development of UWB through-wall radar.

Textile-Based Strain Sensor for Human Motion Detection

Human motion analysis consists of real-time monitoring and recording of human body’s kinematics. It is very essential to track ambulatory and dailylife human motion, which is crucial for many applications and disciplines.Electronic textiles(e-textiles) afford a valid alternative to traditional solidstate sensors due to their merits of low cost, lightweight, flexibility, and feasibility to fit various human bodies. In this mini-review, textile-based sensor platforms and human motion analysis are well discussed in Section 1.Second, theoretical principles of textile-based strain sensors are introduced including resistive, capacitive, and piezoelectrical sensors. Section 3 focuses on various types of textile materials that are functionalized as sensing systems by intrinsic or extrinsic modifications. Section 4 summaries various types of e-textile-based strain sensors for human motion analysis. The final two sections mainly present perspectives and challenges, and conclusions,respectively.

A Lightweight Temporal Convolutional Network for Human Motion Prediction

A lightweight multi-layer residual temporal convolutional network model(RTCN)is proposed to target the highly complex kinematics and temporal correlation of human motion.RTCN uses 1-D convolution to efficiently obtain the spatial structure information of human motion and extract the correlation in the time series of human motion.The residual structure is applied to the proposed network model to alleviate the problem of gradient disappearance in the deep network.Experiments on the Human 3.6M dataset demonstrate that the proposed method effectively reduces the errors of motion prediction compared with previous methods,especially of long-term prediction.

Investigation of human motion effects on 60GHz indoor office propagation

A modified random walk model for human motion is proposed to investigate characteristics of 60GHz indoor office propagation.Compared with the classic random walk model,the movement tendency in the walking process is taken into account in the modified model.Based on the proposed model,path gains of the propagation environment are simulated under a variety of settings by using a ray tracing method.Simulation results and analysis show that human motion is a major source of disturbance to the indoor office propagation and results in performance degradation in some areas.

Design and Analysis of a Mechanical Device to Harvest Energy from Human Footstep Motion

Portable electronics is usually powered by battery,which is not sustainable not only to the longtime outdoor use but also to our living environment.There is rich kinetic energy in footstep motion during walking,so it is ideal to harvest the kinetic energy from human footstep motion as power source for portable electronic devices.In this paper,a novel mechanism based on dual-oscillating mode is designed to harvest the kinetic energy from footstep motion.The harvester contains two oscillating sub-mechanisms：one is spring-mass oscillator to absorb the vibration from external excitation,i.e.,the footstep motion,and the other is cantilever beam with tip mass for amplifying the vibration.Theoretic analysis shows that the dual-oscillating mechanism can be more effectively harness the foot step motion.The energy conversion sub-mechanism is based on the electromagnetic induction,where the wire coils fixed at the tip end of the cantilever beam serves as the slider and permanent magnets and yoke form the changing magnetic field.Simulation shows that the harvester,with total mass 70 g,can produce about 100 mW of electricity at the walking speed of 2 steps per second.

Recent Techniques for Harvesting Energy from the Human Body

The human body contains a near-infinite supply of energy in chemical,thermal,and mechanical forms.However,the majority of implantable and wear-able devices are still operated by batteries,whose insufficient capacity and large size limit their lifespan and increase the risk of hazardous material leakage.Such energy can be used to exceed the battery power limits of implantable and wear-able devices.Moreover,novel materials and fabrication methods can be used to create various medical therapies and life-enhancing technologies.This review paper focuses on energy-harvesting technologies used in medical and health applications,primarily power collectors from the human body.Current approaches to energy harvesting from the bodies of living subjects for self-powered electronics are summarized.Using the human body as an energy source encompasses numer-ous topics:thermoelectric generators,power harvesting by kinetic energy,cardi-ovascular energy harvesting,and blood pressure.The review considers various perspectives on future research,which can provide a new forum for advancing new technologies for the diagnosis,treatment,and prevention of diseases by integrating different energy harvesters with advanced electronics.

A survey on monocular 3D human pose estimation

Recovering human pose from RGB images and videos has drawn increasing attention in recent years owing to minimum sensor requirements and applicability in diverse fields such as human-computer interaction,robotics,video analytics,and augmented reality.Although a large amount of work has been devoted to this field,3D human pose estimation based on monocular images or videos remains a very challenging task due to a variety of difficulties such as depth ambiguities,occlusion,background clutters,and lack of training data.In this survey,we summarize recent advances in monocular 3D human pose estimation.We provide a general taxonomy to cover existing approaches and analyze their capabilities and limitations.We also present a summary of extensively used datasets and metrics,and provide a quantitative comparison of some representative methods.Finally,we conclude with a discussion on realistic challenges and open problems for future research directions.

Human Motion Recognition Using Ultra-Wideband Radar and Cameras on Mobile Robot

Cameras can reliably detect human motions in a normal environment, but they are usually affected by sudden illumination changes and complex conditions, which are the major obstacles to the reliability and robustness of the system. To solve this problem, a novel integration method was proposed to combine hi-static ultra-wideband radar and cameras. In this recognition system, two cameras are used to localize the object＇s region, regions while a radar is used to obtain its 3D motion models on a mobile robot. The recognition results can be matched in the 3D motion library in order to recognize its motions. To confirm the effectiveness of the proposed method, the experimental results of recognition using vision sensors and those of recognition using the integration method were compared in different environments. Higher correct-recognition rate is achieved in the experiment.

Impact of human motion on TVOCs inhalation dose under side re-circulated ventilation

The main objectives were to (1) calculate the total volatile organic compounds (TVOCs) inhalation dose, (2) analyze the proportions of human’s inhaled contaminant dose from different sources, and (3) present a newly defined ratio of relative inhalation dose level (RIDL) to assess indoor air quality (IAQ). A user defined function based on CFD (computational fluid dynamics) was developed, which integrated human motion model with TVOCs emission model in a high sidewall air supply ventilation mode. Based on simulation results of 10 cases, it is shown that the spatial concentration distribution of TVOCs is affected by human motion. TVOCs diffusion characteristic of building material is the most effective way to impact the TVOCs inhalation dose. From the RIDL index, case A-2 has the most serious IAQ problem, while case D-1 is of the best IAQ.

Key Pose Frame Extraction Method of Human Motion Based on 3D Framework and X-Means

The key pose frames of a human motion pose sequence,play an important role in the compression,retrieval and semantic analysis of continuous human motion.The current available clustering methods in literatures are difficult to determine the number of key pose frames automatically,and may destroy the postures’ temporal relationships while extracting key frames.To deal with this problem,this paper proposes a new key pose frames extraction method on the basis of 3D space distances of joint points and the improved X-means clustering algorithm.According to the proposed extraction method,the final key pose frame sequence could be obtained by describing the posture of human body with space distance of particular joint points and then the time-constraint X-mean algorithm is applied to cluster and filtrate the posture sequence.The experimental results show that the proposed method can automatically determine the number of key frames and save the temporal characteristics of motion frames according to the motion pose sequence.

STTG-net:a Spatio-temporal network for human motion prediction based on transformer and graph convolution network

In recent years,human motion prediction has become an active research topic in computer vision.However,owing to the complexity and stochastic nature of human motion,it remains a challenging problem.In previous works,human motion prediction has always been treated as a typical inter-sequence problem,and most works have aimed to capture the temporal dependence between successive frames.However,although these approaches focused on the effects of the temporal dimension,they rarely considered the correlation between different joints in space.Thus,the spatio-temporal coupling of human joints is considered,to propose a novel spatio-temporal network based on a transformer and a gragh convolutional network(GCN)(STTG-Net).The temporal transformer is used to capture the global temporal dependencies,and the spatial GCN module is used to establish local spatial correlations between the joints for each frame.To overcome the problems of error accumulation and discontinuity in the motion prediction,a revision method based on fusion strategy is also proposed,in which the current prediction frame is fused with the previous frame.The experimental results show that the proposed prediction method has less prediction error and the prediction motion is smoother than previous prediction methods.The effectiveness of the proposed method is also demonstrated comparing it with the state-of-the-art method on the Human3.6 M dataset.

Human Motion Recognition Based on Incremental Learning and Smartphone Sensors

Batch processing mode is widely used in the training process of human motiun recognition. After training, the motion elassitier usually remains invariable. However, if the classifier is to be expanded, all historical data must be gathered for retraining. This consumes a huge amount of storage space, and the new training process will be more complicated. In this paper, we use an incremental learning method to model the motion classifier. A weighted decision tree is proposed to help illustrate the process, and the probability sampling method is also used. The resuhs show that with continuous learning, the motion classifier is more precise. The average classification precision for the weighted decision tree was 88.43% in a typical test. Incremental learning consumes much less time than the batch processing mode when the input training data comes continuously.

Kinematic calibration under the expectation maximization framework for exoskeletal inertial motion capture system

This study presents a kinematic calibration method for exoskeletal inertial motion capture (EI-MoCap) system with considering the random colored noise such as gyroscopic drift.In this method, the geometric parameters are calibrated by the traditional calibration method at first. Then, in order to calibrate the parameters affected by the random colored noise, the expectation maximization (EM) algorithm is introduced. Through the use of geometric parameters calibrated by the traditional calibration method, the iterations under the EM framework are decreased and the efficiency of the proposed method on embedded system is improved. The performance of the proposed kinematic calibration method is compared to the traditional calibration method. Furthermore, the feasibility of the proposed method is verified on the EI-MoCap system. The simulation and experiment demonstrate that the motion capture precision is significantly improved by 16.79%and 7.16%respectively in comparison to the traditional calibration method.

Inferring object properties from human interaction and transferring them to new motions

Humans regularly interact with their surrounding objects.Such interactions often result in strongly correlated motions between humans and the interacting objects.We thus ask:"Is it possible to infer object properties from skeletal motion alone,even without seeing the interacting object itself?"In this paper,we present a fine-grained action recognition method that learns to infer such latent object properties from human interaction motion alone.This inference allows us to disentangle the motion from the object property and transfer object properties to a given motion.We collected a large number of videos and 3 D skeletal motions of performing actors using an inertial motion capture device.We analyzed similar actions and learned subtle differences between them to reveal latent properties of the interacting objects.In particular,we learned to identify the interacting object,by estimating its weight,or its spillability.Our results clearly demonstrate that motions and interacting objects are highly correlated and that related object latent properties can be inferred from 3 D skeleton sequences alone,leading to new synthesis possibilities for motions involving human interaction.Our dataset is available at http://vcc.szu.edu.cn/research/2020/IT.html.

Triboelectric gait sensing analysis system for self-powered IoT-based human motion monitoring

Quantitative analysis of gait parameters,such as stride frequency and step speed,is essential for optimizing physical exercise for the human body.However,the current electronic sensors used in human motion monitoring remain constrained by factors such as battery life and accuracy.This study developed a self-powered gait analysis system(SGAS)based on a triboelectric nanogenerator(TENG)fabricated electrospun composite nanofibers for motion monitoring and gait analysis for regulating exercise programs.The SGAS consists of a sensing module,a charging module,a data acquisition and processing module,and an Internet of Things(IoT)platform.Within the sensing module,two specialized sensing units,TENG-S1 and TENG-S2,are positioned at the forefoot and heel to generate synchronized signals in tandem with the user's footsteps.These signals are instrumental for real-time step count and step speed monitoring.The output of the two TENG units is significantly improved by systematically investigating and optimizing the electrospun composite nanofibers'composition,strength,and wear resistance.Additionally,a charge amplifier circuit is implemented to process the raw voltage signal,consequently bolstering the reliability of the sensing signal.This refined data is then ready for further reading and calculation by the micro-controller unit(MCU)during the signal transmission process.Finally,the well-conditioned signals are wirelessly transmitted to the IoT platform for data analysis,storage,and visualization,enhancing human motion monitoring.

A Multi-Task Motion Generation Model that Fuses a Discriminator and a Generator

The human motion generation model can extract structural features from existing human motion capture data,and the generated data makes animated characters move.The 3D human motion capture sequences contain complex spatial-temporal structures,and the deep learning model can fully describe the potential semantic structure of human motion.To improve the authenticity of the generated human motion sequences,we propose a multi-task motion generation model that consists of a discriminator and a generator.The discriminator classifies motion sequences into different styles according to their similarity to the mean spatial-temporal templates from motion sequences of 17 crucial human joints in three-freedom degrees.And target motion sequences are created with these styles by the generator.Unlike traditional related works,our model can handle multiple tasks,such as identifying styles and generating data.In addition,by extracting 17 crucial joints from 29 human joints,our model avoids data redundancy and improves the accuracy of model recognition.The experimental results show that the discriminator of the model can effectively recognize diversified movements,and the generated data can correctly fit the actual data.The combination of discriminator and generator solves the problem of low reuse rate of motion data,and the generated motion sequences are more suitable for actual movement.