Robotic Etiquette:Socially Acceptable Navigation of Service Robots with Human Motion Pattern Learning and Prediction

Nonverbal and noncontact behaviors play a significant role in allowing service robots to structure their interactions withhumans.In this paper, a novel human-mimic mechanism of robot’s navigational skills was proposed for developing sociallyacceptable robotic etiquette.Based on the sociological and physiological concerns of interpersonal interactions in movement,several criteria in navigation were represented by constraints and incorporated into a unified probabilistic cost grid for safemotion planning and control, followed by an emphasis on the prediction of the human’s movement for adjusting the robot’spre-collision navigational strategy.The human motion prediction utilizes a clustering-based algorithm for modeling humans’indoor motion patterns as well as the combination of the long-term and short-term tendency prediction that takes into accountthe uncertainties of both velocity and heading direction.Both simulation and real-world experiments verified the effectivenessand reliability of the method to ensure human’s safety and comfort in navigation.A statistical user trials study was also given tovalidate the users’favorable views of the human-friendly navigational behavior.

Superflexible and Lead-Free Piezoelectric Nanogenerator as a Highly Sensitive Self-Powered Sensor for Human Motion Monitoring

For traditional piezoelectric sensors based on poled ceramics,a low curie tem-perature(T_(c))is a fatal flaw due to the depolarization phenomenon.However,in this study,we find the low T_(c) would be a benefit for flex-ible piezoelectric sensors because small alterations of force trig-ger large changes in polarization.BaTi_(0.88)Sn_(0.12)O_(3)(BTS)with high piezoelectric coefficient and low T_(c) close to human body temperature is taken as an example for materials of this kind.Continuous piezo-electric BTS films were deposited on the flexible glass fiber fabrics(GFF),self-powered sensors based on the ultra-thin,superflexible,and polarization-free BTS-GFF/PVDF composite piezoelectric films are used for human motion sensing.In the low force region(1-9 N),the sensors have the outstanding performance with voltage sensitivity of 1.23 V N^(−1) and current sensitivity of 41.0 nA N^(−1).The BTS-GFF/PVDF sensors can be used to detect the tiny forces of falling water drops,finger joint motion,tiny surface deformation,and fatigue driving with high sensitivity.This work provides a new paradigm for the preparation of superflexible,highly sensitive and wearable self-powered piezoelectric sensors,and this kind of sensors will have a broad application prospect in the fields of medical rehabilitation,human motion monitoring,and intelligent robot.

Dataset of human motion status using IR-UWB through-wall radar

Ultra-wideband(UWB)through-wall radar has a wide range of applications in non-contact human information detection and monitoring.With the integration of machine learning technology,its potential prospects include the physiological monitoring of patients in the hospital environment and the daily monitoring at home.Although many target detection methods of UWB through-wall radar based on machine learning have been proposed,there is a lack of an opensource dataset to evaluate the performance of the algorithm.This published dataset is measured by impulse radio UWB(IR-UWB)through-wall radar system.Three test subjects are measured in different environments and several defined motion status.Using the presented dataset,we propose a human-motion-status recognition method using a convolutional neural network(CNN),and the detailed dataset partition method and the recognition process flow are given.On the well-trained network,the recognition accuracy of testing data for three kinds of motion status is higher than 99.7%.The dataset presented in this paper considers a simple environment.Therefore,we call on all organizations in the UWB radar field to cooperate to build opensource datasets to further promote the development of UWB through-wall radar.

Textile-Based Strain Sensor for Human Motion Detection

Human motion analysis consists of real-time monitoring and recording of human body’s kinematics. It is very essential to track ambulatory and dailylife human motion, which is crucial for many applications and disciplines.Electronic textiles(e-textiles) afford a valid alternative to traditional solidstate sensors due to their merits of low cost, lightweight, flexibility, and feasibility to fit various human bodies. In this mini-review, textile-based sensor platforms and human motion analysis are well discussed in Section 1.Second, theoretical principles of textile-based strain sensors are introduced including resistive, capacitive, and piezoelectrical sensors. Section 3 focuses on various types of textile materials that are functionalized as sensing systems by intrinsic or extrinsic modifications. Section 4 summaries various types of e-textile-based strain sensors for human motion analysis. The final two sections mainly present perspectives and challenges, and conclusions,respectively.

Key Pose Frame Extraction Method of Human Motion Based on 3D Framework and X-Means

The key pose frames of a human motion pose sequence,play an important role in the compression,retrieval and semantic analysis of continuous human motion.The current available clustering methods in literatures are difficult to determine the number of key pose frames automatically,and may destroy the postures’ temporal relationships while extracting key frames.To deal with this problem,this paper proposes a new key pose frames extraction method on the basis of 3D space distances of joint points and the improved X-means clustering algorithm.According to the proposed extraction method,the final key pose frame sequence could be obtained by describing the posture of human body with space distance of particular joint points and then the time-constraint X-mean algorithm is applied to cluster and filtrate the posture sequence.The experimental results show that the proposed method can automatically determine the number of key frames and save the temporal characteristics of motion frames according to the motion pose sequence.

Human Motion Recognition Based on Incremental Learning and Smartphone Sensors

Batch processing mode is widely used in the training process of human motiun recognition. After training, the motion elassitier usually remains invariable. However, if the classifier is to be expanded, all historical data must be gathered for retraining. This consumes a huge amount of storage space, and the new training process will be more complicated. In this paper, we use an incremental learning method to model the motion classifier. A weighted decision tree is proposed to help illustrate the process, and the probability sampling method is also used. The resuhs show that with continuous learning, the motion classifier is more precise. The average classification precision for the weighted decision tree was 88.43% in a typical test. Incremental learning consumes much less time than the batch processing mode when the input training data comes continuously.

STTG-net:a Spatio-temporal network for human motion prediction based on transformer and graph convolution network

In recent years,human motion prediction has become an active research topic in computer vision.However,owing to the complexity and stochastic nature of human motion,it remains a challenging problem.In previous works,human motion prediction has always been treated as a typical inter-sequence problem,and most works have aimed to capture the temporal dependence between successive frames.However,although these approaches focused on the effects of the temporal dimension,they rarely considered the correlation between different joints in space.Thus,the spatio-temporal coupling of human joints is considered,to propose a novel spatio-temporal network based on a transformer and a gragh convolutional network(GCN)(STTG-Net).The temporal transformer is used to capture the global temporal dependencies,and the spatial GCN module is used to establish local spatial correlations between the joints for each frame.To overcome the problems of error accumulation and discontinuity in the motion prediction,a revision method based on fusion strategy is also proposed,in which the current prediction frame is fused with the previous frame.The experimental results show that the proposed prediction method has less prediction error and the prediction motion is smoother than previous prediction methods.The effectiveness of the proposed method is also demonstrated comparing it with the state-of-the-art method on the Human3.6 M dataset.

Investigation of human motion effects on 60GHz indoor office propagation

A modified random walk model for human motion is proposed to investigate characteristics of 60GHz indoor office propagation.Compared with the classic random walk model,the movement tendency in the walking process is taken into account in the modified model.Based on the proposed model,path gains of the propagation environment are simulated under a variety of settings by using a ray tracing method.Simulation results and analysis show that human motion is a major source of disturbance to the indoor office propagation and results in performance degradation in some areas.

Human Motion Recognition Using Ultra-Wideband Radar and Cameras on Mobile Robot

Cameras can reliably detect human motions in a normal environment,but they are usually affected by sudden illumination changes and complex conditions,which are the major obstacles to the reliability and robustness of the system.To solve this problem,a novel integration method was proposed to combine bi-static ultra-wideband radar and cameras.In this recognition system,two cameras are used to localize the object's region,regions while a radar is used to obtain its 3D motion models on a mobile robot.The recognition results can be matched in the 3D motion library in order to recognize its motions.To confirm the effectiveness of the proposed method,the experimental results of recognition using vision sensors and those of recognition using the integration method were compared in different environments.Higher correct-recognition rate is achieved in the experiment.

Impact of human motion on TVOCs inhalation dose under side re-circulated ventilation

The main objectives were to (1) calculate the total volatile organic compounds (TVOCs) inhalation dose, (2) analyze the proportions of human’s inhaled contaminant dose from different sources, and (3) present a newly defined ratio of relative inhalation dose level (RIDL) to assess indoor air quality (IAQ). A user defined function based on CFD (computational fluid dynamics) was developed, which integrated human motion model with TVOCs emission model in a high sidewall air supply ventilation mode. Based on simulation results of 10 cases, it is shown that the spatial concentration distribution of TVOCs is affected by human motion. TVOCs diffusion characteristic of building material is the most effective way to impact the TVOCs inhalation dose. From the RIDL index, case A-2 has the most serious IAQ problem, while case D-1 is of the best IAQ.

Highly sensitive flexible strain sensor based on microstructured biphasic hydrogels for human motion monitoring

Flexible strain sensors have been extensively used in human motion detection,medical aids,electronic skins,and other civilian or military fields.Conventional strain sensors made of metal or semiconductor materials suffer from insufficient stretchability and sensitivity,imposing severe constraints on their utilization in wearable devices.Herein,we design a flexible strain sensor based on biphasic hydrogel via an in-situ polymerization method,which possesses superior electrical response and mechanical performance.External stress could prompt the formation of conductive microchannels within the biphasic hydrogel,which originates from the interaction between the conductive water phase and the insulating oil phase.The device performance could be optimized by carefully regulating the volume ratio of the oil/water phase.Consequently,the flexible strain sensor with oil phase ratio of 80%demonstrates the best sensitivity with gauge factor of 33 upon a compressive strain range of 10%,remarkable electrical stability of 100 cycles,and rapid resistance response of 190 ms.Furthermore,the human motions could be monitored by this flexible strain sensor,thereby highlighting its potential for seamless integration into wearable devices.

Facile and Economical Fabrication of Superhydrophobic Flexible Resistive Strain Sensors for Human Motion Detection

Superhydrophobic flexible strain sensors have great application value in the fields of personal health monitoring,human motion detection,and soft robotics due to their good flexibility and high sensitivity.However,complicated preparation processes and costly processing procedures have limited their development.To overcome these limitations,in this work we develop a facile and low-cost method for fabricating superhydrophobic flexible strain sensor via spraying carbon black(CB)nanoparticles dispersed in a thermoplastic elastomer(SEBS)solution on a polydimethylsiloxane(PDMS)flexible substrate.The prepared strain sensor had a large water contact angle of 153±2.83°and a small rolling angle of 8.5±1.04°,and exhibited excellent self-cleaning property.Due to the excellent superhydrophobicity,aqueous acid,salt,and alkali could quickly roll off the flexible strain sensor.In addition,the sensor showed excellent sensitivity(gauge factor(GF)of 5.4–7.35),wide sensing ranges(stretching:over 70%),good linearity(three linear regions),low hysteresis(hysteresis error of 4.8%),and a stable response over 100 stretching-releasing cycles.Moreover,the sensor was also capable of effectively detecting human motion signals like finger bending and wrist bending,showing promising application prospects in wearable electronic devices,personalized health monitoring,etc.

Movement Function Assessment Based on Human Pose Estimation from Multi-View

Human pose estimation is a basic and critical task in the field of computer vision that involves determining the position(or spatial coordinates)of the joints of the human body in a given image or video.It is widely used in motion analysis,medical evaluation,and behavior monitoring.In this paper,the authors propose a method for multi-view human pose estimation.Two image sensors were placed orthogonally with respect to each other to capture the pose of the subject as they moved,and this yielded accurate and comprehensive results of three-dimensional(3D)motion reconstruction that helped capture their multi-directional poses.Following this,we propose a method based on 3D pose estimation to assess the similarity of the features of motion of patients with motor dysfunction by comparing differences between their range of motion and that of normal subjects.We converted these differences into Fugl–Meyer assessment(FMA)scores in order to quantify them.Finally,we implemented the proposed method in the Unity framework,and built a Virtual Reality platform that provides users with human–computer interaction to make the task more enjoyable for them and ensure their active participation in the assessment process.The goal is to provide a suitable means of assessing movement disorders without requiring the immediate supervision of a physician.

A Multi-Task Motion Generation Model that Fuses a Discriminator and a Generator

The human motion generation model can extract structural features from existing human motion capture data,and the generated data makes animated characters move.The 3D human motion capture sequences contain complex spatial-temporal structures,and the deep learning model can fully describe the potential semantic structure of human motion.To improve the authenticity of the generated human motion sequences,we propose a multi-task motion generation model that consists of a discriminator and a generator.The discriminator classifies motion sequences into different styles according to their similarity to the mean spatial-temporal templates from motion sequences of 17 crucial human joints in three-freedom degrees.And target motion sequences are created with these styles by the generator.Unlike traditional related works,our model can handle multiple tasks,such as identifying styles and generating data.In addition,by extracting 17 crucial joints from 29 human joints,our model avoids data redundancy and improves the accuracy of model recognition.The experimental results show that the discriminator of the model can effectively recognize diversified movements,and the generated data can correctly fit the actual data.The combination of discriminator and generator solves the problem of low reuse rate of motion data,and the generated motion sequences are more suitable for actual movement.

Markerless human motion capture by Markov random field and dynamic graph cuts with color constraints

Currently, many vision-based motion capture systems require passive markers attached to key loca- tions on the human body. However, such systems are intrusive with limited application. The algorithm that we use for human motion capture in this paper is based on Markov random field （MRF） and dynamic graph cuts. It takes full account of the impact of 3D reconstruction error and integrates human motion capture and 3D reconstruction into MRF-MAP framework. For more accurate and robust performance, we extend our algorithm by incorporating color constraints into the pose estimation process. The advantages of incorporating color constraints are demonstrated by experimental results on several video sequences.

Natural polymers based triboelectric nanogenerator for harvesting biomechanical energy and monitoring human motion

Triboelectric nanogenerator(TENG)has been proved as a promising energy harvester in recent years,but the challenges of exploring economically triboelectric materials still exist and have aroused interests of many researchers.In this paper,chitosan-silk fibroin-airlaid paper composite film(CSA film)was fabricated and then the CSA film based-triboelectric nanogenerator(CSA-TENG)was constructed,which presents an opportunity for natural polymers to be applied in triboelectric materials.Due to the excellent electron donating ability of CSA film,the CSA-TENG can harvest environmental energy with a high efficiency.More importantly,the as-designed CSA film based dual-electrode triboelectric nanogenerator(CSA-D-TENG)is successfully assembled into hand clapper and trampoline to harvest mechanical energies generated by human bodies,it is also capable of monitoring human movement while harvesting biomechanical energies.This work provides a simple and environmental-friendly way to develop TENG for biomechanical energies harvesting and human motion monitoring.

Natural human–computer interaction control of multi operators in collaborative virtual maintenance based on optical human motion capture system

Aiming at the human–computer interaction control(HCIC)requirements of multi operators in collaborative virtual maintenance(CVM),real-time motion capture and simulation drive of multi operators with optical human motion capture system(HMCS)is proposed.The detailed realization process of real-time motion capture and data drive for virtual operators in CVM environment is presented to actualize the natural and online interactive operations.In order to ensure the cooperative and orderly interactions of virtual operators with the input operations of actual operators,collaborative HCIC model is established according to specific planning,allocating and decision-making of different maintenance tasks as well as the human–computer interaction features and collaborative maintenance operation features among multi maintenance trainees in CVM process.Finally,results of the experimental implementation validate the effectiveness and practicability of proposed methods,models,strategies and mechanisms.

Human motion prediction using optimized sliding window polynomial fitting and recursive least squares

Human motion prediction is a critical issue in human-robot collaboration(HRC)tasks.In order to reduce the local error caused by the limitation of the capture range and sampling frequency of the depth sensor,a hybrid human motion prediction algorithm,optimized sliding window polynomial fitting and recursive least squares(OSWPF-RLS)was proposed.The OSWPF-RLS algorithm uses the human body joint data obtained under the HRC task as input,and uses recursive least squares(RLS)to predict the human movement trajectories within the time window.Then,the optimized sliding window polynomial fitting(OSWPF)is used to calculate the multi-step prediction value,and the increment of multi-step prediction value was appropriately constrained.Experimental results show that compared with the existing benchmark algorithms,the OSWPF-RLS algorithm improved the multi-step prediction accuracy of human motion and enhanced the ability to respond to different human movements.

Design and Analysis of a Mechanical Device to Harvest Energy from Human Footstep Motion

Portable electronics is usually powered by battery,which is not sustainable not only to the longtime outdoor use but also to our living environment.There is rich kinetic energy in footstep motion during walking,so it is ideal to harvest the kinetic energy from human footstep motion as power source for portable electronic devices.In this paper,a novel mechanism based on dual-oscillating mode is designed to harvest the kinetic energy from footstep motion.The harvester contains two oscillating sub-mechanisms：one is spring-mass oscillator to absorb the vibration from external excitation,i.e.,the footstep motion,and the other is cantilever beam with tip mass for amplifying the vibration.Theoretic analysis shows that the dual-oscillating mechanism can be more effectively harness the foot step motion.The energy conversion sub-mechanism is based on the electromagnetic induction,where the wire coils fixed at the tip end of the cantilever beam serves as the slider and permanent magnets and yoke form the changing magnetic field.Simulation shows that the harvester,with total mass 70 g,can produce about 100 mW of electricity at the walking speed of 2 steps per second.

An Ultrasensitive, Durable and Stretchable Strain Sensor with Crack-wrinkle Structure for Human Motion Monitoring

Flexible strain sensor has promising features in successful application of health monitoring, electronic skins and smart robotics, etc.Here, we report an ultrasensitive strain sensor with a novel crack-wrinkle structure(CWS) based on graphite nanoplates(GNPs)/thermoplastic urethane(TPU)/polydimethylsiloxane(PDMS) nanocomposite. The CWS is constructed by pressing and dragging GNP layer on TPU substrate,followed by encapsulating with PDMS as a protective layer. On the basis of the area statistics, the ratio of the crack and wrinkle structures accounts for 31.8% and 9.5%, respectively. When the sensor is stretched, the cracks fracture, the wrinkles could reduce the unrecoverable destruction of cracks, resulting in an excellent recoverability and stability. Based on introduction of the designed CWS in the sensor, the hysteresis effect is limited effectively. The CWS sensor possesses a satisfactory sensitivity(GF=750 under 24% strain), an ultralow detectable limit(strain=0.1%) and a short respond time of 90 ms. For the sensing service behaviors, the CWS sensor exhibits an ultrahigh durability(high stability>2×10^(4) stretching-releasing cycles). The excellent practicality of CWS sensor is demonstrated through various human motion tests,including vigorous exercises of various joint bending, and subtle motions of phonation, facial movements and wrist pulse. The present CWS sensor shows great developing potential in the field of cost-effective, portable and high-performance electronic skins.