Digital Twin Technology of Human-Machine Integration in Cross-Belt Sorting System

The Chinese express delivery industry processes nearly 110 billion items in 2022,averaging an annual growth rate of 200%.Among the various types of sorting systems used for handling express items,cross-belt sorting systems stand out as the most crucial.However,despite their high degree of automation,the workload for operators has intensified owing to the surging volume of express items.In the era of Industry 5.0,it is imperative to adopt new technologies that not only enhance worker welfare but also improve the efficiency of cross-belt systems.Striking a balance between efficiency in handling express items and operator well-being is challenging.Digital twin technology offers a promising solution in this respect.A realization method of a human-machine integrated digital twin is proposed in this study,enabling the interaction of biological human bodies,virtual human bodies,virtual equipment,and logistics equipment in a closed loop,thus setting an operating framework.Key technologies in the proposed framework include a collection of heterogeneous data from multiple sources,construction of the relationship between operator fatigue and operation efficiency based on physiological measurements,virtual model construction,and an online optimization module based on real-time simulation.The feasibility of the proposed method was verified in an express distribution center.

Deep Learning for EMG-based Human-Machine Interaction:A Review

Electromyography(EMG)has already been broadly used in human-machine interaction(HMI)applications.Determining how to decode the information inside EMG signals robustly and accurately is a key problem for which we urgently need a solution.Recently,many EMG pattern recognition tasks have been addressed using deep learning methods.In this paper,we analyze recent papers and present a literature review describing the role that deep learning plays in EMG-based HMI.An overview of typical network structures and processing schemes will be provided.Recent progress in typical tasks such as movement classification,joint angle prediction,and force/torque estimation will be introduced.New issues,including multimodal sensing,inter-subject/inter-session,and robustness toward disturbances will be discussed.We attempt to provide a comprehensive analysis of current research by discussing the advantages,challenges,and opportunities brought by deep learning.We hope that deep learning can aid in eliminating factors that hinder the development of EMG-based HMI systems.Furthermore,possible future directions will be presented to pave the way for future research.

Fully Fabric-Based Triboelectric Nanogenerators as Self-Powered Human-Machine Interactive Keyboards

Combination flexible and stretchable textiles with self-powered sensors bring a novel insight into wearable functional electronics and cyber security in the era of Internet of Things.This work presents a highly flexible and self-powered fully fabric-based triboelectric nanogenerator(F-TENG)with sandwiched structure for biomechanical energy harvesting and real-time biometric authentication.The prepared F-TENG can power a digital watch by low-frequency motion and respond to the pressure change by the fall of leaves.A self-powered wearable keyboard(SPWK)is also fabricated by integrating large-area F-TENG sensor arrays,which not only can trace and record electrophysiological signals,but also can identify individuals’typing characteristics by means of the Haar wavelet.Based on these merits,the SPWK has promising applications in the realm of wearable electronics,self-powered sensors,cyber security,and artificial intelligences.

Human-machine compatibility and dynamic analysis of a novel unpowered and self-adaptive shoulder rehabilitation exoskeleton

To reduce the complexity of the configuration and control strategy for shoulder rehabilitation exoskeleton,a 2R1R1P2R serial of shoulder exoskeleton based on gravity balance is proposed.Based on three basic rotatory shoulder joints,an exact kinematic constraint system can be formed between the exoskeleton and the upper arm by introducing a passive sliding pair and a center of glenohumeral(CGH)unpowered compensation mechanism,which realizes the human-machine kinematic compatibility.Gravity balance is used in the CGH compensation mechanism to provide shoulder joint support.Meanwhile,the motion of the compensation mechanism is pulled by doing reverse leading through the arm to realize the kinematic self-adaptive,which decreases control complexity.Besides,a simple and intuitive spring adjustment strategy is proposed to ensure the gravity balance of any prescribed quality.Furthermore,according to the influencing factors analysis of the scapulohumeral rhythm,the kinematic analysis of CGH mechanism is performed,which shows that the mechanism can fit the trajectory of CGH under various conditions.Finally,the dynamic simulation of the mechanism is carried out.Results indicate that the compensation torques are reduced to below 0.22 N·m,and the feasibility of the mechanism is also verified.

Web GIS Human-Machine Interactive Interface Design with VISI

The fusion of VlSI （visual identity system Internet）, digital maps and Web GIS is presented. Web GIS interface interactive design with VISI needs to consider more new factors. VISI can provide the design principle, elements and contents for the Web GIS. The design of the Wuhan Bus Search System is fulfilled to confirm the validity and practicability of the fusion.

BLIND SPEECH SEPARATION FOR ROBOTS WITH INTELLIGENT HUMAN-MACHINE INTERACTION

Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.

A Robotic Teleoperation System for Precise Robotic Manipulation by Human-Machine Interaction

Teleoperation is of great importance in the area of robotics,especially when people are unavailable in the robot workshop.It provides a way for people to control robots remotely using human intelligence.In this paper,a robotic teleoperation system for precise robotic manipulation is established.The data glove and the 7-degrees of freedom(DOFs)force feedback controller are used for the remote control interaction.The control system and the monitor system are designed for the remote precise manipulation.The monitor system contains an image acquisition system and a human-machine interaction module,and aims to simulate and detect the robot running state.Besides,a visual object tracking algorithm is developed to estimate the states of the dynamic system from noisy observations.The established robotic teleoperation systemis applied to a series of experiments,and high-precision results are obtained,showing the effectiveness of the physical system.

Designing interactive glazing through an engineering psychology approach:Six augmented reality scenarios that envision future car human-machine interface

Background With an increasing number of vehicles becoming autonomous,intelligent,and connected,paying attention to the future usage of car human-machine interface with these vehicles should become more relevant.Several studies have addressed car HMI but were less attentive to designing and implementing interactive glazing for every day(autonomous)driving contexts.Methods Reflecting on the literature,we describe an engineering psychology practice and the design of six novel future user scenarios,which envision the application of a specific set of augmented reality(AR)support user interactions.Additionally,we conduct evaluations on specific scenarios and experiential prototypes,which reveal that these AR scenarios aid the target user groups in experiencing a new type of interaction.The overall evaluation is positive with valuable assessment results and suggestions.Conclusions This study can interest applied psychology educators who aspire to teach how AR can be operationalized in a human-centered design process to students with minimal pre-existing expertise or minimal scientific knowledge in engineering psychology.

The Human-Machine Safety Design of Mechanical Toys for Children

In order to ensure the safety of children while using machineries and avoid harm caused by mechanical toys,this paper analyzes the types and detection standards of children’s toys,discusses the reasons for the harm caused by these toys,and proposes human-machine safety design strategies for children’s toys as reference.

Framework and Key Technologies of Human-machine Hybrid-augmented Intelligence System for Large-scale Power Grid Dispatching and Control

With integration of large-scale renewable energy,new controllable devices,and required reinforcement of power grids,modern power systems have typical characteristics such as uncertainty,vulnerability and openness,which makes operation and control of power grids face severe security challenges.Application of artificial intelligence(AI)technologies represented by machine learning in power grid regulation is limited by reliability,interpretability and generalization ability of complex modeling.Mode of hybrid-augmented intelligence(HAI)based on human-machine collaboration(HMC)is a pivotal direction for future development of AI technology in this field.Based on characteristics of applications in power grid regulation,this paper discusses system architecture and key technologies of human-machine hybrid-augmented intelligence(HHI)system for large-scale power grid dispatching and control(PGDC).First,theory and application scenarios of HHI are introduced and analyzed;then physical and functional architectures of HHI system and human-machine collaborative regulation process are proposed.Key technologies are discussed to achieve a thorough integration of human/machine intelligence.Finally,state-of-theart and future development of HHI in power grid regulation are summarized,aiming to efficiently improve the intelligent level of power grid regulation in a human-machine interactive and collaborative way.

Flexible and multifunctional triboelectric nanogenerator based on liquid metal/polyvinyl alcohol hydrogel for energy harvesting and self-powered wearable human-machine interaction

Hydrogel-based triboelectric nanoge nerator(TENG)has a promising applied prospect in wearable electronic devices.However,its low performance,poor stability,insufficient recyclability and inferior self-healing seriously hinder its development.Herein,we report a robust route to a liquid metal(LM)/polyvinyl alcohol(PVA)hydrogel-based TENG(LP-TENG).Owing to the intrinsically liquid feature of conductive LM within the flexible PVA hydrogel,the as-prepared LP-TENG exhibited comprehensiye advantages of adaptability,biocompatibility,outstanding electrical performance,superior stability,recyclability and diverse applications,which were unattainable by traditional systems.Concretely,the LP-TENG delivered appealing open circuit voltage of 250 V,short circuit current of 4μA and transferred charge of 120 nC with high stability,outperforming most advanced TENG systems.The LP-TENG was successfully employed for versatile applications with multifunctionality,including human motion detection,handwriting recognition,energy collection,message transmission and human-machine interaction.This work presents significant prospects for crafting advanced materials and devices in the fields of wearable electronics,flexible skin and smart robots.

Human-Machine Shared Lateral Control Strategy for Intelligent Vehicles Based on Human Driver Risk Perception Reliability

Intelligent vehicle(Ⅳ)technology has developed rapidly in recent years.However,achieving fully unmanned driving still presents numerous challenges,which means that human drivers will continue to play a vital role in vehicle operation for the foreseeable future.Human-machine shared driving,involving cooperation between a human driver and an automated driving system(AVS),has been widely regarded as a necessary stage for the development of IVs.Focusing onⅣdriving safety,this study proposed a human-machine shared lateral control strategy(HSLCS)based on the reliability of driver risk perception.The HSLCS starts by identifying the effective areas of driver risk perception based on eye movements.It establishes an anisotropic driving risk field,which serves as the foundation for the AVS to assess risk levels.Building upon the cumulative and diminishing effects of risk perception,the proposed approach leverages the driver's risk perception effective area and converts the risk field into a representation aligned with the driver's perspective.Subsequently,it quantifies the reliability of the driver's risk perception by using area-matching rules.Finally,based on the driver’s risk perception reliability and dif-ferences in lateral driving operation between the human driver and the AVS,the dynamic distribution of driving authority is achieved through a fuzzy rule-based system,and the human-machine shared lateral control is completed by using model predictive control.The HSLCS was tested across various scenarios on a driver-in-the-loop test platform.The results show that the HSLCS can realize the synergy and complementarity of human and machine intelligence,effectively ensuring the safety ofⅣoperation.

Challenges of human-machine collaboration in risky decision-making

The purpose of this paper is to delineate the research challenges of human-machine collaboration in risky decision-making.Technological advances in machine intelligence have enabled a growing number of applications in human-machine collaborative decisionmaking.Therefore,it is desirable to achieve superior performance by folly leveraging human and machine capabilities.In risky decision-making,a human decisionmaker is vulnerable to cognitive biases when judging the possible outcomes of a risky event,whereas a machine decision-maker cannot handle new and dynamic contexts with incomplete information well.We first summarize features of risky decision-making and possible biases of human decision-makers therein.Then,we argue the necessity and urgency of advancing human-machine collaboration in risky decision-making.Afterward,we review the literature on human-machine collaboration in a general decision context,from the perspectives of human-machine organization,relationship,and collaboration.Lastly,we propose challenges of enhancing human-machine communication and teamwork in risky decisionmaking,followed by future research avenues.

Human-like lane-changing trajectory planning algorithm for human-machine conflict mitigation

The purpose of this paper is to alleviate the potential safety problems associated with the human driver and the automatic system competing for the right of way due to different objectives by mitigating the human-machine conflict phenomenon in human-machine shared driving(HMSD)technology from the automation system.Firstly,a basic lane-changing trajectory algorithm based on the quintic polynomial in the Frenet coordinate system is developed.Then,in order to make the planned trajectory close to human behavior,naturalistic driving data is collected,based on which some lane-changing performance features are selected and analyzed.There are three aspects have been taken into consideration for the human-like lane-changing trajectory:vehicle dynamic stability performance,driving cost optimization,and collision avoidance.Finally,the HMSD experiments are conducted with the driving simulator to test the potential of the human-like lane-changing trajectory planning algorithm.The results demonstrate that the lane-changing trajectory planning algorithm with the highest degree of personalization is highly consistent with human driver behavior and consequently would potentially mitigate the human-machine conflict with the HMSD application.Furthermore,it could be further employed as an empirical trajectory prediction result.The algorithm employs the distribution state of the historical trajectory for human-like processing,simplifying the operational process and ensuring the credibility,integrity,and interpretability of the results.Moreover,in terms of optimization processing,the form of optimization search followed by collision avoidance detection is adopted to in principle reduce the calculation difficulty.Additionally,a new convex polygon collision detection method,namely the vertex embedding method,is proposed for collision avoidance detection.

Optical Micro/Nano Fibers Enabled Smart Textiles for Human-Machine Interface

Wearable human-machine interface(HMI)is an advanced technology that has a wide range of applications from robotics to augmented/virtual reality(AR/VR).In this study,an optically driven wearable human-interactive smart textile is proposed by integrating a polydimethylsiloxane(PDMS)patch embedded with optical micro/nanofibers(MNF)array with a piece of textiles.Enabled by the highly sensitive pressure dependent bending loss of MNF,the smart textile shows high sensitivity(65.5 kPa^(−1))and fast response(25 ms)for touch sensing.Benefiting from the warp and weft structure of the textile,the optical smart textile can feel slight finger slip along the MNF.Furthermore,machine learning is utilized to classify the touch manners,achieving a recognition accuracy as high as 98.1%.As a proof-of-concept,a remote-control robotic hand and a smart interactive doll are demonstrated based on the optical smart textile.This optical smart textile represents an ideal HMI for AR/VR and robotics applications.

An environment-stable hydrogel with skin-matchable performance for human-machine interface

Ionic hydrogel-based sensors have shined a spotlight on wearable electronics.However,the sensitivity and reliability of hydrogel devices are significantly hampered by the weak adhesion of skin-sensor interface as well as inferior temperature tolerance.Here,inspired by the structure and composition of dermis,a novel skin-attachable and environment-stable hydrogel was designed by integrating collagen into the LiCl-containing chemically cross-linked polyacrylamide hydrogel.The hydrogel exhibited skin-like mechanical properties of low modulus,superior stretchability as well as excellent elasticity.Furthermore,the introduction of collagen endowed the hydrogel with robust and seamless interfaces with diverse materials,including the curved skin.As a result,the hydrogel is capable of serving as a human-machine interface for collecting reliable electrocardiography(ECG)signals and discerning various human motions,with high sensitivity(gauge factor=10.7),fast response,negligible hysteresis as well as extensive monitoring range.Notably,the hydrogel that can mimick the temperature-tolerant mechanism of most organisms possesses persistent stabilization of adhesive,conductive,sensory and mechanical performances at subzero or ambient conditions.The skin-inspired strategy paves an effective way for the design of multifunctional materials with potential applications in next-generation electronics.

Mutually trustworthy human-machine knowledge automation and hybrid augmented intelligence:mechanisms and applications of cognition,management,and control for complex systems

In this paper,we aim to illustrate the concept of mutually trustworthy human-machine knowledge automation(HM-KA)as the technical mechanism of hybrid augmented intelligence(HAI)based complex system cognition,management,and control(CMC).We describe the historical development of complex system science and analyze the limitations of human intelligence and machine intelligence.The need for using human-machine HAI in complex systems is then explained in detail.The concept of“mutually trustworthy HM-KA”mechanism is proposed to tackle the CMC challenge,and its technical procedure and pathway are demonstrated using an example of corrective control in bulk power grid dispatch.It is expected that the proposed mutually trustworthy HM-KA concept can provide a novel and canonical mechanism and benefit real-world practices of complex system CMC.

Design issues for human-machine platform interface in cable-based parallel manipulators for physiotherapy applications

We outline problems and potential solutions for feasible human-machine interfaces using cable-based parallel manipulators for physiotherapy applications.From an engineering perspective,we discuss the design constraints related to acceptance by patients and physiotherapist users.To date,most designs have focused on mobile platforms that are designed to be operated as an end-effector connected to human limbs for direct patient interaction.Some specific examples are illustrated from the authors' experience with prototypes available at Laboratory of Robotics and Mechatronics (LARM),Italy.

Human-machine interaction force control: using a model-referenced adaptive impedance device to control an index finger exoskeleton

Exoskeleton robots and their control methods have been extensively developed to aid post-stroke rehabilitation. Most of the existing methods using linear controllers are designed for position control and are not suitable for human-machine interaction(HMI) force control, as the interaction system between the human body and exoskeleton is uncertain and nonlinear. We present an approach for HMI force control via model reference adaptive impedance control(MRAIC) to solve this problem in case of index finger exoskeleton control. First, a dynamic HMI model, which is based on a position control inner loop, is formulated. Second, the theoretical MRAC framework is implemented in the control system. Then, the adaptive controllers are designed according to the Lyapunov stability theory. To verify the performance of the proposed method, we compare it with a proportional-integral-derivative(PID) method in the time domain with real experiments and in the frequency domain with simulations. The results illustrate the effectiveness and robustness of the proposed method in solving the nonlinear HMI force control problem in hand exoskeleton.

Swarm Intelligence Research:From Bio-inspired Single-population Swarm Intelligence to Human-machine Hybrid Swarm Intelligence

Swarm intelligence has become a hot research field of artificial intelligence.Considering the importance of swarm intelli-gence for the future development of artificial intelligence,we discuss and analyze swarm intelligence from a broader and deeper perspect-ive.In a broader sense,we are talking about not only bio-inspired swarm intelligence,but also human-machine hybrid swarm intelli-gence.In a deeper sense,we discuss the research using a three-layer hierarchy:in the first layer,we divide the research of swarm intelli-gence into bio-inspired swarm intelligence and human-machine hybrid swarm intelligence;in the second layer,the bio-inspired swarm intelligence is divided into single-population swarm intelligence and multi-population swarm intelligence;and in the third layer,we re-view single-population,multi-population and human-machine hybrid models from different perspectives.Single-population swarm intel-ligence is inspired by biological intelligence.To further solve complex optimization problems,researchers have made preliminary explor-ations in multi-population swarm intelligence.However,it is difficult for bio-inspired swarm intelligence to realize dynamic cognitive in-telligent behavior that meets the needs of human cognition.Researchers have introduced human intelligence into computing systems and proposed human-machine hybrid swarm intelligence.In addition to single-population swarm intelligence,we thoroughly review multi-population and human-machine hybrid swarm intelligence in this paper.We also discuss the applications of swarm intelligence in optimization,big data analysis,unmanned systems and other fields.Finally,we discuss future research directions and key issues to be studied in swarm intelligence.