Digital Twin Technology of Human-Machine Integration in Cross-Belt Sorting System

The Chinese express delivery industry processes nearly 110 billion items in 2022,averaging an annual growth rate of 200%.Among the various types of sorting systems used for handling express items,cross-belt sorting systems stand out as the most crucial.However,despite their high degree of automation,the workload for operators has intensified owing to the surging volume of express items.In the era of Industry 5.0,it is imperative to adopt new technologies that not only enhance worker welfare but also improve the efficiency of cross-belt systems.Striking a balance between efficiency in handling express items and operator well-being is challenging.Digital twin technology offers a promising solution in this respect.A realization method of a human-machine integrated digital twin is proposed in this study,enabling the interaction of biological human bodies,virtual human bodies,virtual equipment,and logistics equipment in a closed loop,thus setting an operating framework.Key technologies in the proposed framework include a collection of heterogeneous data from multiple sources,construction of the relationship between operator fatigue and operation efficiency based on physiological measurements,virtual model construction,and an online optimization module based on real-time simulation.The feasibility of the proposed method was verified in an express distribution center.

Designing interactive glazing through an engineering psychology approach:Six augmented reality scenarios that envision future car human-machine interface

Background With an increasing number of vehicles becoming autonomous,intelligent,and connected,paying attention to the future usage of car human-machine interface with these vehicles should become more relevant.Several studies have addressed car HMI but were less attentive to designing and implementing interactive glazing for every day(autonomous)driving contexts.Methods Reflecting on the literature,we describe an engineering psychology practice and the design of six novel future user scenarios,which envision the application of a specific set of augmented reality(AR)support user interactions.Additionally,we conduct evaluations on specific scenarios and experiential prototypes,which reveal that these AR scenarios aid the target user groups in experiencing a new type of interaction.The overall evaluation is positive with valuable assessment results and suggestions.Conclusions This study can interest applied psychology educators who aspire to teach how AR can be operationalized in a human-centered design process to students with minimal pre-existing expertise or minimal scientific knowledge in engineering psychology.

Deep Learning for EMG-based Human-Machine Interaction:A Review

Electromyography(EMG)has already been broadly used in human-machine interaction(HMI)applications.Determining how to decode the information inside EMG signals robustly and accurately is a key problem for which we urgently need a solution.Recently,many EMG pattern recognition tasks have been addressed using deep learning methods.In this paper,we analyze recent papers and present a literature review describing the role that deep learning plays in EMG-based HMI.An overview of typical network structures and processing schemes will be provided.Recent progress in typical tasks such as movement classification,joint angle prediction,and force/torque estimation will be introduced.New issues,including multimodal sensing,inter-subject/inter-session,and robustness toward disturbances will be discussed.We attempt to provide a comprehensive analysis of current research by discussing the advantages,challenges,and opportunities brought by deep learning.We hope that deep learning can aid in eliminating factors that hinder the development of EMG-based HMI systems.Furthermore,possible future directions will be presented to pave the way for future research.

Fully Fabric-Based Triboelectric Nanogenerators as Self-Powered Human-Machine Interactive Keyboards

Combination flexible and stretchable textiles with self-powered sensors bring a novel insight into wearable functional electronics and cyber security in the era of Internet of Things.This work presents a highly flexible and self-powered fully fabric-based triboelectric nanogenerator(F-TENG)with sandwiched structure for biomechanical energy harvesting and real-time biometric authentication.The prepared F-TENG can power a digital watch by low-frequency motion and respond to the pressure change by the fall of leaves.A self-powered wearable keyboard(SPWK)is also fabricated by integrating large-area F-TENG sensor arrays,which not only can trace and record electrophysiological signals,but also can identify individuals’typing characteristics by means of the Haar wavelet.Based on these merits,the SPWK has promising applications in the realm of wearable electronics,self-powered sensors,cyber security,and artificial intelligences.

BLIND SPEECH SEPARATION FOR ROBOTS WITH INTELLIGENT HUMAN-MACHINE INTERACTION

Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.

A Robotic Teleoperation System for Precise Robotic Manipulation by Human-Machine Interaction

Teleoperation is of great importance in the area of robotics,especially when people are unavailable in the robot workshop.It provides a way for people to control robots remotely using human intelligence.In this paper,a robotic teleoperation system for precise robotic manipulation is established.The data glove and the 7-degrees of freedom(DOFs)force feedback controller are used for the remote control interaction.The control system and the monitor system are designed for the remote precise manipulation.The monitor system contains an image acquisition system and a human-machine interaction module,and aims to simulate and detect the robot running state.Besides,a visual object tracking algorithm is developed to estimate the states of the dynamic system from noisy observations.The established robotic teleoperation systemis applied to a series of experiments,and high-precision results are obtained,showing the effectiveness of the physical system.

The Human-Machine Safety Design of Mechanical Toys for Children

In order to ensure the safety of children while using machineries and avoid harm caused by mechanical toys,this paper analyzes the types and detection standards of children’s toys,discusses the reasons for the harm caused by these toys,and proposes human-machine safety design strategies for children’s toys as reference.

Flexible and multifunctional triboelectric nanogenerator based on liquid metal/polyvinyl alcohol hydrogel for energy harvesting and self-powered wearable human-machine interaction

Hydrogel-based triboelectric nanoge nerator(TENG)has a promising applied prospect in wearable electronic devices.However,its low performance,poor stability,insufficient recyclability and inferior self-healing seriously hinder its development.Herein,we report a robust route to a liquid metal(LM)/polyvinyl alcohol(PVA)hydrogel-based TENG(LP-TENG).Owing to the intrinsically liquid feature of conductive LM within the flexible PVA hydrogel,the as-prepared LP-TENG exhibited comprehensiye advantages of adaptability,biocompatibility,outstanding electrical performance,superior stability,recyclability and diverse applications,which were unattainable by traditional systems.Concretely,the LP-TENG delivered appealing open circuit voltage of 250 V,short circuit current of 4μA and transferred charge of 120 nC with high stability,outperforming most advanced TENG systems.The LP-TENG was successfully employed for versatile applications with multifunctionality,including human motion detection,handwriting recognition,energy collection,message transmission and human-machine interaction.This work presents significant prospects for crafting advanced materials and devices in the fields of wearable electronics,flexible skin and smart robots.

Recent advances in stretchable triboelectric nanogenerators for use in wearable bioelectronic devices

Wearable bioelectronic devices have the capacity for real-time human health monitoring,the provision of tailored services,and natural interaction with smart devices.However,these wearable bioelectronic devices rely on conventional rigid batteries that are frequently charged or replaced and are incompatible with the skin,leading to a discontinuity in complex therapeutic tasks related to human health monitoring and human-machine interaction.Stretchable triboelectric nanogenerator(TENG)is a high-efficiency energy harvesting technology that converts mechanical into electrical energy,effectively powering wearable bioelectronic devices.This study comprehensively overviews recent advances in stretchable TENG for use in wearable bioelectronic devices.The working mechanism of stretchable TENG is initially explained.A comprehensive discussion presents the approaches for fabricating stretchable TENG,including the design of stretchable structures and the selection of stretchable materials.Furthermore,applications of wearable bioelectronic devices based on stretchable TENG in human health monitoring(body movements,pulse,and respiration)and human-machine interaction(touch panels,machine control,and virtual reality)are introduced.Ultimately,the challenges and developmental trends regarding wearable bioelectronic devices based on stretchable TENG are elaborated.

Framework and Key Technologies of Human-machine Hybrid-augmented Intelligence System for Large-scale Power Grid Dispatching and Control

With integration of large-scale renewable energy,new controllable devices,and required reinforcement of power grids,modern power systems have typical characteristics such as uncertainty,vulnerability and openness,which makes operation and control of power grids face severe security challenges.Application of artificial intelligence(AI)technologies represented by machine learning in power grid regulation is limited by reliability,interpretability and generalization ability of complex modeling.Mode of hybrid-augmented intelligence(HAI)based on human-machine collaboration(HMC)is a pivotal direction for future development of AI technology in this field.Based on characteristics of applications in power grid regulation,this paper discusses system architecture and key technologies of human-machine hybrid-augmented intelligence(HHI)system for large-scale power grid dispatching and control(PGDC).First,theory and application scenarios of HHI are introduced and analyzed;then physical and functional architectures of HHI system and human-machine collaborative regulation process are proposed.Key technologies are discussed to achieve a thorough integration of human/machine intelligence.Finally,state-of-theart and future development of HHI in power grid regulation are summarized,aiming to efficiently improve the intelligent level of power grid regulation in a human-machine interactive and collaborative way.

Human-Machine Shared Lateral Control Strategy for Intelligent Vehicles Based on Human Driver Risk Perception Reliability

Intelligent vehicle(Ⅳ)technology has developed rapidly in recent years.However,achieving fully unmanned driving still presents numerous challenges,which means that human drivers will continue to play a vital role in vehicle operation for the foreseeable future.Human-machine shared driving,involving cooperation between a human driver and an automated driving system(AVS),has been widely regarded as a necessary stage for the development of IVs.Focusing onⅣdriving safety,this study proposed a human-machine shared lateral control strategy(HSLCS)based on the reliability of driver risk perception.The HSLCS starts by identifying the effective areas of driver risk perception based on eye movements.It establishes an anisotropic driving risk field,which serves as the foundation for the AVS to assess risk levels.Building upon the cumulative and diminishing effects of risk perception,the proposed approach leverages the driver's risk perception effective area and converts the risk field into a representation aligned with the driver's perspective.Subsequently,it quantifies the reliability of the driver's risk perception by using area-matching rules.Finally,based on the driver’s risk perception reliability and dif-ferences in lateral driving operation between the human driver and the AVS,the dynamic distribution of driving authority is achieved through a fuzzy rule-based system,and the human-machine shared lateral control is completed by using model predictive control.The HSLCS was tested across various scenarios on a driver-in-the-loop test platform.The results show that the HSLCS can realize the synergy and complementarity of human and machine intelligence,effectively ensuring the safety ofⅣoperation.

Machine learning and human‐machine trust in healthcare:A systematic survey

As human‐machine interaction(HMI)in healthcare continues to evolve,the issue of trust in HMI in healthcare has been raised and explored.It is critical for the development and safety of healthcare that humans have proper trust in medical machines.Intelligent machines that have applied machine learning(ML)technologies continue to penetrate deeper into the medical environment,which also places higher demands on intelligent healthcare.In order to make machines play a role in HMI in healthcare more effectively and make human‐machine cooperation more harmonious,the authors need to build good humanmachine trust(HMT)in healthcare.This article provides a systematic overview of the prominent research on ML and HMT in healthcare.In addition,this study explores and analyses ML and three important factors that influence HMT in healthcare,and then proposes a HMT model in healthcare.Finally,general trends are summarised and issues to consider addressing in future research on HMT in healthcare are identified.

Transparent,Ultra-Stretching,Tough,Adhesive Carboxyethyl Chitin/Polyacrylamide Hydrogel Toward High-Performance Soft Electronics

To date,hydrogels have gained increasing attentions as a flexible conductive material in fabricating soft electronics.However,it remains a big challenge to integrate multiple functions into one gel that can be used widely under various conditions.Herein,a kind of multifunc-tional hydrogel with a combination of desirable characteristics,including remarkable transparency,high conductivity,ultra-stretchability,tough-ness,good fatigue resistance,and strong adhesive ability is presented,which was facilely fabricated through multiple noncovalent crosslinking strategy.The resultant versatile sensors are able to detect both weak and large deformations,which owns a low detection limit of 0.1%strain,high stretchability up to 1586%,ultrahigh sensitivity with a gauge factor up to 18.54,as well as wide pressure sensing range(0-600 kPa).Meanwhile,the fabrication of conductive hydrogel-based sensors is demonstrated for various soft electronic devices,including a flexible human-machine interactive system,the soft tactile switch,an integrated electronic skin for unprecedented nonplanar visualized pressure sensing,and the stretchable triboelectric nanogenerators with excellent biomechanical energy harvesting ability.This work opens up a simple route for multifunctional hydrogel and promises the practical application of soft and self-powered wearable electronics in various complex scenes.

Swarm Intelligence Research:From Bio-inspired Single-population Swarm Intelligence to Human-machine Hybrid Swarm Intelligence

Swarm intelligence has become a hot research field of artificial intelligence.Considering the importance of swarm intelli-gence for the future development of artificial intelligence,we discuss and analyze swarm intelligence from a broader and deeper perspect-ive.In a broader sense,we are talking about not only bio-inspired swarm intelligence,but also human-machine hybrid swarm intelli-gence.In a deeper sense,we discuss the research using a three-layer hierarchy:in the first layer,we divide the research of swarm intelli-gence into bio-inspired swarm intelligence and human-machine hybrid swarm intelligence;in the second layer,the bio-inspired swarm intelligence is divided into single-population swarm intelligence and multi-population swarm intelligence;and in the third layer,we re-view single-population,multi-population and human-machine hybrid models from different perspectives.Single-population swarm intel-ligence is inspired by biological intelligence.To further solve complex optimization problems,researchers have made preliminary explor-ations in multi-population swarm intelligence.However,it is difficult for bio-inspired swarm intelligence to realize dynamic cognitive in-telligent behavior that meets the needs of human cognition.Researchers have introduced human intelligence into computing systems and proposed human-machine hybrid swarm intelligence.In addition to single-population swarm intelligence,we thoroughly review multi-population and human-machine hybrid swarm intelligence in this paper.We also discuss the applications of swarm intelligence in optimization,big data analysis,unmanned systems and other fields.Finally,we discuss future research directions and key issues to be studied in swarm intelligence.

AI Safety Approach for Minimizing Collisions in Autonomous Navigation

Autonomous agents can explore the environment around them when equipped with advanced hardware and software systems that help intelligent agents minimize collisions.These systems are developed under the term Artificial Intelligence(AI)safety.AI safety is essential to provide reliable service to consumers in various fields such asmilitary,education,healthcare,and automotive.This paper presents the design of an AI safety algorithmfor safe autonomous navigation using Reinforcement Learning(RL).Machine Learning Agents Toolkit(ML-Agents)was used to train the agentwith a proximal policy optimizer algorithmwith an intrinsic curiositymodule(PPO+ICM).This training aims to improve AI safety and minimize or prevent any mistakes that can cause dangerous collisions by the intelligent agent.Four experiments have been executed to validate the results of our research.The designed algorithmwas tested in a virtual environment with four differentmodels.A comparison was presented in four cases to identify the best-performing model for improvingAI safety.The designed algorithmenabled the intelligent agent to perform the required task safely using RL.A goal collision ratio of 64%was achieved,and the collision incidents were minimized from 134 to 52 in the virtual environment within 30min.

Human-machine interactive streaming anomaly detection by online self-adaptive forest

At nomaly detectors are used to distinguish differences between normal and abnormal data,which are usually implemented by evaluating and ranking the anomaly scores of each instance.A static unsupervised streaming anomaly detector is difficult to dynamically adjust anomaly score calculation.In real scenarios,anomaly detection often needs to be regulated by human feedback,which benefits adjusting anomaly detectors.In this paper,we propose a human-machine interactive streaming anomaly detection method,named ISPForest,which can be adaptively updated online under the guidance of human feedback.In particular,the feedback will be used to adjust the anomaly score calculation and structure of the detector,ideally attaining more accurate anomaly scores in the future.Our main contribution is to improve the tree-based streaming anomaly detection model that can be updated online from perspectives of anomaly score calculation and model structure.Our approach is instantiated for the powerful class of tree-based streaming anomaly detectors,and we conduct experiments on a range of benchmark datasets.The results demonstrate that the utility of incorporating feedback can improve the performance of anomaly detectors with a few human efforts.

Split-Ring Structured All-Inorganic Perovskite Photodetector Arrays for Masterly Internet of Things

Photodetectors with long detection distances and fast response are important media in constructing a non-contact human-machine interface for the Masterly Internet of Things(MIT).All-inorganic perovskites have excellent optoelectronic performance with high moisture and oxygen resistance,making them one of the promising candidates for high-performance photodetectors,but a simple,low-cost and reliable fabrication technology is urgently needed.Here,a dual-function laser etching method is developed to complete both the lyophilic split-ring structure and electrode patterning.This novel split-ring structure can capture the perovskite precursor droplet efficiently and achieve the uniform and compact deposition of CsPbBr3 films.Furthermore,our devices based on laterally conducting split-ring structured photodetectors possess outstanding performance,including the maximum responsivity of 1.44×105 mA W^(−1),a response time of 150μs in 1.5 kHz and one-unit area<4×10-2 mm2.Based on these split-ring photodetector arrays,we realized three-dimensional gesture detection with up to 100 mm distance detection and up to 600 mm s^(−1) speed detection,for low-cost,integrative,and non-contact human-machine interfaces.Finally,we applied this MIT to wearable and flexible digital gesture recognition watch panel,safe and comfortable central controller integrated on the car screen,and remote control of the robot,demonstrating the broad potential applications.

Configuration Design and Kinematic Performance Analysis of a Novel 4-DOF Parallel Ankle Rehabilitation Mechanism with Two Virtual Motion Centers

Aiming at the problem that the existing ankle rehabilitation robot is difficult to fully fit the complex motion of human ankle joint and has poor human-machine motion compatibility,an equivalent series mechanism model that is highly matched with the actual bone structure of the human ankle joint is proposed and mapped into a parallel rehabilita-tion mechanism.The parallel rehabilitation mechanism has two virtual motion centers(VMCs),which can simulate the complex motion of the ankle joint,adapt to the individual differences of various patients,and can meet the reha-bilitation needs of both left and right feet of patients.Firstly,based on the motion properties and physiological structure of the human ankle joint,the mapping relationship between the rehabilitation mechanism and ankle joint is determined,and the series equivalent model of the ankle joint is established.According to the kinematic and con-straint properties of the ankle equivalent model,the configuration design of the parallel ankle rehabilitation robot is carried out.Secondly,according to the intersecting motion planes theory,the full-cycle mobility of the mechanism is proved,and the continuous axis of the mechanism is judged based on the constraint power and its derivative.Then,the kinematics of the parallel ankle rehabilitation robot is analyzed.Finally,based on the OpenSim biomechanical soft-ware,a human-machine coupling rehabilitation simulation model is established to evaluate the rehabilitation effect,which lays the foundation for the formulation of a rehabilitation strategy for the later prototype.

Skin‑Inspired Ultra‑Tough Supramolecular Multifunctional Hydrogel Electronic Skin for Human–Machine Interaction

Multifunctional supramolecular ultra-tough bionic e-skin with unique durability for human–machine interaction in complex scenarios still remains challenging.Herein,we develop a skininspired ultra-tough e-skin with tunable mechanical properties by a physical cross-linking salting-freezing-thawing method.The gelling agent(β-Glycerophosphate sodium:Gp)induces the aggregation and binding of PVA molecular chains and thereby toughens them(stress up to 5.79 MPa,toughness up to 13.96 MJ m^(−3)).Notably,due to molecular self-assembly,hydrogels can be fully recycled and reprocessed by direct heating(100°C for a few seconds),and the tensile strength can still be maintained at about 100%after six recoveries.The hydrogel integrates transparency(>60%),super toughness(up to 13.96 MJ m^(−3),bearing 1500 times of its own tensile weight),good antibacterial properties(E.coli and S.aureus),UV protection(Filtration:80%–90%),high electrical conductivity(4.72 S m^(−1)),anti-swelling and recyclability.The hydrogel can not only monitor daily physiological activities,but also be used for complex activities underwater and message encryption/decryption.We also used it to create a complete finger joint rehabilitation system with an interactive interface that dynamically presents the user’s health status.Our multifunctional electronic skin will have a profound impact on the future of new rehabilitation medical,human–machine interaction,VR/AR and the metaverse fields.

Deep-reinforcement-learning-based robot motion strategies for grabbing objects from human hands

Background Robot grasping encompasses a wide range of research areas;however, most studies have been focused on the grasping of only stationary objects in a scene;only a few studies on how to grasp objects from a user's hand have been conducted. In this paper, a robot grasping algorithm based on deep reinforcement learning (RGRL) is proposed. Methods The RGRL takes the relative positions of the robot and the object in a user's hand as input and outputs the best action of the robot in the current state. Thus, the proposed algorithm realizes the functions of autonomous path planning and grasping objects safely from the hands of users. A new method for improving the safety of human-robot cooperation is explored. To solve the problems of a low utilization rate and slow convergence of reinforcement learning algorithms, the RGRL is first trained in a simulation scene, and then, the model para-meters are applied to a real scene. To reduce the difference between the simulated and real scenes, domain randomization is applied to randomly change the positions and angles of objects in the simulated scenes at regular intervals, thereby improving the diversity of the training samples and robustness of the algorithm. Results The RGRL's effectiveness and accuracy are verified by evaluating it on both simulated and real scenes, and the results show that the RGRL can achieve an accuracy of more than 80% in both cases. Conclusions RGRL is a robot grasping algorithm that employs domain randomization and deep reinforcement learning for effective grasping in simulated and real scenes. However, it lacks flexibility in adapting to different grasping poses, prompting future research in achieving safe grasping for diverse user postures.