Designing interactive glazing through an engineering psychology approach:Six augmented reality scenarios that envision future car human-machine interface

Background With an increasing number of vehicles becoming autonomous,intelligent,and connected,paying attention to the future usage of car human-machine interface with these vehicles should become more relevant.Several studies have addressed car HMI but were less attentive to designing and implementing interactive glazing for every day(autonomous)driving contexts.Methods Reflecting on the literature,we describe an engineering psychology practice and the design of six novel future user scenarios,which envision the application of a specific set of augmented reality(AR)support user interactions.Additionally,we conduct evaluations on specific scenarios and experiential prototypes,which reveal that these AR scenarios aid the target user groups in experiencing a new type of interaction.The overall evaluation is positive with valuable assessment results and suggestions.Conclusions This study can interest applied psychology educators who aspire to teach how AR can be operationalized in a human-centered design process to students with minimal pre-existing expertise or minimal scientific knowledge in engineering psychology.

Deep Learning for EMG-based Human-Machine Interaction:A Review

Electromyography(EMG)has already been broadly used in human-machine interaction(HMI)applications.Determining how to decode the information inside EMG signals robustly and accurately is a key problem for which we urgently need a solution.Recently,many EMG pattern recognition tasks have been addressed using deep learning methods.In this paper,we analyze recent papers and present a literature review describing the role that deep learning plays in EMG-based HMI.An overview of typical network structures and processing schemes will be provided.Recent progress in typical tasks such as movement classification,joint angle prediction,and force/torque estimation will be introduced.New issues,including multimodal sensing,inter-subject/inter-session,and robustness toward disturbances will be discussed.We attempt to provide a comprehensive analysis of current research by discussing the advantages,challenges,and opportunities brought by deep learning.We hope that deep learning can aid in eliminating factors that hinder the development of EMG-based HMI systems.Furthermore,possible future directions will be presented to pave the way for future research.

Fully Fabric-Based Triboelectric Nanogenerators as Self-Powered Human-Machine Interactive Keyboards

Combination flexible and stretchable textiles with self-powered sensors bring a novel insight into wearable functional electronics and cyber security in the era of Internet of Things.This work presents a highly flexible and self-powered fully fabric-based triboelectric nanogenerator(F-TENG)with sandwiched structure for biomechanical energy harvesting and real-time biometric authentication.The prepared F-TENG can power a digital watch by low-frequency motion and respond to the pressure change by the fall of leaves.A self-powered wearable keyboard(SPWK)is also fabricated by integrating large-area F-TENG sensor arrays,which not only can trace and record electrophysiological signals,but also can identify individuals’typing characteristics by means of the Haar wavelet.Based on these merits,the SPWK has promising applications in the realm of wearable electronics,self-powered sensors,cyber security,and artificial intelligences.

BLIND SPEECH SEPARATION FOR ROBOTS WITH INTELLIGENT HUMAN-MACHINE INTERACTION

Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.

A Robotic Teleoperation System for Precise Robotic Manipulation by Human-Machine Interaction

Teleoperation is of great importance in the area of robotics,especially when people are unavailable in the robot workshop.It provides a way for people to control robots remotely using human intelligence.In this paper,a robotic teleoperation system for precise robotic manipulation is established.The data glove and the 7-degrees of freedom(DOFs)force feedback controller are used for the remote control interaction.The control system and the monitor system are designed for the remote precise manipulation.The monitor system contains an image acquisition system and a human-machine interaction module,and aims to simulate and detect the robot running state.Besides,a visual object tracking algorithm is developed to estimate the states of the dynamic system from noisy observations.The established robotic teleoperation systemis applied to a series of experiments,and high-precision results are obtained,showing the effectiveness of the physical system.

The Human-Machine Safety Design of Mechanical Toys for Children

In order to ensure the safety of children while using machineries and avoid harm caused by mechanical toys,this paper analyzes the types and detection standards of children’s toys,discusses the reasons for the harm caused by these toys,and proposes human-machine safety design strategies for children’s toys as reference.

Framework and Key Technologies of Human-machine Hybrid-augmented Intelligence System for Large-scale Power Grid Dispatching and Control

With integration of large-scale renewable energy,new controllable devices,and required reinforcement of power grids,modern power systems have typical characteristics such as uncertainty,vulnerability and openness,which makes operation and control of power grids face severe security challenges.Application of artificial intelligence(AI)technologies represented by machine learning in power grid regulation is limited by reliability,interpretability and generalization ability of complex modeling.Mode of hybrid-augmented intelligence(HAI)based on human-machine collaboration(HMC)is a pivotal direction for future development of AI technology in this field.Based on characteristics of applications in power grid regulation,this paper discusses system architecture and key technologies of human-machine hybrid-augmented intelligence(HHI)system for large-scale power grid dispatching and control(PGDC).First,theory and application scenarios of HHI are introduced and analyzed;then physical and functional architectures of HHI system and human-machine collaborative regulation process are proposed.Key technologies are discussed to achieve a thorough integration of human/machine intelligence.Finally,state-of-theart and future development of HHI in power grid regulation are summarized,aiming to efficiently improve the intelligent level of power grid regulation in a human-machine interactive and collaborative way.

Human-machine interactive streaming anomaly detection by online self-adaptive forest

At nomaly detectors are used to distinguish differences between normal and abnormal data,which are usually implemented by evaluating and ranking the anomaly scores of each instance.A static unsupervised streaming anomaly detector is difficult to dynamically adjust anomaly score calculation.In real scenarios,anomaly detection often needs to be regulated by human feedback,which benefits adjusting anomaly detectors.In this paper,we propose a human-machine interactive streaming anomaly detection method,named ISPForest,which can be adaptively updated online under the guidance of human feedback.In particular,the feedback will be used to adjust the anomaly score calculation and structure of the detector,ideally attaining more accurate anomaly scores in the future.Our main contribution is to improve the tree-based streaming anomaly detection model that can be updated online from perspectives of anomaly score calculation and model structure.Our approach is instantiated for the powerful class of tree-based streaming anomaly detectors,and we conduct experiments on a range of benchmark datasets.The results demonstrate that the utility of incorporating feedback can improve the performance of anomaly detectors with a few human efforts.

Swarm Intelligence Research:From Bio-inspired Single-population Swarm Intelligence to Human-machine Hybrid Swarm Intelligence

Swarm intelligence has become a hot research field of artificial intelligence.Considering the importance of swarm intelli-gence for the future development of artificial intelligence,we discuss and analyze swarm intelligence from a broader and deeper perspect-ive.In a broader sense,we are talking about not only bio-inspired swarm intelligence,but also human-machine hybrid swarm intelli-gence.In a deeper sense,we discuss the research using a three-layer hierarchy:in the first layer,we divide the research of swarm intelli-gence into bio-inspired swarm intelligence and human-machine hybrid swarm intelligence;in the second layer,the bio-inspired swarm intelligence is divided into single-population swarm intelligence and multi-population swarm intelligence;and in the third layer,we re-view single-population,multi-population and human-machine hybrid models from different perspectives.Single-population swarm intel-ligence is inspired by biological intelligence.To further solve complex optimization problems,researchers have made preliminary explor-ations in multi-population swarm intelligence.However,it is difficult for bio-inspired swarm intelligence to realize dynamic cognitive in-telligent behavior that meets the needs of human cognition.Researchers have introduced human intelligence into computing systems and proposed human-machine hybrid swarm intelligence.In addition to single-population swarm intelligence,we thoroughly review multi-population and human-machine hybrid swarm intelligence in this paper.We also discuss the applications of swarm intelligence in optimization,big data analysis,unmanned systems and other fields.Finally,we discuss future research directions and key issues to be studied in swarm intelligence.

Challenges of human-machine collaboration in risky decision-making

The purpose of this paper is to delineate the research challenges of human-machine collaboration in risky decision-making.Technological advances in machine intelligence have enabled a growing number of applications in human-machine collaborative decisionmaking.Therefore,it is desirable to achieve superior performance by folly leveraging human and machine capabilities.In risky decision-making,a human decisionmaker is vulnerable to cognitive biases when judging the possible outcomes of a risky event,whereas a machine decision-maker cannot handle new and dynamic contexts with incomplete information well.We first summarize features of risky decision-making and possible biases of human decision-makers therein.Then,we argue the necessity and urgency of advancing human-machine collaboration in risky decision-making.Afterward,we review the literature on human-machine collaboration in a general decision context,from the perspectives of human-machine organization,relationship,and collaboration.Lastly,we propose challenges of enhancing human-machine communication and teamwork in risky decisionmaking,followed by future research avenues.

Mutually trustworthy human-machine knowledge automation and hybrid augmented intelligence:mechanisms and applications of cognition,management,and control for complex systems

In this paper,we aim to illustrate the concept of mutually trustworthy human-machine knowledge automation(HM-KA)as the technical mechanism of hybrid augmented intelligence(HAI)based complex system cognition,management,and control(CMC).We describe the historical development of complex system science and analyze the limitations of human intelligence and machine intelligence.The need for using human-machine HAI in complex systems is then explained in detail.The concept of“mutually trustworthy HM-KA”mechanism is proposed to tackle the CMC challenge,and its technical procedure and pathway are demonstrated using an example of corrective control in bulk power grid dispatch.It is expected that the proposed mutually trustworthy HM-KA concept can provide a novel and canonical mechanism and benefit real-world practices of complex system CMC.

Optical Micro/Nano Fibers Enabled Smart Textiles for Human-Machine Interface

Wearable human-machine interface(HMI)is an advanced technology that has a wide range of applications from robotics to augmented/virtual reality(AR/VR).In this study,an optically driven wearable human-interactive smart textile is proposed by integrating a polydimethylsiloxane(PDMS)patch embedded with optical micro/nanofibers(MNF)array with a piece of textiles.Enabled by the highly sensitive pressure dependent bending loss of MNF,the smart textile shows high sensitivity(65.5 kPa^(−1))and fast response(25 ms)for touch sensing.Benefiting from the warp and weft structure of the textile,the optical smart textile can feel slight finger slip along the MNF.Furthermore,machine learning is utilized to classify the touch manners,achieving a recognition accuracy as high as 98.1%.As a proof-of-concept,a remote-control robotic hand and a smart interactive doll are demonstrated based on the optical smart textile.This optical smart textile represents an ideal HMI for AR/VR and robotics applications.

Design issues for human-machine platform interface in cable-based parallel manipulators for physiotherapy applications

We outline problems and potential solutions for feasible human-machine interfaces using cable-based parallel manipulators for physiotherapy applications.From an engineering perspective,we discuss the design constraints related to acceptance by patients and physiotherapist users.To date,most designs have focused on mobile platforms that are designed to be operated as an end-effector connected to human limbs for direct patient interaction.Some specific examples are illustrated from the authors' experience with prototypes available at Laboratory of Robotics and Mechatronics (LARM),Italy.

Human-machine interaction force control: using a model-referenced adaptive impedance device to control an index finger exoskeleton

Exoskeleton robots and their control methods have been extensively developed to aid post-stroke rehabilitation. Most of the existing methods using linear controllers are designed for position control and are not suitable for human-machine interaction(HMI) force control, as the interaction system between the human body and exoskeleton is uncertain and nonlinear. We present an approach for HMI force control via model reference adaptive impedance control(MRAIC) to solve this problem in case of index finger exoskeleton control. First, a dynamic HMI model, which is based on a position control inner loop, is formulated. Second, the theoretical MRAC framework is implemented in the control system. Then, the adaptive controllers are designed according to the Lyapunov stability theory. To verify the performance of the proposed method, we compare it with a proportional-integral-derivative(PID) method in the time domain with real experiments and in the frequency domain with simulations. The results illustrate the effectiveness and robustness of the proposed method in solving the nonlinear HMI force control problem in hand exoskeleton.

Stretchable human-machine interface based on skin-conformal sEMG electrodes with self-similar geometry

Current stretchable surface electrodes have attracted increasing attention owing to their potential applications in biological signal monitoring, wearable human-machine interfaces（HMIs） and the Internet of Things. The paper proposed a stretchable HMI based on a surface electromyography（sEMG） electrode with a self-similar serpentine configuration. The sEMG electrode was transfer-printed onto the skin surface conformally to monitor biological signals, followed by signal classification and controlling of a mobile robot. Such electrodes can bear rather large deformation（such as 〉30%） under an appropriate areal coverage. The sEMG electrodes have been used to record electrophysiological signals from different parts of the body with sharp curvature, such as the index finger,back of the neck and face, and they exhibit great potential for HMI in the fields of robotics and healthcare. The electrodes placed onto the two wrists would generate two different signals with the fist clenched and loosened. It is classified to four kinds of signals with a combination of the gestures from the two wrists, that is, four control modes. Experiments demonstrated that the electrodes were successfully used as an HMI to control the motion of a mobile robot remotely.

Mouse simulation in human-machine interface using kinect and 3 gear systems

We never stop finding better ways to communicate with machines.To interact with computers we tried several ways,from punched tape and tape reader to QWERTY keyboards and command lines,from graphic user interface and mouse to multi-touch screens.The way we communicate with computers or devices are getting more direct and easier.In this paper,we give gesture mouse simulation in human–computer interface based on 3 Gear Systems using two Kinect sensors.The Kinect sensor is the perfect device to achieve dynamic gesture tracking and pose recognition.We hope the 3 Gear Systems can work as a mouse,to be more specific,use gestures to do click,double click and scroll.We use Coordinate Converting Matrix and Kalman Filter to reduce the shaking caused by errors and makes the interface create a better user experience.Finally the future of human-computer interface is discussed.

High-performance optical noncontact controlling system based on broadband PtTex/Si heterojunction photodetectors for human-machine interaction

Noncontact interaction systems have attracted considerable research attention in recent years because of convenient operation,sterility,and injury prevention.However,the insufficient sensing distance and weak robustness of noncontact interaction systems for complex environments limit their practical applications.Here,we designed an integrated optical noncontact controlling system(ONCS)based on PtTe_(x)/Si optoelectronic heterojunction array.Broadband sensitive photoresponse is realized at zero bias voltage,with excellent detectivity and responsivity,boosting the noncontact sensing distance to at least 150 mm.Consequently,the system can perform noncontact detection,encoding,and control by recognizing shadow-induced spatiotemporal sequence changes in heterojunction array photocurrents.As a proof of concept,different interactive functions have been demonstrated with good accuracy and robustness by encoding finger movement above the ONCS.This study provides a new perspective for constructing high-performance noncontact interaction systems.

A distributed-parameter control system using electromagnetic neural stimulation for human-machine perception interface

This article presents a new design of a distributed-parameter control system for human-machine perception interface,which is capable of precisely stimulating the neural systems with electromagnetic fields.By discretising the neural systems,a state-space representation of the electromagnetic stimulation is developed to facilitate the following design and analysis.A forward controller with multiple inputs and multiple outputs is consequently designed to estimate the excitation current.This novel approach enables the applications of the well-established control theory to analyse the system.The feasibility and accuracy of the control system are numerically illustrated and validated with the applications of Transcranial Magnetic Stimulation(TMS)and retinal stimulation.The results indicate that the newly designed control system can not only generate electromagnetic stimulation with better attenuation and focality than the most widely used Figure-8 coil,but also transmit the patterns extracted from images with electromagnetic stimulations to human retinas.

Flexible and multifunctional triboelectric nanogenerator based on liquid metal/polyvinyl alcohol hydrogel for energy harvesting and self-powered wearable human-machine interaction

Hydrogel-based triboelectric nanoge nerator(TENG)has a promising applied prospect in wearable electronic devices.However,its low performance,poor stability,insufficient recyclability and inferior self-healing seriously hinder its development.Herein,we report a robust route to a liquid metal(LM)/polyvinyl alcohol(PVA)hydrogel-based TENG(LP-TENG).Owing to the intrinsically liquid feature of conductive LM within the flexible PVA hydrogel,the as-prepared LP-TENG exhibited comprehensiye advantages of adaptability,biocompatibility,outstanding electrical performance,superior stability,recyclability and diverse applications,which were unattainable by traditional systems.Concretely,the LP-TENG delivered appealing open circuit voltage of 250 V,short circuit current of 4μA and transferred charge of 120 nC with high stability,outperforming most advanced TENG systems.The LP-TENG was successfully employed for versatile applications with multifunctionality,including human motion detection,handwriting recognition,energy collection,message transmission and human-machine interaction.This work presents significant prospects for crafting advanced materials and devices in the fields of wearable electronics,flexible skin and smart robots.

Human-Machine Shared Lateral Control Strategy for Intelligent Vehicles Based on Human Driver Risk Perception Reliability

Intelligent vehicle(Ⅳ)technology has developed rapidly in recent years.However,achieving fully unmanned driving still presents numerous challenges,which means that human drivers will continue to play a vital role in vehicle operation for the foreseeable future.Human-machine shared driving,involving cooperation between a human driver and an automated driving system(AVS),has been widely regarded as a necessary stage for the development of IVs.Focusing onⅣdriving safety,this study proposed a human-machine shared lateral control strategy(HSLCS)based on the reliability of driver risk perception.The HSLCS starts by identifying the effective areas of driver risk perception based on eye movements.It establishes an anisotropic driving risk field,which serves as the foundation for the AVS to assess risk levels.Building upon the cumulative and diminishing effects of risk perception,the proposed approach leverages the driver's risk perception effective area and converts the risk field into a representation aligned with the driver's perspective.Subsequently,it quantifies the reliability of the driver's risk perception by using area-matching rules.Finally,based on the driver’s risk perception reliability and dif-ferences in lateral driving operation between the human driver and the AVS,the dynamic distribution of driving authority is achieved through a fuzzy rule-based system,and the human-machine shared lateral control is completed by using model predictive control.The HSLCS was tested across various scenarios on a driver-in-the-loop test platform.The results show that the HSLCS can realize the synergy and complementarity of human and machine intelligence,effectively ensuring the safety ofⅣoperation.