Deep Learning for EMG-based Human-Machine Interaction:A Review

Electromyography(EMG)has already been broadly used in human-machine interaction(HMI)applications.Determining how to decode the information inside EMG signals robustly and accurately is a key problem for which we urgently need a solution.Recently,many EMG pattern recognition tasks have been addressed using deep learning methods.In this paper,we analyze recent papers and present a literature review describing the role that deep learning plays in EMG-based HMI.An overview of typical network structures and processing schemes will be provided.Recent progress in typical tasks such as movement classification,joint angle prediction,and force/torque estimation will be introduced.New issues,including multimodal sensing,inter-subject/inter-session,and robustness toward disturbances will be discussed.We attempt to provide a comprehensive analysis of current research by discussing the advantages,challenges,and opportunities brought by deep learning.We hope that deep learning can aid in eliminating factors that hinder the development of EMG-based HMI systems.Furthermore,possible future directions will be presented to pave the way for future research.

BLIND SPEECH SEPARATION FOR ROBOTS WITH INTELLIGENT HUMAN-MACHINE INTERACTION

Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.

A Robotic Teleoperation System for Precise Robotic Manipulation by Human-Machine Interaction

Teleoperation is of great importance in the area of robotics,especially when people are unavailable in the robot workshop.It provides a way for people to control robots remotely using human intelligence.In this paper,a robotic teleoperation system for precise robotic manipulation is established.The data glove and the 7-degrees of freedom(DOFs)force feedback controller are used for the remote control interaction.The control system and the monitor system are designed for the remote precise manipulation.The monitor system contains an image acquisition system and a human-machine interaction module,and aims to simulate and detect the robot running state.Besides,a visual object tracking algorithm is developed to estimate the states of the dynamic system from noisy observations.The established robotic teleoperation systemis applied to a series of experiments,and high-precision results are obtained,showing the effectiveness of the physical system.

Web GIS Human-Machine Interactive Interface Design with VISI

The fusion of VlSI （visual identity system Internet）, digital maps and Web GIS is presented. Web GIS interface interactive design with VISI needs to consider more new factors. VISI can provide the design principle, elements and contents for the Web GIS. The design of the Wuhan Bus Search System is fulfilled to confirm the validity and practicability of the fusion.

Designing interactive glazing through an engineering psychology approach:Six augmented reality scenarios that envision future car human-machine interface

Background With an increasing number of vehicles becoming autonomous,intelligent,and connected,paying attention to the future usage of car human-machine interface with these vehicles should become more relevant.Several studies have addressed car HMI but were less attentive to designing and implementing interactive glazing for every day(autonomous)driving contexts.Methods Reflecting on the literature,we describe an engineering psychology practice and the design of six novel future user scenarios,which envision the application of a specific set of augmented reality(AR)support user interactions.Additionally,we conduct evaluations on specific scenarios and experiential prototypes,which reveal that these AR scenarios aid the target user groups in experiencing a new type of interaction.The overall evaluation is positive with valuable assessment results and suggestions.Conclusions This study can interest applied psychology educators who aspire to teach how AR can be operationalized in a human-centered design process to students with minimal pre-existing expertise or minimal scientific knowledge in engineering psychology.

Human Interaction Dynamics for Its Use in Mobile Robotics:Impedance Control for Leader-follower Formation

A complete characterization of the behavior in human-robot interactions(HRI) includes both: the behavioral dynamics and the control laws that characterize how the behavior is regulated with the perception data. In this way, this work proposes a leader-follower coordinate control based on an impedance control that allows to establish a dynamic relation between social forces and motion error. For this, a scheme is presented to identify the impedance based on fictitious social forces, which are described by distance-based potential fields.As part of the validation procedure, we present an experimental comparison to select the better of two different fictitious force structures. The criteria are determined by two qualities: least impedance errors during the validation procedure and least parameter variance during the recursive estimation procedure.Finally, with the best fictitious force and its identified impedance,an impedance control is designed for a mobile robot Pioneer 3AT,which is programmed to follow a human in a structured scenario.According to results, and under the hypothesis that moving like humans will be acceptable by humans, it is believed that the proposed control improves the social acceptance of the robot for this kind of interaction.

Magnetic Array Assisted Triboelectric Nanogenerator Sensor for Real‑Time Gesture Interaction

In human-machine interaction,robotic hands are useful in many scenarios.To operate robotic hands via gestures instead of handles will greatly improve the convenience and intuition of human-machine interaction.Here,we present a magnetic array assisted sliding triboelectric sensor for achieving a real-time gesture interaction between a human hand and robotic hand.With a finger’s traction movement of flexion or extension,the sensor can induce positive/negative pulse signals.Through counting the pulses in unit time,the degree,speed,and direction of finger motion can be judged in realtime.The magnetic array plays an important role in generating the quantifiable pulses.The designed two parts of magnetic array can transform sliding motion into contact-separation and constrain the sliding pathway,respectively,thus improve the durability,low speed signal amplitude,and stability of the system.This direct quantization approach and optimization of wearable gesture sensor provide a new strategy for achieving a natural,intuitive,and real-time human-robotic interaction.

Skin‑Inspired Ultra‑Tough Supramolecular Multifunctional Hydrogel Electronic Skin for Human–Machine Interaction

Multifunctional supramolecular ultra-tough bionic e-skin with unique durability for human–machine interaction in complex scenarios still remains challenging.Herein,we develop a skininspired ultra-tough e-skin with tunable mechanical properties by a physical cross-linking salting-freezing-thawing method.The gelling agent(β-Glycerophosphate sodium:Gp)induces the aggregation and binding of PVA molecular chains and thereby toughens them(stress up to 5.79 MPa,toughness up to 13.96 MJ m^(−3)).Notably,due to molecular self-assembly,hydrogels can be fully recycled and reprocessed by direct heating(100°C for a few seconds),and the tensile strength can still be maintained at about 100%after six recoveries.The hydrogel integrates transparency(>60%),super toughness(up to 13.96 MJ m^(−3),bearing 1500 times of its own tensile weight),good antibacterial properties(E.coli and S.aureus),UV protection(Filtration:80%–90%),high electrical conductivity(4.72 S m^(−1)),anti-swelling and recyclability.The hydrogel can not only monitor daily physiological activities,but also be used for complex activities underwater and message encryption/decryption.We also used it to create a complete finger joint rehabilitation system with an interactive interface that dynamically presents the user’s health status.Our multifunctional electronic skin will have a profound impact on the future of new rehabilitation medical,human–machine interaction,VR/AR and the metaverse fields.

Flexible and multifunctional triboelectric nanogenerator based on liquid metal/polyvinyl alcohol hydrogel for energy harvesting and self-powered wearable human-machine interaction

Hydrogel-based triboelectric nanoge nerator(TENG)has a promising applied prospect in wearable electronic devices.However,its low performance,poor stability,insufficient recyclability and inferior self-healing seriously hinder its development.Herein,we report a robust route to a liquid metal(LM)/polyvinyl alcohol(PVA)hydrogel-based TENG(LP-TENG).Owing to the intrinsically liquid feature of conductive LM within the flexible PVA hydrogel,the as-prepared LP-TENG exhibited comprehensiye advantages of adaptability,biocompatibility,outstanding electrical performance,superior stability,recyclability and diverse applications,which were unattainable by traditional systems.Concretely,the LP-TENG delivered appealing open circuit voltage of 250 V,short circuit current of 4μA and transferred charge of 120 nC with high stability,outperforming most advanced TENG systems.The LP-TENG was successfully employed for versatile applications with multifunctionality,including human motion detection,handwriting recognition,energy collection,message transmission and human-machine interaction.This work presents significant prospects for crafting advanced materials and devices in the fields of wearable electronics,flexible skin and smart robots.

Interaction and the Genesis of Experience: A Phenomenological Contribution for Meaningful Embodied Artificial Intelligence

In this article I will address the issue of the meaning of Embodied Artificial Intelligence(EAI)as it is configured today.My starting point is the refined interactive perspective on the semantics of EAI,as was recently suggested by Froese and colleagues.This perspective rests on the assumption that the concept of human bodily subjectivity must be extended to include meaning-making processes,which are enabled by advanced AI systems that may be incorporated in the human biological body.After having clarified the technical background,I will introduce the genetic component of the phenomenological method as a suitable tool to face the aforementioned issue.Towards this end,I will place the genetic method in the context of the so-called New Human-Machine Interaction(New HMI).I will further outline a genetic phenomenology of visual embodiment,suggesting a futuristic application based on the thesis of the“technological supplementation of phenomenological methodology”through the synthetic method.The case at stake is that of patients with a severe clinical picture characterised by the loss of corneal function,who in the near future could be treated with synthetic corneal prosthetic implants produced by a 3D bio-printing process by using an advanced EAI technique.I will conclude this article with a brief review of the main problems that still remain open.

A Rapid Adaptation Approach for Dynamic Air‑Writing Recognition Using Wearable Wristbands with Self‑Supervised Contrastive Learning

Wearable wristband systems leverage deep learning to revolutionize hand gesture recognition in daily activities.Unlike existing approaches that often focus on static gestures and require extensive labeled data,the proposed wearable wristband with selfsupervised contrastive learning excels at dynamic motion tracking and adapts rapidly across multiple scenarios.It features a four-channel sensing array composed of an ionic hydrogel with hierarchical microcone structures and ultrathin flexible electrodes,resulting in high-sensitivity capacitance output.Through wireless transmission from a Wi-Fi module,the proposed algorithm learns latent features from the unlabeled signals of random wrist movements.Remarkably,only few-shot labeled data are sufficient for fine-tuning the model,enabling rapid adaptation to various tasks.The system achieves a high accuracy of 94.9%in different scenarios,including the prediction of eight-direction commands,and air-writing of all numbers and letters.The proposed method facilitates smooth transitions between multiple tasks without the need for modifying the structure or undergoing extensive task-specific training.Its utility has been further extended to enhance human–machine interaction over digital platforms,such as game controls,calculators,and three-language login systems,offering users a natural and intuitive way of communication.

Human-machine interaction force control: using a model-referenced adaptive impedance device to control an index finger exoskeleton

Exoskeleton robots and their control methods have been extensively developed to aid post-stroke rehabilitation. Most of the existing methods using linear controllers are designed for position control and are not suitable for human-machine interaction(HMI) force control, as the interaction system between the human body and exoskeleton is uncertain and nonlinear. We present an approach for HMI force control via model reference adaptive impedance control(MRAIC) to solve this problem in case of index finger exoskeleton control. First, a dynamic HMI model, which is based on a position control inner loop, is formulated. Second, the theoretical MRAC framework is implemented in the control system. Then, the adaptive controllers are designed according to the Lyapunov stability theory. To verify the performance of the proposed method, we compare it with a proportional-integral-derivative(PID) method in the time domain with real experiments and in the frequency domain with simulations. The results illustrate the effectiveness and robustness of the proposed method in solving the nonlinear HMI force control problem in hand exoskeleton.

High-performance optical noncontact controlling system based on broadband PtTex/Si heterojunction photodetectors for human-machine interaction

Noncontact interaction systems have attracted considerable research attention in recent years because of convenient operation,sterility,and injury prevention.However,the insufficient sensing distance and weak robustness of noncontact interaction systems for complex environments limit their practical applications.Here,we designed an integrated optical noncontact controlling system(ONCS)based on PtTe_(x)/Si optoelectronic heterojunction array.Broadband sensitive photoresponse is realized at zero bias voltage,with excellent detectivity and responsivity,boosting the noncontact sensing distance to at least 150 mm.Consequently,the system can perform noncontact detection,encoding,and control by recognizing shadow-induced spatiotemporal sequence changes in heterojunction array photocurrents.As a proof of concept,different interactive functions have been demonstrated with good accuracy and robustness by encoding finger movement above the ONCS.This study provides a new perspective for constructing high-performance noncontact interaction systems.

Research of Natural Gesture Recognition and Interactive Technology Compatible with YCb Crand HSV Color Space

In view of the current gesture recognition algorithm based on skin color segmentation is not flexible and has weak resistance to the environment, this paper puts forward a new method of skin color modeling to improve the adaptability of gesture segmentation when it face to different states. The modeling built by double color space instead of only one is compatible both in YCbCr and HSV color space to training the Gaussian model which can update the threshold value for binarization. Finally, this paper designed a natural gesture recognition and interactive systems based on the double color space model. It has shown that the system has a good interactive experience in different environments.

Recent advances in stretchable triboelectric nanogenerators for use in wearable bioelectronic devices

Wearable bioelectronic devices have the capacity for real-time human health monitoring,the provision of tailored services,and natural interaction with smart devices.However,these wearable bioelectronic devices rely on conventional rigid batteries that are frequently charged or replaced and are incompatible with the skin,leading to a discontinuity in complex therapeutic tasks related to human health monitoring and human-machine interaction.Stretchable triboelectric nanogenerator(TENG)is a high-efficiency energy harvesting technology that converts mechanical into electrical energy,effectively powering wearable bioelectronic devices.This study comprehensively overviews recent advances in stretchable TENG for use in wearable bioelectronic devices.The working mechanism of stretchable TENG is initially explained.A comprehensive discussion presents the approaches for fabricating stretchable TENG,including the design of stretchable structures and the selection of stretchable materials.Furthermore,applications of wearable bioelectronic devices based on stretchable TENG in human health monitoring(body movements,pulse,and respiration)and human-machine interaction(touch panels,machine control,and virtual reality)are introduced.Ultimately,the challenges and developmental trends regarding wearable bioelectronic devices based on stretchable TENG are elaborated.

轻质隔声夹层板结构声振耦合拓扑优化

为了满足夹层板结构轻质及隔声性能优化需求,提出了一种基于线性材料插值模型的三场浮动投影动力学拓扑优化方法,进行夹层板结构声振耦合拓扑优化设计。首先,结合混合位移/声压(u/p)有限元格式和线性材料插值模型,解决多场耦合界面依赖性问题和低密度区域局部模态问题;其次,面向三维轻质夹层板结构声振耦合特性,建立以隔声性能最大化为目标,轻量化体积分数为约束的优化范式;第三,依托阻抗管隔声性能试验场景,对三维夹层板结构有限元数值模型进行拓扑优化设计,并结合阻抗管试验验证方法的准确性和有效性;最后,将所得优化构型与常见夹层板构型的隔声性能进行对比。研究结果表明:采用所提方法可获得性能优越的新颖拓扑构型,1100~1300 Hz优化目标频段对应的优化目标隔声性能即传声损失总值从197.62 dB提升到205.09 dB,且数值仿真结果与试验结果吻合良好。在结构同等质量下,优化目标频段拓扑构型的隔声量性能优于常见开孔/闭孔矩形蜂窝状、蜘蛛网状夹层板结构的隔声量性能。

融合双阶段特征与Transformer编码的交互式图像分割

为了快速、精确地分割用户感兴趣的前景目标,获得高质量且低成本的标注分割数据,提出一种基于双阶段特征融合与Transformer编码的交互式图像分割算法.首先采用轻量化Transformer骨干网络对输入图像提取多尺度特征编码,更好地利用上下文信息;然后使用点击交互的方式引入主观先验知识,依次通过初级与加强阶段将交互特征融入Transformer网络;最后结合空洞卷积、注意力机制和多层感知机对骨干网络获取的特征图解码.实验结果表明,所提算法在GrabCut,Berkeley和DAVIS数据集上的mNoC@90%值分别达到2.18,4.04和7.39,优于其他对比算法;且算法的时间与空间复杂度低于f-BRS-B,对交互点击位置及点击类型的扰动变化具有较好的稳定性,说明该算法能够快速、精确与稳定地分割用户感兴趣目标,提升用户交互的使用体验感.

基于维度融合注意力的行人重识别

对于行人重识别,注意力机制具有增强显著特征和抑制不相关特征的优点,而先前基于注意力的方法大多独立关注通道域和空间域而忽略了通道与空间的对应关系。针对这一问题,提出一种维度融合注意力的行人重识别方法。在提取特征期间,采用滑动窗口的方法融合通道维度和空间维度,使用超大一维卷积核不断学习通道与空间之间的关系;在网络的最后阶段引入ECA注意力,ECA注意力具有局部跨通道交互的作用,与维度融合注意力相配合使用能够显著提高重识别率。实验结果表明,该方法在计算成本有限的情况下优于当前大多数方法。

A wave-shaped electrode flexible sensor capable of sensitively responding to wrinkle excitation for a multifunctional human-computer interaction system

Human-machine interactions(HMIs)have advanced rapidly in recent decades in the fields of healthcare,work,and life.However,people with disabilities and other mobility problems do not have corresponding high-tech aids for them to enjoy the convenience of HMIs.In this paper,we propose a sensor with a wave-shaped(corrugated)electrode embedded in a friction layer,which exhibits high sensitivity to skin fold excitation and enormous potential in HMIs.Attributing to the wave-shaped electrode design,it has no built-in cavities,and its small size allows it to flexibly cope with folds at different angles.By specifying the carbon nanotube hybrid silicone film as the electrode layer material and silicone film as the friction layer,good electrical output performance,tensile properties,and biocompatibility can be achieved.Then,the sensor is tested on various joints and skin folds of the human body,the output signals of which can be distinguished between normal physiological behavior and test behavior.Based on this sensor,we designed a medical alarm system,a robotic arm assistive system,and a cell phone application control system for the disabled to help them in the fields of healthcare,work,and life.In conclusion,our research presents a feasible technology to enhance HMIs and makes a valuable contribution to the development of high-tech aids for the disabled.

Assistive Robotics for Upper Limb Physical Rehabilitation:A Systematic Review and Future Prospects

Physical assistive robotics are oriented to support and improve functional capacities of people.In physical rehabilitation,robots are indeed useful for functional recovery of affected limb.However,there are still open questions related to technological aspects.This work presents a systematic review of upper limb rehabilitation robotics in order to analyze and establish technological challenges and future directions in this area.A bibliometric analysis was performed for the systematic literature review.Literature from the last six years,conducted between August 2020 and May 2021,was reviewed.The methodology for the literature search and a bibliometric analysis of the metadata are presented.After a preliminary search resulted in 820 articles,a total of 66 articles were included.A concurrency network and bibliographic analysis were provided.And an analysis of occurrences,taxonomy,and rehabilitation robotics reported in the literature is presented.This review aims to provide to the scientific community an overview of the state of the art in assistive robotics for upper limb physical rehabilitation.The literature analysis allows access to a gap of unexplored options to define the technological prospects applied to upper limb physical rehabilitation robotics.