Designing interactive glazing through an engineering psychology approach:Six augmented reality scenarios that envision future car human-machine interface

Background With an increasing number of vehicles becoming autonomous,intelligent,and connected,paying attention to the future usage of car human-machine interface with these vehicles should become more relevant.Several studies have addressed car HMI but were less attentive to designing and implementing interactive glazing for every day(autonomous)driving contexts.Methods Reflecting on the literature,we describe an engineering psychology practice and the design of six novel future user scenarios,which envision the application of a specific set of augmented reality(AR)support user interactions.Additionally,we conduct evaluations on specific scenarios and experiential prototypes,which reveal that these AR scenarios aid the target user groups in experiencing a new type of interaction.The overall evaluation is positive with valuable assessment results and suggestions.Conclusions This study can interest applied psychology educators who aspire to teach how AR can be operationalized in a human-centered design process to students with minimal pre-existing expertise or minimal scientific knowledge in engineering psychology.

Deep Learning for EMG-based Human-Machine Interaction:A Review

Electromyography(EMG)has already been broadly used in human-machine interaction(HMI)applications.Determining how to decode the information inside EMG signals robustly and accurately is a key problem for which we urgently need a solution.Recently,many EMG pattern recognition tasks have been addressed using deep learning methods.In this paper,we analyze recent papers and present a literature review describing the role that deep learning plays in EMG-based HMI.An overview of typical network structures and processing schemes will be provided.Recent progress in typical tasks such as movement classification,joint angle prediction,and force/torque estimation will be introduced.New issues,including multimodal sensing,inter-subject/inter-session,and robustness toward disturbances will be discussed.We attempt to provide a comprehensive analysis of current research by discussing the advantages,challenges,and opportunities brought by deep learning.We hope that deep learning can aid in eliminating factors that hinder the development of EMG-based HMI systems.Furthermore,possible future directions will be presented to pave the way for future research.

BLIND SPEECH SEPARATION FOR ROBOTS WITH INTELLIGENT HUMAN-MACHINE INTERACTION

Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.

A Robotic Teleoperation System for Precise Robotic Manipulation by Human-Machine Interaction

Teleoperation is of great importance in the area of robotics,especially when people are unavailable in the robot workshop.It provides a way for people to control robots remotely using human intelligence.In this paper,a robotic teleoperation system for precise robotic manipulation is established.The data glove and the 7-degrees of freedom(DOFs)force feedback controller are used for the remote control interaction.The control system and the monitor system are designed for the remote precise manipulation.The monitor system contains an image acquisition system and a human-machine interaction module,and aims to simulate and detect the robot running state.Besides,a visual object tracking algorithm is developed to estimate the states of the dynamic system from noisy observations.The established robotic teleoperation systemis applied to a series of experiments,and high-precision results are obtained,showing the effectiveness of the physical system.

Skin‑Inspired Ultra‑Tough Supramolecular Multifunctional Hydrogel Electronic Skin for Human–Machine Interaction

Multifunctional supramolecular ultra-tough bionic e-skin with unique durability for human–machine interaction in complex scenarios still remains challenging.Herein,we develop a skininspired ultra-tough e-skin with tunable mechanical properties by a physical cross-linking salting-freezing-thawing method.The gelling agent(β-Glycerophosphate sodium:Gp)induces the aggregation and binding of PVA molecular chains and thereby toughens them(stress up to 5.79 MPa,toughness up to 13.96 MJ m^(−3)).Notably,due to molecular self-assembly,hydrogels can be fully recycled and reprocessed by direct heating(100°C for a few seconds),and the tensile strength can still be maintained at about 100%after six recoveries.The hydrogel integrates transparency(>60%),super toughness(up to 13.96 MJ m^(−3),bearing 1500 times of its own tensile weight),good antibacterial properties(E.coli and S.aureus),UV protection(Filtration:80%–90%),high electrical conductivity(4.72 S m^(−1)),anti-swelling and recyclability.The hydrogel can not only monitor daily physiological activities,but also be used for complex activities underwater and message encryption/decryption.We also used it to create a complete finger joint rehabilitation system with an interactive interface that dynamically presents the user’s health status.Our multifunctional electronic skin will have a profound impact on the future of new rehabilitation medical,human–machine interaction,VR/AR and the metaverse fields.

Flexible and multifunctional triboelectric nanogenerator based on liquid metal/polyvinyl alcohol hydrogel for energy harvesting and self-powered wearable human-machine interaction

Hydrogel-based triboelectric nanoge nerator(TENG)has a promising applied prospect in wearable electronic devices.However,its low performance,poor stability,insufficient recyclability and inferior self-healing seriously hinder its development.Herein,we report a robust route to a liquid metal(LM)/polyvinyl alcohol(PVA)hydrogel-based TENG(LP-TENG).Owing to the intrinsically liquid feature of conductive LM within the flexible PVA hydrogel,the as-prepared LP-TENG exhibited comprehensiye advantages of adaptability,biocompatibility,outstanding electrical performance,superior stability,recyclability and diverse applications,which were unattainable by traditional systems.Concretely,the LP-TENG delivered appealing open circuit voltage of 250 V,short circuit current of 4μA and transferred charge of 120 nC with high stability,outperforming most advanced TENG systems.The LP-TENG was successfully employed for versatile applications with multifunctionality,including human motion detection,handwriting recognition,energy collection,message transmission and human-machine interaction.This work presents significant prospects for crafting advanced materials and devices in the fields of wearable electronics,flexible skin and smart robots.

Virtual brain surgery simulation system based on haptic interaction

To improve the accuracy and interactivity of soft tissue delormatlon simulation, a new plate spring model based on physics is proposed. The model is parameterized and thus can be adapted to simulate different organs. Different soft tissues are modeled by changing the width, number of pieces, thickness, and length of a single plate spring. In this paper, the structural design, calcula- tion of soft tissue deformation and real-time feedback operations of our system are also introduced. To evaluate the feasibility of the system and validate the model, an experimental system of haptic in- teraction, in which users can use virtual hands to pull virtual brain tissues, is built using PHANTOM OMNI devices. Experimental results show that the proposed system is stable, accurate and promising for modeling instantaneous soft tissue deformation.

Multi-dimensional force sensor for haptic interaction:a review

Haptic interaction plays an important role in the virtual reality technology,which let a person not only view the 3D virtual environment but also realistically touch the virtual environment.As a key part of haptic interaction,force feedback has become an essential function for the haptic interaction.Therefore,multi-dimensional force sensors are widely used in the fields of virtual reality and augmented reality.In this paper,some conventional multi-dimensional force sensors based on different measurement principles,such as resistive,capacitive,piezoelectric,are briefly introduced.Then the mechanical structures of the elastic body of multi-dimensional force sensors are reviewed.It is obvious that the performance of the multi-dimensional force sensor is mainly dependent upon the mechanical structure of elastic body.Furthermore,the calibration process of the force sensor is analyzed,and problems in calibration are discussed.Interdimensional coupling error is one of the main factors affecting the measurement precision of the multi-dimensional force sensors.Therefore,reducing or even eliminating dimensional coupling error becomes a fundamental requirement in the design of multi-dimensional force sensors,and the decoupling state-of-art of the multi-dimensional force sensors are introduced in this paper.At last,the trends and current challenges of multi-dimensional force sensing technology are proposed.

Web GIS Human-Machine Interactive Interface Design with VISI

The fusion of VlSI （visual identity system Internet）, digital maps and Web GIS is presented. Web GIS interface interactive design with VISI needs to consider more new factors. VISI can provide the design principle, elements and contents for the Web GIS. The design of the Wuhan Bus Search System is fulfilled to confirm the validity and practicability of the fusion.

Human Interaction Dynamics for Its Use in Mobile Robotics:Impedance Control for Leader-follower Formation

A complete characterization of the behavior in human-robot interactions(HRI) includes both: the behavioral dynamics and the control laws that characterize how the behavior is regulated with the perception data. In this way, this work proposes a leader-follower coordinate control based on an impedance control that allows to establish a dynamic relation between social forces and motion error. For this, a scheme is presented to identify the impedance based on fictitious social forces, which are described by distance-based potential fields.As part of the validation procedure, we present an experimental comparison to select the better of two different fictitious force structures. The criteria are determined by two qualities: least impedance errors during the validation procedure and least parameter variance during the recursive estimation procedure.Finally, with the best fictitious force and its identified impedance,an impedance control is designed for a mobile robot Pioneer 3AT,which is programmed to follow a human in a structured scenario.According to results, and under the hypothesis that moving like humans will be acceptable by humans, it is believed that the proposed control improves the social acceptance of the robot for this kind of interaction.

Magnetic Array Assisted Triboelectric Nanogenerator Sensor for Real‑Time Gesture Interaction

In human-machine interaction,robotic hands are useful in many scenarios.To operate robotic hands via gestures instead of handles will greatly improve the convenience and intuition of human-machine interaction.Here,we present a magnetic array assisted sliding triboelectric sensor for achieving a real-time gesture interaction between a human hand and robotic hand.With a finger’s traction movement of flexion or extension,the sensor can induce positive/negative pulse signals.Through counting the pulses in unit time,the degree,speed,and direction of finger motion can be judged in realtime.The magnetic array plays an important role in generating the quantifiable pulses.The designed two parts of magnetic array can transform sliding motion into contact-separation and constrain the sliding pathway,respectively,thus improve the durability,low speed signal amplitude,and stability of the system.This direct quantization approach and optimization of wearable gesture sensor provide a new strategy for achieving a natural,intuitive,and real-time human-robotic interaction.

Teaching the User By Learning From the User:Personalizing Movement Control in Physical Human-robot Interaction

This paper proposes a novel approach for physical human-robot interactions(pHRI), where a robot provides guidance forces to a user based on the user performance. This framework tunes the forces in regards to behavior of each user in coping with different tasks, where lower performance results in higher intervention from the robot. This personalized physical human-robot interaction(p2HRI) method incorporates adaptive modeling of the interaction between the human and the robot as well as learning from demonstration(LfD) techniques to adapt to the users' performance. This approach is based on model predictive control where the system optimizes the rendered forces by predicting the performance of the user. Moreover, continuous learning of the user behavior is added so that the models and personalized considerations are updated based on the change of user performance over time. Applying this framework to a field such as haptic guidance for skill improvement, allows a more personalized learning experience where the interaction between the robot as the intelligent tutor and the student as the user,is better adjusted based on the skill level of the individual and their gradual improvement. The results suggest that the precision of the model of the interaction is improved using this proposed method,and the addition of the considered personalized factors to a more adaptive strategy for rendering of guidance forces.

Interaction and the Genesis of Experience: A Phenomenological Contribution for Meaningful Embodied Artificial Intelligence

In this article I will address the issue of the meaning of Embodied Artificial Intelligence(EAI)as it is configured today.My starting point is the refined interactive perspective on the semantics of EAI,as was recently suggested by Froese and colleagues.This perspective rests on the assumption that the concept of human bodily subjectivity must be extended to include meaning-making processes,which are enabled by advanced AI systems that may be incorporated in the human biological body.After having clarified the technical background,I will introduce the genetic component of the phenomenological method as a suitable tool to face the aforementioned issue.Towards this end,I will place the genetic method in the context of the so-called New Human-Machine Interaction(New HMI).I will further outline a genetic phenomenology of visual embodiment,suggesting a futuristic application based on the thesis of the“technological supplementation of phenomenological methodology”through the synthetic method.The case at stake is that of patients with a severe clinical picture characterised by the loss of corneal function,who in the near future could be treated with synthetic corneal prosthetic implants produced by a 3D bio-printing process by using an advanced EAI technique.I will conclude this article with a brief review of the main problems that still remain open.

沉浸式虚拟现实中加入触觉反馈对学习效果的影响——来自消防安全教育的证据

随着虚拟现实硬件设备的快速普及和使用成本的降低,沉浸式虚拟现实(IVR)在教学中的应用逐渐增多,但其研究大多关注视觉和听觉体验,对引入触觉反馈后在学习效果提升上的有效性仍缺乏证据。消防安全教育是IVR的典型应用场景,探究在IVR消防安全教育系统中加入触觉反馈(力反馈和热感)对学习者学业成绩、学习动机和认知负荷的影响,有助于为优化IVR教学应用提供依据。基于该情境的对照实验结果表明:IVR能够显著促进学习者对程序性知识、事实性知识和概念性知识的学习,并且加入触觉反馈后的知识增益更高,更加有利于程序性知识的习得;触觉反馈的加入显著降低了学习者的认知负荷,尤其是外在认知负荷,但对学习者学习动机的影响并不显著。研究结论为多媒体学习中触觉通道的整合提供了新的证据,拓展了原有多媒体学习的认知理论框架,有助于指导IVR教育资源的设计、开发及应用。

基于纹理图像和用户动作信息的纹理触觉再现方法研究

近年来,对虚拟表面纹理进行工具介导的振动触觉再现已经成为触觉领域的一个研究热点。针对已有的纹理触觉渲染方法还存在适用范围窄、泛化能力弱、交互真实感低等问题,以改进后的MelGAN为基本网络,构建了一种新的纹理触觉渲染模型。该模型以纹理图像和用户的实时动作信息为输入,在能够高保真地生成振动触觉信号的同时,对常见纹理图像拥有更好的泛化能力。此外,设计了一款具有动作信息实时采集和振动触觉表达功能的笔式装置。在采集了数据库之外的真实纹理表面的振动触觉信号后,比较了提出的模型与已有方法在信号生成方面的性能差异。结果表明,模型实现了最低的均方根误差(0.173),验证了其对未建模纹理进行触觉渲染的能力。最后,使用笔式装置进行了两项用户实验。平均6.01的主观相似度评分说明,即使是数据库之外的新纹理,模型也能为用户提供较高的纹理交互真实感。

空间机械臂操作子系统设计与实现

空间机械臂作为中国空间站建设和在轨运营的重要平台,具有自由度多、转动惯量大、操作精度高、模式多样等特点.针对空间机械臂在轨任务特点,建立了支持长期在轨、多模式工作的空间机械臂操作子系统.该操作子系统由机械臂操作台、虚拟仿真平台、显示器等构成,可满足多任务需求.通过对操作子系统的架构、工作模式、技术特点的分析,重点研究了操作子系统的多总线体系结构、多模态人机交互技术、力觉渲染算法,并根据地面验证和在轨试验,给出了空间机械臂操作子系统的验证情况.结果表明,空间机械臂操作子系统可有效模拟在轨任务,准确完成对机械臂的控制与精细操作,为支撑空间机械臂任务实施积累了经验和数据.

Recent advances in stretchable triboelectric nanogenerators for use in wearable bioelectronic devices

Wearable bioelectronic devices have the capacity for real-time human health monitoring,the provision of tailored services,and natural interaction with smart devices.However,these wearable bioelectronic devices rely on conventional rigid batteries that are frequently charged or replaced and are incompatible with the skin,leading to a discontinuity in complex therapeutic tasks related to human health monitoring and human-machine interaction.Stretchable triboelectric nanogenerator(TENG)is a high-efficiency energy harvesting technology that converts mechanical into electrical energy,effectively powering wearable bioelectronic devices.This study comprehensively overviews recent advances in stretchable TENG for use in wearable bioelectronic devices.The working mechanism of stretchable TENG is initially explained.A comprehensive discussion presents the approaches for fabricating stretchable TENG,including the design of stretchable structures and the selection of stretchable materials.Furthermore,applications of wearable bioelectronic devices based on stretchable TENG in human health monitoring(body movements,pulse,and respiration)and human-machine interaction(touch panels,machine control,and virtual reality)are introduced.Ultimately,the challenges and developmental trends regarding wearable bioelectronic devices based on stretchable TENG are elaborated.

情感化社交触觉界面的交互原型开发工具设计研究

目的触觉交互是自然人机交互中的热点方向,触觉交互界面具备支持用户之间或人机之间情感交流的潜力。本研究旨在通过开发一种流程化、低门槛、轻成本且可体验的触觉界面原型工具,帮助设计师克服在触觉交互界面设计中的挑战,包括触觉生理学及触觉交互原理等知识基础、设计流程引导和技术开发支持,从而实现更有效的用户实验和设计深化。方法首先,利用参与式设计方法,确定交互设计师在触觉交互界面设计中的主要挑战和核心需求;其次,以此为基础构建触觉交互设计流程引导及情感触觉信号编码界面设计的必要元素,开发支持情感触觉交互原型迭代的硬件工具,并完成原型开发工具的系统设计;最后,通过用户实验中的场景化测试验证初步的气动微流控柔性薄膜阵列的基础情感交互效果,并建立了基础的情感-触觉映射参考效果库。结论本研究开发了触觉界面原型工具,通过系统化的设计流程和简易的操作方式,有助于触觉交互设计师更加灵活便捷地进行设计研究和实践。该工具创新性地提升了触觉界面的设计效率,为情感触觉交互体验设计提供了丰富的可能性。

A wave-shaped electrode flexible sensor capable of sensitively responding to wrinkle excitation for a multifunctional human-computer interaction system

Human-machine interactions(HMIs)have advanced rapidly in recent decades in the fields of healthcare,work,and life.However,people with disabilities and other mobility problems do not have corresponding high-tech aids for them to enjoy the convenience of HMIs.In this paper,we propose a sensor with a wave-shaped(corrugated)electrode embedded in a friction layer,which exhibits high sensitivity to skin fold excitation and enormous potential in HMIs.Attributing to the wave-shaped electrode design,it has no built-in cavities,and its small size allows it to flexibly cope with folds at different angles.By specifying the carbon nanotube hybrid silicone film as the electrode layer material and silicone film as the friction layer,good electrical output performance,tensile properties,and biocompatibility can be achieved.Then,the sensor is tested on various joints and skin folds of the human body,the output signals of which can be distinguished between normal physiological behavior and test behavior.Based on this sensor,we designed a medical alarm system,a robotic arm assistive system,and a cell phone application control system for the disabled to help them in the fields of healthcare,work,and life.In conclusion,our research presents a feasible technology to enhance HMIs and makes a valuable contribution to the development of high-tech aids for the disabled.

Assistive Robotics for Upper Limb Physical Rehabilitation:A Systematic Review and Future Prospects

Physical assistive robotics are oriented to support and improve functional capacities of people.In physical rehabilitation,robots are indeed useful for functional recovery of affected limb.However,there are still open questions related to technological aspects.This work presents a systematic review of upper limb rehabilitation robotics in order to analyze and establish technological challenges and future directions in this area.A bibliometric analysis was performed for the systematic literature review.Literature from the last six years,conducted between August 2020 and May 2021,was reviewed.The methodology for the literature search and a bibliometric analysis of the metadata are presented.After a preliminary search resulted in 820 articles,a total of 66 articles were included.A concurrency network and bibliographic analysis were provided.And an analysis of occurrences,taxonomy,and rehabilitation robotics reported in the literature is presented.This review aims to provide to the scientific community an overview of the state of the art in assistive robotics for upper limb physical rehabilitation.The literature analysis allows access to a gap of unexplored options to define the technological prospects applied to upper limb physical rehabilitation robotics.