用于重建物理和虚拟抓取的可重构数据手套

In this work,we present a reconfigurable data glove design to capture different modes of human hand-object interactions,which are critical in training embodied artificial intelligence(AI)agents for fine manipulation tasks.To achieve various downstream tasks with distinct features,our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time.In the tactile-sensing mode,the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material;this design minimizes interference during complex hand movements.The virtual reality(VR)mode enables real-time interaction in a physically plausible fashion:A caging-based approach is devised to determine stable grasps by detecting collision events.Leveraging a state-of-the-art finite element method,the simulation mode collects data on fine-grained four-dimensionalmanipulation events comprising hand and object motions in three-dimensional space and how the object's physical properties(e.g.,stress and energy)change in accordance with manipulation over time.Notably,the glove system presented here is the first to use high-fidelity simulation to investigate the unobservable physical and causal factors behind manipulation actions.In a series of experiments,we characterize our data glove in terms of individual sensors and the overall system.More specifically,we evaluate the system's three modes by①recording hand gestures and associated forces,②improving manipulation fluency in VR,and③producing realistic simulation effects of various tool uses,respectively.Based on these three modes,our reconfigurable data glove collects and reconstructs fine-grained human grasp data in both physical and virtual environments,thereby opening up new avenues for the learning of manipulation skills for embodied AI agents.

Highly Sensitive and Mechanically Stable MXene Textile Sensors for Adaptive Smart Data Glove Embedded with Near-Sensor Edge Intelligence

Smart data gloves capable of monitoring finger activities and inferring hand gestures are of significance to human-machine interfaces,robotics,healthcare,and Metaverse.Yet,most current smart data gloves present unstable mechanical contacts,limited sensitivity,as well as offline training and updating of machine learning models,leading to uncomfortable wear and suboptimal performance during practical applications.Herein,highly sensitive and mechanically stable textile sensors are developed through the construction of loose MXene-modified textile interface structures and a thermal transfer printing method with the melting-infiltration-solidification adhesion procedure.Then,a smart data glove with adaptive gesture recognition is reported,based on the integration of 10-channel MXene textile bending sensors and a near-sensor adaptive machine learning model.The near-sensor adaptive machine learning model achieves a 99.5%accuracy using the proposed post-processing algorithm for 14 gestures.Also,the model features the ability to locally update model parameters when gesture types change,without additional computation on any external device.A high accuracy of 98.1%is still preserved when further expanding the dataset to 20 gestures,where the accuracy is recovered by 27.6%after implementing the model updates locally.Lastly,an auto-recognition and control system for wireless robotic sorting operations with locally trained hand gestures is demonstrated,showing the great potential of the smart data glove in robotics and human-machine interactions.

Robotic Electrotactile Feedback Glove for Tele Presence System

Progress is described regarding the development of a new electrotactile feedback glove designed for application to dexterous robot. The sensitivity of operator's finger against electrical stimulus pulse is considered. It is found that frequency, duty ratio, and voltage amplitude of electrical stimulus pulse determine the sensitivity of finger. The effects of materials, sizes, arrangements and shapes of electrodes on sensitivity of finger are analyzed. Finally, the tactile tele presence system is designed to experimentally confirm that the robot with electrotactile feedback glove can manipulate dexterous robotic multi fingered hand and identify and classify three sorts of objects.

Gesture Recognition Based on BP Neural Network Improved by Chaotic Genetic Algorithm

Aim at the defects of easy to fall into the local minimum point and the low convergence speed of back propagation（BP）neural network in the gesture recognition, a new method that combines the chaos algorithm with the genetic algorithm（CGA） is proposed. According to the ergodicity of chaos algorithm and global convergence of genetic algorithm, the basic idea of this paper is to encode the weights and thresholds of BP neural network and obtain a general optimal solution with genetic algorithm, and then the general optimal solution is optimized to the accurate optimal solution by adding chaotic disturbance. The optimal results of the chaotic genetic algorithm are used as the initial weights and thresholds of the BP neural network to recognize the gesture. Simulation and experimental results show that the real-time performance and accuracy of the gesture recognition are greatly improved with CGA.

Highly durable machine-learned waterproof electronic glove based on low-cost thermal transfer printing for amphibious wearable applications

Gesture recording,modeling,and understanding based on a robust electronic glove(E-glove)are of great significance for efficient human-machine cooperation in harsh environments.However,such robust edge-intelligence interfaces remain challenging as existing E-gloves are limited in terms of integration,waterproofness,scalability,and interface stability between different components.Here,we report on the design,manufacturing,and application scenarios for a waterproof E-glove,which is of low cost,lightweight,and scalable for mass production,as well as environmental robustness,waterproofness,and washability.An improved neural network architecture is proposed to implement environment-adaptive learning and inference for hand gestures,which achieves an amphibious recognition accuracy of 100%in 26 categories by analyzing 2,600 hand gesture patterns.We demonstrate that the E-glove can be used for amphibious remote vehicle navigation via hand gestures,potentially opening the way for efficient human-human and human-machine cooperation in harsh environments.