自适应跑步机多运动模式人机交互技术

Multi-locomotion mode human-robot interaction technology for self-paced treadmills

自适应跑步机是虚拟现实(virtual reality,VR)环境中用于人机交互的重要设备,实现该设备在VR环境下的多运动模式交互是丰富其应用场景的关键。该文构建了一种基于自适应跑步机的多运动模式交互控制架构,该架构包括感知层和控制层,感知层基于足底压力鞋垫实现高泛化性能的人体运动模式识别;控制层依据人体稳定性条件对不同运动模式(站立、行走、倒退、奔跑和跳跃)下的控制策略进行切换。该架构自动将运动模式识别结果与跑步机运动控制相结合,实现了多运动模式人机交互控制。实验结果表明:分层控制方法能够实现稳定、平滑的多运动模式人机交互,保证用户在交互过程中步态自然、姿态稳定,满足自适应跑步机的人机交互需求。

[Objective] A self-paced treadmill(SPT) is key human-robot interactive equipment for virtual reality, which can enable a user to walk at the intended speed by using a re-positioning technology. Realizing multimode interactions of SPTs is crucial for enriching their applications. However, existing studies only realizes a few interaction modes. To realize multimode interactions in self-paced treadmills, a novel multilayer control framework is proposed in this paper. [Methods] In this study, the control system is divided into two layers: the recognition layer and the control layer. First, a novel hybrid spatial-temporal graph convolutional neural network is proposed to realize user-independent human locomotion mode recognition based on plantar pressure insoles in the recognition layer. The proposed network separates the pressure and acceleration signals and dealt with them individually. It is also utilized to extract the natural spatial topology between pressure nodes. Long short-term memory layers are used to individually extract temporal-dependent features of pressure and acceleration signals and to fuse multimodal features for final recognition. A multilayer perceptron is utilized to map the fusion features to the locomotion modes. By extracting the natural spatial-temporal features of multimodal data during human locomotion, a high generalization capability of the recognition results can be expected. Second, control strategies for different locomotion modes are designed in the control layer according to the stability condition of different human locomotion modes. Meanwhile, a walking speed feedforward control strategy is proposed to re-position the user and ensure natural gaits for the walking mode. Variable gain control strategies are adopted to manipulate the acceleration for the running and back walking modes. A buffer control strategy is proposed to improve the stability during jump landing for the jumping mode. Then, a finite state machine is used to automatically switch the control strategies. The states are transited based on the recognition results. [Results] 1) The proposed locomotion mode recognition method was evaluated on a dataset that comprises eight subjects with five locomotion modes through the leave-one-subject-out cross validation. Then, it was compared with the convolutional neural network(CNN) and domain-adversarial neural network(DANN). Experimental results indicated that the mean and standard deviation classification accuracies of the CNN, DANN, and HSTGCN are(90.26±8.54)%,(97.71±3.60)%, and(97.37±1.40)%, respectively. These results validated that the proposed method can achieve high generalization capability without any dependency on the data of target subjects. Hence, the burden of repeated data collection and network training was reduced. 2) Based on the recognition results, experiments on the multi-locomotion mode human-robot interaction were conducted using a finite state machine. Experimental results indicated that a user can freely change the locomotion modes on the treadmill, and the balance was not significantly affected by the treadmill acceleration. [Conclusions] The proposed framework can automatically combine the recognition results with the treadmill control and can realize the control of multi-locomotion mode human-robot interactions. Further, experimental results validate that the proposed multilayer control strategy can achieve a stable and smooth multi-locomotion mode human-robot interaction, ensure natural gaits and posture stability of the user, and meet the requirements of multi-locomotion mode human-robot interactions for self-paced treadmills.