基于手势交互的虚实融合实验仿真技术

Virtual and Real Fusion Experiment Simulation Technology Based on Gesture Interaction

在实验教学中,有些实验因实验环境、实验设备、师资等限定而不能开展。为了解决上述问题,探讨了利用深度相机构建虚实融合仿真实验的技术。利用Aruco标记算法实现了对虚拟场景与真实场景一一对应配准,结合深度相机实时采集的彩色图像和深度图像,构建出虚实融合的实验环境。为了达到操作虚拟器材的目的,利用阈值分割和成像原理,提出了一种手势交互的方法。数值实验表明位姿配准和手势交互方法基本满足虚拟实验的应用要求,为虚拟实验的构建提供了一套可行的技术方案。

In experimental teaching, some traditional experiments cannot be carried out due to limitations of experimental environment, experimental equipment, and faculty. In order to solve the above problems, the method of constructing virtual and real fusion simulation experiment using depth cameras is explored.. The Aruco labeling algorithm is used to achieve one-to-one registration of the virtual scene and the real scene, and the experimental environment of virtual and real fusion is constructed combining the colour images and depth images collected by the depth camera in real time. In order to achieve the purpose of operating virtual equipment, a gesture interaction method is proposed based on threshold segmentation and imaging principles. Numerical experiments show that pose registration and gesture interaction methods basically meet the application requirements of virtual experiments, and provide a set of feasible technical solutions for the construction of virtual experiments.