面向航天员训练的混合现实场景理解方法研究

Research on Scene Understanding Method in Mixed Reality for Astronaut Training

针对航天员混合现实系统在人机交互过程中尚未实现自主智能诱导的局限性,结合航天器舱内环境的特殊性,构建了一种航天员混合现实空间场景理解原型系统架构,提出了基于深度学习的空间场景自主理解方法,采用合成样本的方法建立了舱内设备数据库,实现了基于深度学习算法的自主舱内设备识别。实验结果表明,对舱内设备识别的mAP值达到79.8%,混合现实空间场景理解原型架构和场景理解方法可以有效解决航天员舱内训练设备操作状态变化的自然识别需求,并实现混合现实虚拟流程的自主引导。

Considering the limitations of autonomous intelligence induction in the process of human-computer interaction and the particularity of the spacecraft cabin environment,the architecture of a prototype system for astronaut mixed reality scene understanding was constructed and a visual spatial scene understanding method based on deep learning was proposed.The deep learning model was trained with the synthetic data set which was mixed by the large-scale general data set and the in-cabin models.The experiment results showed that the mAP of in-cabin equipment identification was 79.8%,and they proved that the prototype architecture and scene understanding method proposed could effectively solve the recognition problem of the operational state changes of the astronaut cabin training equipment and the autonomous virtual induction process in mixed reality was realized.