增强现实手势交互设计与实现

Design and Implementation of Augmented Reality Gesture Interaction

人机交互是增强现实乃至虚拟现实领域研究的热点课题之一。沉浸与交互是增强现实的关键要素,通过视觉手势交互可以为用户提供更逼真的互动体验,可实现虚拟物体的移动、旋转等操作。基于视觉的手势跟踪识别近年来快速发展,手势交互可以应用在增强现实类游戏、智能家居、医疗辅助行业。本文首先对当前手势交互方案进行阐述,比较了手势追踪各方案的优缺点,然后介绍了手势追踪识别的步骤以及相应算法,包括MediaPipe与MMPose开源手势识别算法,最后基于Unity3D射线检测、动画以及相应开源手势算法进行了增强现实手势接口的设计,从而实现了增强现实手势交互系统。

Human-computer interaction is one of the hot topics in the field of augmented reality and even virtual reality.Immersion and interaction are the key elements of augmented reality.Visual gesture interaction can provide users with a more realistic interactive experience and realize the movement and rotation of virtual objects.Gesture tracking and recognition based on vision has developed rapidly in recent years.Gesture interaction can be applied to augmented reality games,smart homes,medical auxiliary industries.This paper firstly describes the current gesture interaction schemes,then introduces the steps of gesture tracking recognition and corresponding algorithms,then introduces MediaPipe and MMPOSE open source gesture recognition algorithms.Finally,based on Unity3D ray detection animation and the corresponding open source gesture algorithm,we design and implement the augmented reality gesture interface to complete the augmented reality gesture interaction system.