大场景下自由手势交互技术

Free Gesture Interaction in Large Scene

超大屏幕在元宇宙时代已经是常见的显示终端,用户在超大屏幕场景下通常有不受约束的行走和自由人机交互的需求。针对以上需求,本研究提出一种基于二次坐标转换的3D手势坐标映射方法。将手部骨骼节点的3D坐标转换到视点所在坐标系上,再经坐标转换获得手部节点在真实场景中的实时坐标。经过虚实映射,用户可以通过裸手驱动系统端的虚拟手进行人机交互,系统基于识别到的手势信息给予用户实时反馈。实验表明,本研究方法允许用户在大屏幕或环幕场景中不受约束行走的同时,在其视线所及之处通过裸手进行拾取等交互,达到可见即可交互的效果,提升用户的交互体验。

The super large screen is a common display terminal in the metaverse era,and the users often have the requirement for unrestricted walking and free human-computer interaction in these large screen scenes.In response to the above requirements,a 3D gesture registration method was proposeed based on coordinate transformation in this study.The 3D coordinates of the hand bone nodes were registered onto the coordinate system where the viewpoint was located.Then,the real-time coordinates of hand bone nodes were obtained in the real scene through coordinate transformations.Through virtual-reality mapping,the users could drive the virtual hand on the system end through bare hands for human-machine interaction,and the system provided real-time feedback to the users based on the recognized gesture information.The results showed that the proposed method allows the users to walk freely in large or circular screen scenes,while also engaging in interactions such as picking up through bare hands in the user’s sight,achieving the effect of instant interaction and improving the user’s interaction experience.