基于实时视频感知的虚拟体育交互系统

Virtual Sports Interaction System Based on Real-time Video Perception

针对疫情常态化背景下,传统体育项目受场地、器材等限制,市场上相关产品价格昂贵、可扩展性不足等问题,提出了一种基于实时视频感知的虚拟体育交互系统.该系统设计视频数据采集模块和人体关节点提取模块,结合OpenPose获取人体的关节点坐标,实时捕捉人体手势以及肢体动作.动作语义理解模块包括运动动作理解和绘图动作理解.前者根据运动中肢体关节点的相对位置关系,识别运动动作语义.后者将手腕部关节点绘图动作轨迹生成为草图图像,使用AlexNet进行识别分类,解析为对应的绘制动作语义.该模型在边缘端设备的分类准确率为98.83%.采用基于Unity设计的草图游戏应用作为可视化交互界面,实现在虚拟场景中的运动交互.该系统使用实时视频感知交互方式实现居家运动健身,无需其他的外部设备,具有更强的参与度和趣味性.

A virtual sports interaction system based on real-time video perception is proposed in response to the problems that traditional sports are limited by venues and equipment in the context of ongoing COVID-19 response, and the related products in the market are expensive and not scalable. The system is designed with a video data acquisition module and a human joint point extraction module, which can acquire human joint point coordinates in combination with OpenPose and capture human gestures and body movements in real time. The action semantic understanding module includes motion action understanding and drawing action understanding. The former recognizes the motion action semantics depending on the relative position relationship of the limb joints in motion. The latter generates the drawing action trajectories of wrist joints as sketch images, uses AlexNet to recognize and classify them, and resolves them into the corresponding drawing action semantics. The classification accuracy of the model is 98.83% in edge-side devices. A Unity-based sketch game application is used as the visual interaction interface to realize motion interaction in a virtual scene. The system adopts the interaction mode of real-time video perception to achieve home exercise and fitness without other external devices, which is more participatory and interesting.