视频手势识别的互动演示系统的设计与实现

Design and implementation of interactive demo system with video hand gesture recognition

目前常用的互动演示系统多采用专用硬件,在应用场景及系统造价上均有不足。而最常见的投影演示系统往往不具备交互功能,影响演示效果。在投影演示系统的基础上,增加基于视频的手势识别技术,使投影系统具有交互操作的功能。交互系统功能实现由交互区域检测、手势检测与识别、跟踪与交互三个模块组成。第一个模块主要负责交互区域的检测与定位;第二个模块用于检测并识别交互区域内出现的特定手势;第三个模块通过卡尔曼滤波实现对用户手势的跟踪,并根据手势识别结果操作虚拟硬件,实现人机交互。该系统在复杂环境中进行了大量测试,经验证对照明变化和复杂的背景具有较好的鲁棒性,平均准确度为85.2%。此外利用CPU-GPU并行计算,平均处理速度达到每秒15帧,具有较好的实时处理能力。系统结构简单并且不依赖专用硬件,软件方面使用Python的开源图像和视频处理工具,具有很高的可移植性。

Currently,the interactive demonstration(Demo)system in common use mostly utilizes specialized hardware,but its application scenarios are insufficient and the price is too high.Common projection presentation systems often do not have interactive features,which affects presentation effect.On the basis of projection demonstration system,this paper develops a video-based hand gesture recognition technology,making the projection system have interactive functions.The interactive system consists of three modules:interactive area detection,gesture detection and recognition,and tracking and interaction.The first module mainly detects and locates the interactive area;The second module detects and recognizes specific gestures that appear in the interactive area;The third module uses Kalman filtering to track user gestures,and operates virtual hardware based on gesture recognition results to achieve human-computer interaction.The system has been extensively tested in complex environments and has been verified to be robust to changes in lighting and complex backgrounds with an average accuracy of 85.2%.In addition,the average processing speed under CPU-GPU parallel computing reaches 15 frames per second,and the system has good real-time processing capabilities.The system structure is simple and does not depend on specialized hardware.The software uses Python open source image and video processing tools,which has high portability.