基于WebRTC的残疾人鼠标研究与实现

Research and Implementation of Disabled Mouse Based on WebRTC

文中基于手势识别和WebRTC框架提出了残疾人鼠标的设想,对整个系统模块进行了研究和详细设计。残疾人鼠标系统基于B/S架构,将图像获取、命令执行和图像匹配分离,利用Firefox进行图像获取和鼠标操作的模拟,利用JavaCV和Jetty实现服务器端的图像匹配和命令反馈。文中使用形状比例和转角函数等进行特征抽取,使用背景差分和欧式距离进行图像匹配,在快速响应的同时保证匹配的正确率。B/S架构使得系统可以构建于云环境并具有良好的扩展性、伸缩性和可定制性。实验表明,系统已经达到初期的设计目的,并具有可接受的延迟,符合鼠标操作对于实时性的要求。

It has proposed the idea of disabled mouse based on gesture recognition and WebRTC framework, and has done study and de- tailed design of the entire system modules. Disabled mouse system is based on B/S structure, separates image acquisition, command exe- cut.ion and image matching, using Firefox for image acquisition and mouse operation simulation, using JavaCV and Jetty to implement server-side image matching and command feedback. Shape proportion and comer function is used for feature extraction, and image matc- hing is conducted by background difference and Euclidean distance to ensure the correct rate of the match with rapid response at the same time. B/S architecture,allows the system to be built on the cloud environment and has good scalability, flexibility and customizability. The experiment shows that the system has achieved initial design purpose and acceptable delay, meeting the requirements the mouse operation for the real-time requirements.