基于“面部表情+手”的混合手势交互技术

Hybrid Gesture Interactive Technology Based on“Facial Expression+Hand”

为克服单一输入形式存在的交互缺点,融合手部移动和面部表情两种输入方式的交互特性,将手部移动和面部表情动作相结合,提出了基于“面部表情+手”的混合手势交互技术。混合手势交互技术将7种面部表情和手部移动组合起来,通过手部移动和面部表情识别操控计算机执行一系列目标选择任务。设计的实验中,手部移动用于操控鼠标光标移动,面部表情识别替代鼠标的点击操作用于选中目标按钮。根据设计的多种目标选择任务,详细分析混合手势交互技术的识别错误率和平均识别时间。结果表明,“面部表情+手”的混合手势交互技术的识别准确率可达93.81%,平均识别时间可达2921 ms,完全满足日常的人机交互需求。

In order to overcome the shortcomings of single input form,a“facial expression+hand”hybrid gesture interaction technology is proposed by combining hand and facial expression.The hybrid gesture interaction technology combines seven kinds of facial expressions and hand movements.Through hand movements and facial expression recognition,the computer can be manipulated to perform a series of target selection tasks.In the designed experiment,hand movement is used to control mouse cursor movement,and facial expression recognition is used to select target button instead of mouse click operation.According to the designed multiple target selection tasks,the recognition error rate and average recognition time of the hybrid gesture interaction technology are analyzed in detail.The results show that the recognition accuracy of“facial expression+hand”hybrid gesture interaction technology can reach 93.81%,and the average recognition time can reach 2921 ms,which fully meets the needs of daily human-computer interaction.