VR环境下基于多观测角度人眼成像特性的注视估计研究

Study on gaze estimation based on multi⁃angle eye imaging characteristics for VR environment

在虚拟现实(VR)的沉浸式场景中,基于先进的注视估计技术实现精确的注视点渲染,能够优化计算资源分配效率、缓解用户体验过程中可能产生的眩晕感。目前,可用的VR环境下的注视估计数据集只有单观测角度眼睛图像,缺乏不同观测角度的眼部图像数据集。文中构建了一个包含23 040张多观测角度眼睛图像的注视估计数据集与一个包含15 824张带有瞳孔标注的多观测角度眼睛图像的瞳孔检测数据集,并提出了一种结合多观测角度眼睛图像特征以相互补偿的多分支网络模型。将注视估计数据集样本用于模型的训练过程,预测欧氏距离损失可以达到7.68像素。进一步,将包含瞳孔位置信息的权重地图与图像融合,瞳孔位置信息的融合输入增强了模型的性能,欧氏距离损失降低到7.45像素。这项研究表明,所开发的模型能够提升VR环境下的注视估计精度,从而推动注视估计技术在VR产品中的广泛应用。

In immersive virtual reality(VR)scenes,accurate gaze point rendering based on advanced gaze estimation technology can optimize computational resource allocation efficiency and alleviate possible dizziness during user experience.Currently,the available VR gaze estimation datasets only have eye images from a single observation angle,and lack eye image datasets from different observation angles.In this paper,a gaze estimation dataset containing 23040 eye images with multiple observation angles and a pupil detection dataset including 15824 pupil annotation images with multiple observation angles are constructed,and a multi⁃branch network model combining features of eye images with multiple observation angles compensating for each other is proposed.The samples of the gaze estimation dataset are used in the training process of the model and the predicted Euclidean distance loss can reach 7.68 pixels.Furthermore,the weight map containing pupil position information is fused with the image.The fusion and input of pupil position information enhances the performance of the model,and the Euclidean distance loss is reduced to 7.45 pixels.This study demonstrates that the developed model can improve the accuracy of gaze estimation in VR environments,so as to promote the widespread application of gaze estimation technology in VR products.