EM-Gaze:eye context correlation and metric learning for gaze estimation

In recent years,deep learning techniques have been used to estimate gaze-a significant task in computer vision and human-computer interaction.Previous studies have made significant achievements in predicting 2D or 3D gazes from monocular face images.This study presents a deep neural network for 2D gaze estimation on mobile devices.It achieves state-of-the-art 2D gaze point regression error,while significantly improving gaze classification error on quadrant divisions of the display.To this end,an efficient attention-based module that correlates and fuses the left and right eye contextual features is first proposed to improve gaze point regression performance.Subsequently,through a unified perspective for gaze estimation,metric learning for gaze classification on quadrant divisions is incorporated as additional supervision.Consequently,both gaze point regression and quadrant classification perfor-mances are improved.The experiments demonstrate that the proposed method outperforms existing gaze-estima-tion methods on the GazeCapture and MPIIFaceGaze datasets.

Gaze Estimation via a Differential Eyes’Appearances Network with a Reference Grid

A person’s eye gaze can effectively express that person’s intentions.Thus,gaze estimation is an important approach in intelligent manufacturing to analyze a person’s intentions.Many gaze estimation methods regress the direction of the gaze by analyzing images of the eyes,also known as eye patches.However,it is very difficult to construct a person-independent model that can estimate an accurate gaze direction for every person due to individual differences.In this paper,we hypothesize that the difference in the appearance of each of a person’s eyes is related to the difference in the corresponding gaze directions.Based on this hypothesis,a differential eyes’appearances network(DEANet)is trained on public datasets to predict the gaze differences of pairwise eye patches belonging to the same individual.Our proposed DEANet is based on a Siamese neural network(SNNet)framework which has two identical branches.A multi-stream architecture is fed into each branch of the SNNet.Both branches of the DEANet that share the same weights extract the features of the patches;then the features are concatenated to obtain the difference of the gaze directions.Once the differential gaze model is trained,a new person’s gaze direction can be estimated when a few calibrated eye patches for that person are provided.Because personspecific calibrated eye patches are involved in the testing stage,the estimation accuracy is improved.Furthermore,the problem of requiring a large amount of data when training a person-specific model is effectively avoided.A reference grid strategy is also proposed in order to select a few references as some of the DEANet’s inputs directly based on the estimation values,further thereby improving the estimation accuracy.Experiments on public datasets show that our proposed approach outperforms the state-of-theart methods.

Alternating Direction Finite Volume Element Methods for Three-Dimensional Parabolic Equations

This paper presents alternating direction finite volume element methods for three-dimensional parabolic partial differential equations and gives four computational schemes, one is analogous to Douglas finite difference scheme with second-order splitting error, the other two schemes have third-order splitting error, and the last one is an extended LOD scheme. The L2 norm and H1 semi-norm error estimates are obtained for the first scheme and second one, respectively. Finally, two numerical examples are provided to illustrate the efficiency and accuracy of the methods.

Temporally Consistent Depth Map Estimation for 3D Video Generation and Coding

In this paper, we propose a new algorithm for temporally consistent depth map estimation to generate three-dimensional video. The proposed algorithm adaptively computes the matching cost using a temporal weighting function, which is obtained by block-based moving object detection and motion estimation with variable block sizes. Experimental results show that the proposed algorithm improves the temporal consistency of the depth video and reduces by about 38% both the flickering artefact in the synthesized view and the number of coding bits for depth video coding.

THE CHARACTERISTIC FINITE ELEMENT ALTERNATING-DIRECTION METHOD AND ANALYSIS FOR THREE-DIMENSIONAL NUMERICAL RESERVOIR SIMULATION

Petroleum science has made remarkable progress in organic geochcmistry and in the research into the theories of petroleum origin, its transport and accumulation. In estimating the oil-gas resources of a basin, the knowledge of its evolutionary history and especially the numerical computation of fluid flow and the history of its changes under heat is vital. The mathematical model can be described as a coupled system of nonlinear partial differentical equations with initial-boundary value problems. This thesis, from actual conditions such as the effect of fluid compressibility and the three-dimensional characteristic of large-scale science-engineering computation, we put forward a kind of characteristic finite element alternating-direction schemes and obtain optimal order estimates in L^2 norm for the error in the approximate assumption.

THE CHARACTERISTIC FINITE DIFFERENCE METHOD FOR THREE-DIMENSIONAL MOVING BOUNDARY VALUE PROBLEM

The software for oil-gas transport and accumulation is to describe the history of oil-gas transport and accumulation in basin evolution. It is of great value in rational evaluation of prospecting and exploiting oil-gas resources. The mathematical model can be discribed as a coupled system of nonlinear partial differential equations with moving boundary value problem. For a generic case of the three-demensional bounded region, bi this thesis, the effects of gravitation、buoyancy and capillary pressure are considered, we put forward a kind of characteristic finite difference schemes and make use thick and thin grids to form a complete set, and of calculus of vaviations, the change of variable, the theory of prior estimates and techniques, Optimal order estimates in l^2 norm are derived for the error in approximate assumption, Thus we have completely solved the well-known theoretical problem proposed by J. Douglas, Jr.

基于混合Transformer模型的三维视线估计

针对当前在无约束环境中,进行视线估计任务时准确度不高的问题,提出了一种基于混合Transformer模型的视线估计方法.首先,对MobileNet V3网络进行改进,增加了坐标注意力机制,提高MobileNet V3网络特征提取的有效性;再利用改进的MobileNet V3网络从人脸图像中提取视线估计特征;其次,对Transformer模型的前向反馈神经网络层进行改进,加入一个卷积核大小为3×3的深度卷积层,来提高全局特征整合能力;最后,将提取到的特征输入到改进后的Transformer模型进行整合处理,输出三维视线估计方向.在MPIIFaceGaze数据集上进行评估,该方法的视线估计角度平均误差为3.56°,表明该模型能够较为准确地进行三维视线估计.

基于CB-ViT的青少年视线估计算法研究

视线估计技术在人机交互、虚拟现实和医学辅助诊断等领域有着广泛应用。然而,现有的公开数据集主要针对成年人,导致基于这些数据集训练的视线估计算法在应用于青少年群体时效果通常不尽如人意。为了解决这一问题,收集了一个名为“Young-Gaze”的青少年视线数据集,涵盖了107位青少年的视线数据。还提出了一种2D视线估计算法,该算法基于ViT并引入了一个名为上下文广播的模块,同时通过融合左眼和右眼的不同层次特征,显著增强了网络模型在特征表达上的能力。在实验中,该算法在Young-Gaze数据集上展现了出色的性能,达到了5.42 cm的误差,性能优于当前其他同类2D视线估计算法。除了在Young-Gaze数据集上取得显著性能外,该算法同样在公开的2D视线估计数据集如GazeCapture和MPIIFaceGaze上进行了训练和测试,也展现了良好的性能,表明该算法不仅适用于青少年群体,也能够在成人群体中得到有效应用。

基于类注意力的眼睛凝视估计网络

近年来,眼睛凝视估计引起广泛关注。基于RGB外观的凝视估计方法使用普通摄像机和深度学习来进行凝视估计,避免了像商用眼动仪一样使用昂贵的红外设备,为更准确和成本更低的眼睛凝视估计提供了可能。然而,RGB外观图像中包含如光照强度、肤色等多种与凝视无关的特征,这些无关特征会在深度学习回归的过程中产生干扰,进而影响凝视估计的精度。针对以上问题,提出了一种名为类注意力网络(CA-Net)的新架构,它包含通道、尺度、眼睛3种不同的类注意力模块,通过这些类注意力模块可以提取和融合不同种类的注意力编码,从而降低与凝视无关特征所占的权重。在GazeCapture数据集上的大量实验表明,在基于RGB外观的凝视估计方法中,相比现有的最先进方法,CA-Net在手机和平板上分别能够提高约0.6%和7.4%的凝视估计精度。

眼动跟踪仪的光源布局优化方法

使用角膜瞳孔反射法的眼动跟踪仪相比基于表观的方法具有较高的视线估计精度,在人机交互、AR/VR等领域具有较大的实际应用价值。但是角膜瞳孔反射法受制于角膜非球面的影响,需要光源距离较近,从而限制普尔钦斑处于逼近球面的角膜区域。但光源距离太近又会放大计算的误差,所以针对眼动跟踪仪的硬件结构缺乏量化设计指导的问题,提出了一种光源布局优化方法。通过计算机仿真的手段,建立了角膜的非球面几何模型,理论分析了不同光源布局下的平均视线估计精度,然后通过由粗到精的方式,优化求解得到最优光源布局,抑制了非球面角膜带来的影响。最后以两类典型的设备为例,在典型参数下分别讨论了头戴式和遥测式的最优双光源布局,结果可作为典型硬件设计的理论参考。

基于视频理解的教学过程感知与分析

课堂是教育教学的核心阵地,对教师在课堂上的教学环节进行过程化监测和评价是提高课堂教学质量的有效途径。然而,现有基于人工的评价模式存在评价效率低下、易干扰课堂教学、主观误差等缺点,难以达到理想的效果。鉴于人工智能技术的快速发展,提出将以人为中心的智能感知与分析技术引入教师的教学过程中,对教师主体进行实时识别与分析。首先,通过人脸检测算法定位教师实时位置并进行位移分析;其次,利用视线估计算法对教师的关注区域进行检测;最后,采用基于骨架点的动作识别和表情识别对教师的动作和表情进行感知与分析。同时,对指标进行量化统计,以更为高效、客观地了解教师的教学特点,从而帮助教师针对性地改善其授课质量。在相同配置环境下的实验结果表明,该系统的各模块在相应任务中的表现较好,符合教学场景下的使用要求。从在真实的教学视频上的测试结果来看,所设计的系统能够较为准确地感知教师的教学状态,为提升授课质量提供建设性意见。

基于深度学习的视线估计方法综述

视线估计是一种预测人眼注视位置或注视方向的技术,在人机交互和计算机视觉的应用中发挥重要作用。近几年,深度学习的飞速发展改变了许多计算机视觉任务,利用深度学习进行基于外观的视线估计已成为关注热点。围绕深度学习模型的训练流程,从视线数据预处理、视线特征提取、视线学习策略、视线估计模型结构四个方面对近年基于深度学习的视线估计方法进行了综述和分析;然后介绍视线估计领域主流公开数据集,并对常用数据集分别进行2D和3D视线估计方法的对比分析。最后,探讨了当前视线估计领域的研究难点与挑战,并对未来的发展趋势进行总结与展望。

基于坐标注意力脉冲神经网络的注视估计方法

针对传统相机在拍摄人眼运动时易产生动态模糊、时间分辨率低等问题,采用事件相机近眼拍摄构建Spiking-Eye数据集,并提出一种坐标注意力的脉冲神经网络模型(CA-SpikingRepVGG)。模型读取编码后的事件数据,经过带坐标注意力的主干网络进行特征提取,最后馈入检测头进行检测。实验结果显示:CA-SpikingRepVGG的平均检测精确率R_(P)达到了70.8%,与SpikingVGG-16比较,该模型的R_(P)提高了15.9%,召回率R_(r)提高了14.2%;仅需SpikingDensenet模型1/3的训练时间,比其R_(P)提高1.8%、R_(r)提高0.9%。结果表明:该模型在针对眼球运动这一场景下对人眼的检测追踪能力更强,可以很好地完成注视估计任务。

VR环境下基于多观测角度人眼成像特性的注视估计研究

在虚拟现实(VR)的沉浸式场景中,基于先进的注视估计技术实现精确的注视点渲染,能够优化计算资源分配效率、缓解用户体验过程中可能产生的眩晕感。目前,可用的VR环境下的注视估计数据集只有单观测角度眼睛图像,缺乏不同观测角度的眼部图像数据集。文中构建了一个包含23 040张多观测角度眼睛图像的注视估计数据集与一个包含15 824张带有瞳孔标注的多观测角度眼睛图像的瞳孔检测数据集,并提出了一种结合多观测角度眼睛图像特征以相互补偿的多分支网络模型。将注视估计数据集样本用于模型的训练过程,预测欧氏距离损失可以达到7.68像素。进一步,将包含瞳孔位置信息的权重地图与图像融合,瞳孔位置信息的融合输入增强了模型的性能,欧氏距离损失降低到7.45像素。这项研究表明,所开发的模型能够提升VR环境下的注视估计精度,从而推动注视估计技术在VR产品中的广泛应用。

双向融合CNN与Transformer的三维视线估计

针对当前视线估计任务在无约束环境中易受影响因素干扰,准确度不高的问题,提出一种卷积与注意力双分支并行的特征交叉融合视线估计方法,提升了特征融合的有效性和网络性能.首先,对Mobile-Former网络进行改进,引入了线性注意力机制和部分卷积,有效提高了特征提取能力并且降低了计算成本;其次,增加了基于300W-LP数据集预训练的ResNet50头部姿态特征估计网络分支来增强视线估计的准确度,并使用Sigmoid函数作为门控单元来筛选有效特征;最后,将面部图像输入神经网络进行特征提取和融合,输出三维视线估计方向.在MPIIFace-Gaze和Gaze360数据集上评估模型,该方法的视线平均角度误差为3.70°和10.82°,通过与其他主流三维视线估计方法比较,验证了该网络模型能够比较准确的估计三维视线方向并降低计算复杂度.

基于多尺度聚合和共享注意力的注视估计模型

注视估计是从人脸图像中估计3D注视方向的方法,其中与注视直接相关的眼睛细节信息在人脸图像中集中且对注视估计具有显著影响。然而现有的注视估计模型忽略了小尺度的眼睛细节,且容易被图像特征中与注视无关的信息淹没。为此,提出一种基于多尺度聚合和共享注意力的模型以增强特征的表达能力。首先,使用分流自注意力聚合图像中不同尺度的眼睛和人脸信息,并引导模型学习不同尺度对象之间的相关性,以此处理模型对图像中眼睛细节的遗漏问题;其次,通过建立共享注意力来捕获图像之间的共享特征,减少对注视无关特征的关注;最后,结合多尺度聚合和共享注意力,进一步提高注视估计的精度。在公开数据集MPIIFaceGaze、Gaze360、Gaze360_Processed和GAFA-Head上,所提模型的平均角度误差比GazeTR(Gaze TRansformer)降低了5.74%、4.09%、4.82%和10.55%。在Gaze360背对相机的困难图像上,所提模型的平均角度误差比GazeTR降低了4.70%。实验结果表明,所提模型能有效聚合多尺度的注视信息和共享注意力,提高注视估计的准确性和鲁棒性。

融合高频信息增强和注意力机制的视线估计方法

准确的视线注视方向估计是人机交互和虚拟现实等应用场景中的关键技术。基于外观的视线估计是目前的主流方法,然而,因为眼睛外观、光线条件和头部姿态的多样性,所以无约束环境下的视线估计仍然是一个具有挑战性的任务。提出了一种高频信息视线估计网络(HFA-Net)。首先,在神经网络中加入高频信息提取模块和卷积注意力模块(CBAM),帮助网络减少冗余信息的影响;其次,将视线分为两个角度分别进行回归,并使用独立损失函数进行优化;最后,在公开数据集MPIIGaze上进行训练和测试。实验结果表明,该方法在MPIIGaze上取得了4.17°的最佳角度估计误差,超越目前主流算法。

Study on Estimation Method of Rock Mass Discontinuity Shear Strength Based on Three-Dimensional Laser Scanning and Image Technique

The estimation of shear strength of rock mass discontinuity is always a focal, but difficult, problem in the field of geotechnical engineering. Considering the disadvantages and limitation of exist- ing estimation methods, a new approach based on the shadow area percentage （SAP） that can be used to quantify surface roughness is proposed in this article. Firstly, by the help of laser scanning technique, the three-dimensional model of the surface of rock discontinuity was established. Secondly, a light source was simulated, and there would be some shadows produced on the model surface. Thirdly, to obtain the value of SAP of each specimen, the shadow detection technique was introduced for use. Fourthly, compared with the result from direct shear testing and based on statistics, an empirical for- mula was found among SAP, normal stress, and shear strength. Data of Yujian （~ River were used as an example, and the following conclusions have been made. （1） In the case of equal normal stress, the peak shear stress is positively proportional to the SAP. （2） The formula for estimating was derived, and the predictions of peak-shear strength made with this equation well agreed with the experimental re- suits obtained in laboratory tests.

EasyGaze:Hybrid eye tracking approach for handheld mobile devices

Background Eye-tracking technology for mobile devices has made significant progress.However,owing to limited computing capacity and the complexity of context,the conventional image feature-based technology cannot extract features accurately,thus affecting the performance.Methods This study proposes a novel approach by combining appearance-and feature-based eye-tracking methods.Face and eye region detections were conducted to obtain features that were used as inputs to the appearance model to detect the feature points.The feature points were used to generate feature vectors,such as corner center-pupil center,by which the gaze fixation coordinates were calculated.Results To obtain feature vectors with the best performance,we compared different vectors under different image resolution and illumination conditions,and the results indicated that the average gaze fixation accuracy was achieved at a visual angle of 1.93°when the image resolution was 96×48 pixels,with light sources illuminating from the front of the eye.Conclusions Compared with the current methods,our method improved the accuracy of gaze fixation and it was more usable.

Three-dimensional color photon counting microscopy using Bayesian estimation with adaptive priori information

In this Letter, we propose a novel three-dimeusional （3D） color microscopy for microorganisms under photon- starved conditions using photon counting integral imaging and Bayesian estimation with adaptive priori infor- mation. In photon counting integral imaging, 3D images can be visualized using maximum likelihood estimation （MLE）. However, since MLE does not consider a priori information of objects, the visual quality of 3D images may not be accurate. In addition, the only grayscale image can be reconstructed. Therefore, to enhance the visual quality of 3D images, we propose photon counting microscopy using maximum a posteriori with adaptive priori information. In addition, we consider a wavelength of each basic color channel to reconstruct 3D color images. To verify our proposed method, we carry out optical experiments.