结构纹理感知下的鲁棒光流估计及人脸活检应用

Robust optical flow estimation method based on structure-texture aware retinex model and its application on face anti-spoofing

目的光流估计是计算机视觉研究中的一个重要方向,尽管光流估计方法不断改进,但光照变化条件下光流计算精度的提高仍然是一个尚待解决的挑战。人脸反欺诈方法对于确保人脸识别系统的安全性十分重要,光照鲁棒的脸部运动光流特征能为人脸活体检测提供有关运动和结构的可靠信息。为了获得对含光照变化视频中物体运动的理解能力并应用于人脸活体检测,提高系统性能,提出了一种基于结构纹理感知视网膜模型的鲁棒光流估计方法。方法基于Retinex理论,通过结构纹理感知方式将图像中的反射分量与光照分量充分解耦。由于反射分量具有丰富的纹理信息且光照分量中包含部分有用的结构信息,因此对所提取的光照分量进行滤波操作后再与反射分量一起融合到光流模型中,有效提高了光流估计的鲁棒性。为使模型所获光流具有更好的边缘保持性,采用光滑—稀疏正则化约束方式进行最小化求解。本文给出了求解优化问题的数值方法。结果采用MPI Sintel数据集图像序列,与PWC-Net、DCFlow+KF和FDFlowNet(fast deep flownet)等主流算法进行对比实验,本文方法在Clean和Final数据集中均得到最低的平均终点误差(end-point error,EPE),分别为2.473和4.807,在3个公开数据集上进行的评测进一步验证了本文方法的鲁棒性。最后,将所提取的脸部运动光流特征在人脸反欺诈数据集上进行了活体检测对比实验,对比实验结果验证了提出的光流估计算法更具鲁棒性,改善了人脸活体检测的效果。结论提出的光流计算模型,在不同光照变化条件下具有良好的鲁棒性,更适合于人脸活体检测应用。本项目代码链接为https://github.com/Xiaoxin-Liao/STARFlow。

Objective Optical flow estimation is essential for computer vision and image processing,which is focused on pixel-wise motions between consecutive images.It is beneficial for multiple research domains like target tracking,crowd flow segmentation,and human behavior analysis.In addition,optical flow can be as an effective tool for video-based motion extraction.Hence,it was investigated to find out the face spoofing-relevant motion clues.It is still challenged for accurate optical flow estimation under complex illumination circumstances although optical flow estimation has attracted wide attention in the field of computer vision.Most of dense optical flow estimation methods are based on variational iteration in terms of Horn and Schunck’s framework.The variational framework consists of data and regularization terms in common.Due to the uncontrolled illumination conditions are required to be clarified,the gradient constancy hypothesis can be adopted as the data term for variational model.The brightness constancy assumption can be optimized to a certain extent.However,neither brightness constancy assumption nor gradient constancy hypothesis effective in representing the complex illumination variations.As a result,the calculation of optical flow is mutual-benefited to illumination changes.Optical flow estimations are required to be regularized because of the inherent ill-posedness.The Tikhonov regularization is often based on the L2 norm,and it is minimized in terms of small-amplitude coefficients-distributed preservation,which can capture global patterns well in optical flow calculation.And,the minimization of the L1 norm is focused on more zero-or smallamplitude coefficients,and less large-amplitude ones.For discrete signals,L1 norm can give out better results than L2.L1 norm-based optical flow computation can be more precisely for mathematical modeling.To deal with regularization problems,conventional L1 and L2 norm is still related to current variational optical flow models.Robust penalty functionrelevant L1 or L2 norm regularization can be used to generate smooth flow field and preserve motion inconsistency.However,it could lose the fine-scale motion structures and produce excessive segmentation artifacts.The image pre-processing methods can also be carried out in estimation of optical flow while illumination changes cannot be ignored between two consecutive frames.Optical flow calculation can provide motion features in facial biometric systems as well,which is concerned for such domains like surveillance,access control and forensic investigations.However,one of the challenges of facial biometric systems is the high possibility of the systems being deceived or spoofed by non-real faces.Recent face antispoofing technique can be as an effective pathway for facial biometric systems.Method to strengthen video-based targets motion in relevance with illumination changes,our research is focused on structure-texture-perceptive retinex model and optical flow-robust estimation for human-facial anti-spoofing application.For retinex theory,to improve robustness of optical flow against non-uniform illumination,the components of illumination and reflectance in the image are separated by decoupling.Reflection and illumination components are separated from the image base on a stronger reflectivity constancy assumption.After that,the illumination component is filtered through a low pass filter and it is then integrated into the new optical flow model.Additionally,a smooth-sparse regularization constraint is adopted to preserve edges and improve the accuracy of optical flow estimation.Furthermore,the numerical implementation of the model is demonstrated.Result Comparative analysis is carried out with some state-of-the-art optical flow estimation methods,including the variational based methods and deep learning based approaches on 3 publicity datasets of Middleburry,MPI Sintel and KITTI 2015.The quantitative analysis is carried out in terms of average angular error(AAE),average end-point error(EPE)and the Fl scores in comparison with other related optical flow computations.Moreover,to evaluate the robustness of optical flow with respect to variations of illumination using the datasets,we consider conducting simulations of illumination to the source images.To render synthetic images,illumination patterns of linear,sinusoidal,Gaussian and mixture of Gaussian are involved in.To simulate the regularity and variability of real-world illumination patterns,such parameters like additive factor,multiplicative factor and gamma correlation are used as well.The experimental results show that our model outperforms all other evaluated methods on the three public datasets and their synthetic versions with different illumination patterns.To verify the feasibility of illumination-invariant method,the calculation is applied to obtain human-facial optical flow motion features,and face liveness-detected experiments are conducted on the Institute of Automation,Chinese Academy of Sciences(CASIA)face anti-spoofing database.The proposed STARFlow method is compared to some popular anti-spoofing methods related to optical flow like weighted regularization transform(WRT)and ARFlow.The quantitative and comparative evaluation metrics is composed of the accuracy of face anti-spoofing classification and the half total error rate(HTER).Similarly,to validate the illumination robustness of the proposed method under challenging illumination changes,four synthetic illumination patterns are also appended to the dataset.Experiments are carried out on the basis of the completed CASIA database and the four illumination environments.Conclusion In this study,a new variational optical flow estima‐tion model is facilitated in terms of structure-texture aware retinex theory.Experimental results validate that the proposed model outperforms several state-of-the-art optical flow estimation approaches,including some variational based methods and deep learning based methods.Meanwhile,the proposed STARFlow method can achieve a potential illuminationinvariance in terms of face anti-spoofing under different illumination changes circumstances.The source code of this project is available at:https://github.com/Xiaoxin-Liao/STARFlow.