基于自监督的人脸面部动作单元检测

Human Facial Action Unit Detection Based on Self-supervised Learning

为了能够更好地从无标注图像中提取和人脸信息相关的特征,从而提升面部动作单元检测的准确度,提出了基于自监督的人脸面部动作单元检测方法。该方法将vision transformer作为编码器提取视频序列中每一帧的人脸特征,并利用视频序列中自然存在的时序特性对这些特征构造三元组损失函数。同时利用每个视频中个体信息的一致性,将提取出的人脸特征解耦为个体特征和表情特征,从而为人脸面部动作单元检测等下游任务剔除掉无关的噪音,进而提升下游任务的表现性能。通过在BP4D数据集上进行的实验与其他最先进的自监督方法进行对比,本文的方法在性能上超越了已有的其他方法。

To better extract facial features related from unlabeled images and thus improve the accuracy of facial action unit detection, a human facial action unit detection based on self-supervised learning is proposed. The proposed method uses vision transformer as an encoder to extract face features for each frame in the video sequence. A triple loss function is constructed for these features, based on the natural temporal consistency of facial video sequence. And the extracted facial features are disentangled into identity and expression based on the consistency of identity information in each video. With these improvement, Our method can eliminate irrelevant noise for downstream tasks including facial action unit detection in order to improve the performance of downstream tasks. We performed experiments on the BP4 D dataset, and the performance of our method are compared with other state-of-the-art self-supervised methods.