融合排序与回归的卷积神经网络用于表情强度估计

Convolutional Neural Network Fusing Ranking and Regression for Expression Intensity Estimation

表情强度估计是面部表情分析的重要组成部分,是实现人机自然情感交互的关键技术.表情强度估计面临的主要挑战在于缺乏大量的有标签数据,难以通过有监督的方法来估计表情强度.尽管基于排序的方法能够解决这一问题,但是排序方法只能估计表情的相对强度,无法估计表情的绝对强度.为了解决上述问题,提出了一种融合排序与回归的卷积神经网络用于表情强度估计.其中,排序卷积神经网络采用孪生网络结构,用于学习序列中任意两帧图像的相对强弱关系;孪生网络的每一个子网采用回归卷积神经网络,用于学习有强度标签的样本,从而估计表情的绝对强度.为了验证方法的有效性,在公共数据集PAIN和CK+上进行了实验.实验结果表明,提出的方法在弱监督的条件估计表情强度的各项结果(PAIN数据集上PCC,ICC和MAE分别为0.6551,0.5293和0.9241,CK+数据集上PCC,ICC和MAE分别为0.7391,0.7216和0.1875),均优于现有的方法.

Facial expression intensity estimation is an important part of facial expression analysis.It is also the key technology to realize nature human-machine emotional interaction.The main problem faced by the expression intensity estimation is the lack of abundant labeled data,which makes it difficult to estimate facial expression intensity by supervised methods.Although some ranking based methods can address this problem,these methods only estimate the relative intensity instead of an absolute intensity.To solve the above problems,a convolution neural network fusing rank and regression is proposed for facial expression intensity estimation.The rank-CNN with a Siamese network structure learns the relative relationship between the pair wise data in the sequence.Each subnetwork in the Siamese network adopts a regression-CNN,which is used to learn the samples with intensity labels and to estimate the absolute intensity of expression.In order to verify the effectiveness of the proposed method,experiments are carried out on PAIN and CK+datasets.The experimental results show that the performance of the proposed method in weak supervised learning(PCC,ICC and MAE on PAIN data set are 0.6551,0.5293 and 0.9241 respectively,and PCC,ICC and MAE on CK+data set are 0.7391,0.7216 and 0.1875 respectively)is superior to the state of art.