结合空间注意力机制与光流特征的微表情识别方法

Micro-Expression Recognition Method Based on Spatial Attention Mechanism and Optical Flow Features

针对微表情运动的局部性问题,提出一种将深度学习的空间注意力机制与微表情光流特征相结合的微表情识别自动方法.首先,采用帧差法识别缺少峰值帧标记的微表情样本的峰值帧;然后,利用TV-L1光流法提取微表情起始帧与峰值帧之间的光流水平、垂直分量图,并根据光流的水平、垂直分量图导出对应的光流应变模式图;将3个光流图以通道叠加的方式连接起来,构成微表情的光流特征图;最后,在Inception模块搭建的卷积神经网络中设计了一种包含可学习参数的空间注意力单元,使模型在特征提取过程中能够更加关注存在微表情运动的区域.在空间注意力单元中利用3×3和7×7这2种大小的卷积核进行空间注意力的推断,使模型能够综合地考虑不同尺度卷积核的注意力推断结果.实验结果表明,该方法在MEGC2019综合微表情数据集上的识别准确率达到0.788,优于已有的微表情识别方法.

In order to solve the locality problem of micro-expression movement, an automatic micro-expression recognition method based on spatial attention mechanism of deep learning and optical flow features of micro-expressions is proposed. First, the frame difference method is used to identify apex frame for some micro-expression samples lack of apex frame labels. Then, TV-L1 optical flow method is used to extract the horizontal and vertical components map of optical flow between the onset frame and the apex frame of micro-expression, and the corresponding optical flow strain pattern map is derived according to the horizontal and vertical components of optical flow. The three optical flow maps are connected in the way of channel superposition to form an optical flow characteristic map of micro-expression. Finally, a kind of spatial attention unit with learnable parameters is designed in the convolutional neural network built by the Inception module, which makes the model pay more attention to the regions with micro-expression motion in the feature extraction process. In the spatial attention unit, two convolution kernels of 3×3 and 7×7 are used for spatial attention inference, so that the model can comprehensively consider the attention inference results of different scale convolution kernels. Experiments on the MEGC2019 comprehensive micro-expression datasets show that the accuracy of the method is 0.788, which is better than the existing automatic micro-expression recognition method.