基于多尺度时空Transformer的微表情识别

Micro-Expression Recognition Based on Multi-Scale Spatial-Temporal Transformer

由于微表情持续时间短,通常使用高帧率相机进行拍摄以保证捕获微表情完整的动态信息。然而与常规动态表情不同,微表情的面部肌肉运动微弱,导致微表情视频序列中帧与帧的变化极小,从而导致深度学习模型难以有效表征微表情的时序动态变化信息。为了解决这一难题,本文提出了一种基于时空Transformer模型的微表情识别方法。该方法通过在时间和空间两个维度分别设计尺度划分和关联关系分析的联合建模机制,寻找与微表情具有强关联关系的时间序列片段以及空间局部区域,从而学习到具有高判别性的特征用于识别微表情。为了验证所提方法的有效性,在CASMEⅡ、SMIC-HS和SAMM这3个数据库上进行微表情识别实验。实验结果表明,相比当前先进的微表情识别方法,本文所提出的时空Transformer模型取得了更加优异的性能。

Due to the short duration of micro-expressions,high frame rate cameras are usually used to capture the complete dynamic information of micro-expressions.However,unlike conventional dynamic expressions,the facial muscle movements of micro-expressions are weak,resulting in very small frame-to-frame changes in micro-expression video sequences,making it difficult for deep learning models to effectively represent the time-series dynamic change information of micro-expressions.In order to solve this problem,this paper proposes a micro-expression recognition method based on the spatio-temporal Transformer model.This method designs a joint modeling mechanism of scale division and correlation analysis in two dimensions of time and space and finds time series fragments and spatial local areas that have a strong correlation with micro-expressions,so as to learn highly discriminative features for micro-expression recognition.In order to verify the effectiveness of the method in this paper,micro-expression recognition experiments are carried out on three databases,CASMEⅡ,SMIC-HS,and SAMM.The experimental results show that compared with the current advanced micro-expression recognition methods,the spatio-temporal Transformer model proposed in this paper has achieved better performance.