基于多尺度多层次多任务网络的长视频微表情分析

Multi-scale multi-level multi-task network based micro-expression analysis for long videos

与宏表情不同,微表情具有持续时间短、动作幅度小、覆盖面积小的典型特点,长视频中微表情与宏表情交织在一起,使得微表情的检测和识别困难,并且严重依赖于专家经验。针对以上问题,开发了一种多任务模型用于长视频微表情分析,该模型采用级联的网络结构,分别完成检测子任务与识别子任务。针对微表情仅发生于面部局部区域且因个体差异特征分布不同导致关键帧定位不准或漏检,在检测子网络中采用Dual-CBAM-Inception模块,增强模型空间感受野,对全局与局部区域提取多尺度光流特征增强模型的鲁棒性;针对长视频中表情类别分布不均衡且微表情发生时面部动作细微导致长视频微表情分类识别准确率低,提出模型在识别子网络中采用深度可分离DenseNet模块,在控制模型的运算量和计算成本的前提下,通过多层次提取光流信息的浅层与深层语义特征提高模型的表情识别准确性。所提出的方法在CAS(ME)2长视频以及CASMEⅡ、SMIC短视频数据集上进行实验,结果表明,所提方法能够对长视频进行微表情检测并识别表情类别,性能优于当前诸多方法。

Unlike macro-expressions,micro-expressions are typically characterized by short duration,little movement amplitude,and less coverage area.Micro-expressions are intertwined with macro-expressions in long videos,making the spotting and recognition of micro-expressions more difficult and heavily dependent on expert experience.To address the problem,this paper develops a multi-task model for long video micro-expression analysis.It adopts a cascaded network structure to accomplish the spotting subtask and the recognition subtask respectively.Given the micro-expressions only occur in localized areas of the face and have different distribution of features due to individual differences,resulting in inaccurate spotted or missed detection of key frames,the Dual-CBAM-Inception module is employed in the spotting sub-network.This enhances the spatial sensing field of the model,and extracts multi-scale optical flow features for global and local regions to enhance the robustness of the model.The uneven distribution of expression categories in long videos and the subtle facial movements when micro-expressions occur lead to low accuracy of micro-expression classification and recognition in long videos.A depth-separable DenseNet Model is proposed in the recognition sub-network.The model improves the accuracy of expression recognition by extracting shallow and deep semantic features of optical flow information at multiple levels while controlling the amount of computation and computational cost.Our proposed method is validated on CAS(ME)2 long videos,as well as CASMEⅡand SMIC short video datasets.The results show it is able to spot micro-expression intervals and recognize expression categories for long videos.Moreover,it outperforms many current state-of-the-art methods.