改进伪三维残差网络的微表情识别方法

Micro-expression Recognition Method Based on Improved Pseudo-3DResNet

针对微表情特征捕捉难度大、识别率低存在的问题,提出一种改进伪三维残差网络的微表情识别方法。首先,预处理数据集样本后,在伪三维残差网络(Pseudo-3DResNet)设计阶段,针对基本3D残差单元层间顺序不合理、输出值不稳定及信息传播阻塞3个问题,设计了Pseudo-3D-SS、Pseudo-3D-PS、Pseudo-3D-SSS不同的瓶颈结构,并应用到4个残差块中。其次,将两个独立设计的卷积层和池化层应用在不同残差块间过滤表征不强的微表情序列,以进一步突出有价值的特征信息和去除冗余,实现时空特征提取。最后,为进一步降低亮度变化对光流特征提取的影响,提升追踪特征点速度,改进了基于L_(1)范数的全变分光流法,利用TV-OFM方法提取光流特征以得到每个微表情的水平光流序列和垂直光流序列。实验表明,所提方法相较于近期方法的未加权F_(1)值(UF_(1))、未加权平均召回率(UAR)分别提升3.61%、2.92%;在光流法比较实验中,所提方法追踪特征点的速度相较于比较方法提升30.08%;将包含改进的4个残差块变体与其他网络变体比较发现,所提模型的Avg(UF_(1)+UAR)提升2.5%,网络结构具有更强的泛化和特征提取能力;在层中叠加卷积和池化操作能更平衡地提取特征,进一步提高识别率,使模型更具鲁棒性和先进性。

Aiming at the problems of difficulty in capturing micro expression features and low recognition rate,an improved pseudo 3D residu⁃al network micro expression recognition method is proposed.Firstly,after preprocessing the dataset samples,different bottleneck structures were designed for Pseudo-3D-SS,Pseudo-3D-PS,and Pseudo-3D-SSS in the design phase of the Pseudo 3D Residual Network(Pseudo 3DResNet)to address three issues:unreasonable interlayer order of basic 3D residual units,unstable output values,and information propaga⁃tion blocking.These structures were applied to four residual blocks.Secondly,two independently designed convolutional layers and pooling layers are applied to filter weakly characterized micro expression sequences between different residual blocks,in order to further highlight valuable feature information and remove redundancy,achieving spatiotemporal feature extraction.Finally,in order to further reduce the im⁃pact of brightness changes on optical flow feature extraction and improve the speed of tracking feature points,the L_(1) norm based total variation optical flow method was improved.The TV-OFM method was used to extract optical flow features to obtain horizontal and vertical optical flow sequences for each micro expression.The experiment showed that the proposed method improved the unweighted F_(1) score(UF_(1))and unweight⁃ed average recall(UAR)by 3.61%and 2.92%,respectively,compared to recent methods;In the comparative experiment of optical flow meth⁃od,the proposed method improved the speed of tracking feature points by 30.08%compared to the comparative method;Comparing the four improved residual block variants with other network variants,it was found that the Avg(UF_(1)+UAR)of the proposed model increased by 2.5%,and the network structure has stronger generalization and feature extraction capabilities;Overlaying convolution and pooling operations in lay⁃ers can extract features more evenly,further improve the recognition rate,and make the model more robust and progressiveness.