Micro-Expression Recognition Based on Spatio-Temporal Feature Extraction of Key Regions

Aiming at the problems of short duration,low intensity,and difficult detection of micro-expressions(MEs),the global and local features of ME video frames are extracted by combining spatial feature extraction and temporal feature extraction.Based on traditional convolution neural network(CNN)and long short-term memory(LSTM),a recognition method combining global identification attention network(GIA),block identification attention network(BIA)and bi-directional long short-term memory(Bi-LSTM)is proposed.In the BIA,the ME video frame will be cropped,and the training will be carried out by cropping into 24 identification blocks(IBs),10 IBs and uncropped IBs.To alleviate the overfitting problem in training,we first extract the basic features of the preprocessed sequence through the transfer learning layer,and then extract the global and local spatial features of the output data through the GIA layer and the BIA layer,respectively.In the BIA layer,the input data will be cropped into local feature vectors with attention weights to extract the local features of the ME frames;in the GIA layer,the global features of the ME frames will be extracted.Finally,after fusing the global and local feature vectors,the ME time-series information is extracted by Bi-LSTM.The experimental results show that using IBs can significantly improve the model’s ability to extract subtle facial features,and the model works best when 10 IBs are used.

Micro-expression recognition algorithm based on graph convolutional network and Transformer model

Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.

Adaptive spatio-temporal attention neural network for cross-database micro-expression recognition

Background The use of micro-expression recognition to recognize human emotions is one of the most critical challenges in human-computer interaction applications. In recent years, cross-database micro-expression recognition(CDMER) has emerged as a significant challenge in micro-expression recognition and analysis. Because the training and testing data in CDMER come from different micro-expression databases, CDMER is more challenging than conventional micro-expression recognition. Methods In this paper, an adaptive spatio-temporal attention neural network(ASTANN) using an attention mechanism is presented to address this challenge. To this end, the micro-expression databases SMIC and CASME II are first preprocessed using an optical flow approach,which extracts motion information among video frames that represent discriminative features of micro-expression.After preprocessing, a novel adaptive framework with a spatiotemporal attention module was designed to assign spatial and temporal weights to enhance the most discriminative features. The deep neural network then extracts the cross-domain feature, in which the second-order statistics of the sample features in the source domain are aligned with those in the target domain by minimizing the correlation alignment(CORAL) loss such that the source and target databases share similar distributions. Results To evaluate the performance of ASTANN, experiments were conducted based on the SMIC and CASME II databases under the standard experimental evaluation protocol of CDMER. The experimental results demonstrate that ASTANN outperformed other methods in relevant crossdatabase tasks. Conclusions Extensive experiments were conducted on benchmark tasks, and the results show that ASTANN has superior performance compared with other approaches. This demonstrates the superiority of our method in solving the CDMER problem.

Review of micro-expression spotting and recognition in video sequences

Facial micro-expressions are short and imperceptible expressions that involuntarily reveal the true emotions that a person may be attempting to suppress,hide,disguise,or conceal.Such expressions can reflect a person's real emotions and have a wide range of application in public safety and clinical diagnosis.The analysis of facial micro-expressions in video sequences through computer vision is still relatively recent.In this research,a comprehensive review on the topic of spotting and recognition used in micro expression analysis databases and methods,is conducted,and advanced technologies in this area are summarized.In addition,we discuss challenges that remain unresolved alongside future work to be completed in the field of micro-expression analysis.

Gender-Specific Multi-Task Micro-Expression Recognition Using Pyramid CGBP-TOP Feature

Micro-expression recognition has attracted growing research interests in the field of compute vision.However,micro-expression usually lasts a few seconds,thus it is difficult to detect.This paper presents a new framework to recognize micro-expression using pyramid histogram of Centralized Gabor Binary Pattern from Three Orthogonal Panels(CGBP-TOP)which is an extension of Local Gabor Binary Pattern from Three Orthogonal Panels feature.CGBP-TOP performs spatial and temporal analysis to capture the local facial characteristics of micro-expression image sequences.In order to keep more local information of the face,CGBP-TOP is extracted based on pyramid subregions of the micro-expression video frame.The combination of CGBP-TOP and spatial pyramid can represent well and truly the facial movements of the micro-expression image sequences.However,the dimension of our pyramid CGBP-TOP tends to be very high,which may lead to high data redundancy problem.In addition,it is clear that people of different genders usually have different ways of micro-expression.Therefore,in this paper,in order to select the relevant features of micro-expression,the gender-specific sparse multi-task learning method with adaptive regularization term is adopted to learn a compact subset of pyramid CGBP-TOP feature for micro-expression classification of different sexes.Finally,extensive experiments on widely used CASME II and SMIC databases demonstrate that our method can efficiently extract micro-expression motion features in the micro-expression video clip.Moreover,our proposed approach achieves comparable results with the state-of-the-art methods.

Micro-expression recognition algorithm based on the combination of spatial and temporal domains

Aiming at the problem of unsatisfactory effects of traditional micro-expression recognition algorithms,an efficient micro-expression recognition algorithm is proposed,which uses convolutional neural networks(CNN)to extract spatial features of micro-expressions,and long short-term memory network(LSTM)to extract time domain features.CNN and LSTM are combined as the basis of micro-expression recognition.In many CNN structures,the visual geometry group(VGG)using a small convolution kernel is finally selected as the pre-network through comparison.Due to the difficulty of deep learning training and over-fitting,the dropout method and batch normalization method are used to solve the problem in the VGG network.Two data sets CASME and CASME II are used for test comparison,in order to solve the problem of insufficient data sets,randomly determine the starting frame,and a fixedlength frame sequence is used as the standard,and repeatedly read all sample frames of the entire data set to achieve trayersal and data amplification.Finallv.a hieh recognition rate of 67.48% is achieved.

An improved micro-expression recognition algorithm of 3D convolutional neural network

The micro-expression lasts for a very short time and the intensity is very subtle.Aiming at the problem of its low recognition rate,this paper proposes a new micro-expression recognition algorithm based on a three-dimensional convolutional neural network(3D-CNN),which can extract two-di-mensional features in spatial domain and one-dimensional features in time domain,simultaneously.The network structure design is based on the deep learning framework Keras,and the discarding method and batch normalization(BN)algorithm are effectively combined with three-dimensional vis-ual geometry group block(3D-VGG-Block)to reduce the risk of overfitting while improving training speed.Aiming at the problem of the lack of samples in the data set,two methods of image flipping and small amplitude flipping are used for data amplification.Finally,the recognition rate on the data set is as high as 69.11%.Compared with the current international average micro-expression recog-nition rate of about 67%,the proposed algorithm has obvious advantages in recognition rate.

基于图案解析与视觉在线检测的电喷印控制方法

随着柔性传感器的发展,传统加工工艺难以满足其高精度生产要求。电喷印作为一种新兴的微纳加工工艺,在复杂图案加工和高精度加工方面具备明显优势。研发了一种具备反馈功能的电喷印软件控制系统,实现了图案解析和机器视觉的在线监测。首先研究了不同形式打印目标的复杂图案解析方法,进行了响应曲面法设计实验,设计了对应的实时在线视觉监测算法和复杂图案的打印路径的优化算法,建立了电喷印关键工作参数的预测数学模型,并进行了测试分析。结果表明:该控制系统实现了打印过程的自动反馈,显著缩短了工作时间,使得电喷印过程的泰勒锥可准确地按需控制,打印线段相对偏差的统计值由±11%左右,减小至±6%左右,并打印出了最小半径为103μm的液滴和最小线宽为151μm的线段。研究成果为柔性微纳制造工艺提供了更高效、精确的生产方法。

Multi-scale joint feature network for micro-expression recognition

Micro-expression recognition is a substantive cross-study of psychology and computer science,and it has a wide range of applications(e.g.,psychological and clinical diagnosis,emotional analysis,criminal investigation,etc.).However,the subtle and diverse changes in facial muscles make it difficult for existing methods to extract effective features,which limits the improvement of micro-expression recognition accuracy.Therefore,we propose a multi-scale joint feature network based on optical flow images for micro-expression recognition.First,we generate an optical flow image that reflects subtle facial motion information.The optical flow image is then fed into the multi-scale joint network for feature extraction and classification.The proposed joint feature module(JFM)integrates features from different layers,which is beneficial for the capture of micro-expression features with different amplitudes.To improve the recognition ability of the model,we also adopt a strategy for fusing the feature prediction results of the three JFMs with the backbone network.Our experimental results show that our method is superior to state-of-the-art methods on three benchmark datasets(SMIC,CASME II,and SAMM)and a combined dataset(3 DB).

Micro-Expression Recognition Algorithm Based on Information Entropy Feature

The intensity of the micro-expression is weak,although the directional low frequency components in the image are preserved by many algorithms,the extracted micro-expression ft^ature information is not sufficient to accurately represent its sequences.In order to improve the accuracy of micro-expression recognition,first,each frame image is extracted from,its sequences,and the image frame is pre-processed by using gray normalization,size normalization,and two-dimensional principal component analysis(2DPCA);then,the optical flow method is used to extract the motion characteristics of the reduced-dimensional image,the information entropy value of the optical flow characteristic image is calculated by the information entropy principle,and the information entropy value is analyzed to obtain the eigenvalue.Therefore,more micro-expression feature information is extracted,including more important information,which can further improve the accuracy of micro-expression classification and recognition;finally,the feature images are classified by using the support vector machine(SVM).The experimental results show that the micro-expression feature image obtained by the information entropy statistics can effectively improve the accuracy of micro-expression recognition.

空洞卷积网络下微表情实时识别方法仿真

微表情与普通面部表情不同,是一种面部动作变化微弱、持续时间极短的面部活动,且需要在视频中分析,因此特征提取较为困难。为了解决上述问题,提出基于空洞卷积的实时微表情识别算法。通过混沌蛙跳算法,对人脸微表情图像增强处理。采用时间差值法和局部二值法,提取人脸特征信息。结合空洞卷积构建卷积神经网络,并将提取到的人脸特征输入到构建的卷积神经网络,完成人脸微表情的实时识别。实验结果表明,所提方法的实时微表情识别准确率在97%以上,且识别时间短,说明所提方法具有较好的实际应用价值。

基于TIVC5410 DSP的数字语音识别实时系统

数字信号处理 ( DSP)技术的迅速发展 ,为语音识别的实时实现提供了可能。本文尝试采用 TI公司新型号的DSP芯片 ,建立一个汉语数字的语音实时识别系统 ,实验结果表明 。

用于盲人视觉辅助的多目标快速识别并同步测距方法

提出一种以摄像头实现的可用于盲人视觉辅助的多运动目标快速识别并同步测距方法.该方法以深度学习多目标检测算法(single shot multibox detector,SSD)识别各类目标,并通过SSD输出的目标类别及检测框(bounding box)高度来提出测距模型,从而同步地获取多个目标的测量距离.本方法仅通过普通摄像头便能识别较多类物体且识别类别数量可拓展,能够将测距模块和障碍物识别模块同步执行,从而可对多个物体实时识别并同步测距.实验结果表明,本方法能有效地识别障碍物,具有良好的测距精度,为盲人视觉辅助的一种有效探索.