一种基于GAOF的早期面瘫运动估计方法

面部表情运动估计是面瘫早期诊断和治疗评价等领域即将出现的一种重要的关键技术。本文提出了一种基于遗传算法的光流方法 (OpticalFlowMethodbasedonGeneticAlgorithm ,GAOF)进行早期面瘫运动估计。该方法从面部动态图像序列中估计面部微小运动矢量分布、特征参量及动态模式 ,用于早期面瘫的定位诊断与治疗定量评价。对一组正常被试者与一组面部神经肌肉运动异常病人进行实验 ,结果表明 :该方法与其他诊断方法相结合 ,对于早期面瘫定位诊断和恢复程度定量评价非常有效。与其他方法相比具有定位客观 。

Facial expression feature extraction method based on improved LBP

Local binary pattern(LBP)is an important method for texture feature extraction of facial expression.However,it also has the shortcomings of high dimension,slow feature extraction and noeffective local or global features extracted.To solve these problems,a facial expression feature extraction method is proposed based on improved LBP.Firstly,LBP is converted into double local binary pattern(DLBP).Then by combining Taylor expansion(TE)with DLBP,DLBP-TE algorithm is obtained.Finally,the DLBP-TE algorithm combined with extreme learning machine(ELM)is applied in seven kinds of ficial expression images and the corresponding experiments are carried out in Japanese adult female facial expression(JAFFE)database.The results show that the proposed method can significantly improve facial expression recognition rate.

Facial Expression Recognition Model Depending on Optimized Support Vector Machine

In computer vision,emotion recognition using facial expression images is considered an important research issue.Deep learning advances in recent years have aided in attaining improved results in this issue.According to recent studies,multiple facial expressions may be included in facial photographs representing a particular type of emotion.It is feasible and useful to convert face photos into collections of visual words and carry out global expression recognition.The main contribution of this paper is to propose a facial expression recognitionmodel(FERM)depending on an optimized Support Vector Machine(SVM).To test the performance of the proposed model(FERM),AffectNet is used.AffectNet uses 1250 emotion-related keywords in six different languages to search three major search engines and get over 1,000,000 facial photos online.The FERM is composed of three main phases:(i)the Data preparation phase,(ii)Applying grid search for optimization,and(iii)the categorization phase.Linear discriminant analysis(LDA)is used to categorize the data into eight labels(neutral,happy,sad,surprised,fear,disgust,angry,and contempt).Due to using LDA,the performance of categorization via SVM has been obviously enhanced.Grid search is used to find the optimal values for hyperparameters of SVM(C and gamma).The proposed optimized SVM algorithm has achieved an accuracy of 99%and a 98%F1 score.

Radial-curve-based facial expression recognition

A fully automatic facial-expression recognition （FER） system on 3D expression mesh models was proposed. The system didn＇ t need human interaction from the feature extraction stage till the facial expression classification stage. The features extracted from a 3D expression mesh mod- el were a bunch of radial facial curves to represent the spatial deformation of the geometry features on human face. Each facial curve was a surface line on the 3D face mesh model, begun from the nose tip and ended at the boundary of the previously trimmed 3D face points cloud. Then Euclid dis- tance was employed to calculate the difference between facial curves extracted from the neutral face and each face with different expressions of one person as feature. By employing support vector ma- chine （SVM） as classifier, the experimental results on the well-known 3D-BUFE dataset indicate that the proposed system could better classify the six prototypical facial expressions than state-of-art al- gorithms.

Facial expression recognition using threestage support vector machines

Herein,a three-stage support vector machine(SVM)for facial expression recognition is proposed.The first stage comprises 21 SVMs,which are all the binary combinations of seven expressions.If one expression is dominant,then the first stage will suffice;if two are dominant,then the second stage is used;and,if three are dominant,the third stage is used.These multilevel stages help reduce the possibility of experiencing an error as much as possible.Different image preprocessing stages are used to ensure that the features attained from the face detected have a meaningful and proper contribution to the classification stage.Facial expressions are created as a result of muscle movements on the face.These subtle movements are detected by the histogram-oriented gradient feature,because it is sensitive to the shapes of objects.The features attained are then used to train the three-stage SVM.Two different validation methods were used:the leave-one-out and K-fold tests.Experimental results on three databases(Japanese Female Facial Expression,Extended Cohn-Kanade Dataset,and Radboud Faces Database)show that the proposed system is competitive and has better performance compared with other works.

Landmarks-Driven Triplet Representation for Facial Expression Similarity

The facial landmarks can provide valuable information for expression-related tasks.However,most approaches only use landmarks for segmentation preprocessing or directly input them into the neural network for fully connection.Such simple combination not only fails to pass the spatial information to network,but also increases calculation amounts.The method proposed in this paper aims to integrate facial landmarks-driven representation into the triplet network.The spatial information provided by landmarks is introduced into the feature extraction process,so that the model can better capture the location relationship.In addition,coordinate information is also integrated into the triple loss calculation to further enhance similarity prediction.Specifically,for each image,the coordinates of 68 landmarks are detected,and then a region attention map based on these landmarks is generated.For the feature map output by the shallow convolutional layer,it will be multiplied with the attention map to correct the feature activation,so as to strengthen the key region and weaken the unimportant region.Finally,the optimized embedding output can be further used for downstream tasks.Three embeddings of three images output by the network can be regarded as a triplet representation for similarity computation.Through the CK+dataset,the effectiveness of such an optimized feature extraction is verified.After that,it is applied to facial expression similarity tasks.The results on the facial expression comparison(FEC)dataset show that the accuracy rate will be significantly improved after the landmark information is introduced.

Facial Expression Recognition by Split Rectangle Based Adaboost

The facial expression recognition systn using the Ariaboost based on the Split Rectangle feature is proposed in this paper. This system provides more various featmes in increasing speed and accuracy than the Haarolike featrue of Viola, which is commonly used for the Adaboost training algorithm. The Split Rectangle feature uses the nmsk-like shape composed with 2 independent rectangles, instead of using mask-like shape of Haar-like feature, which is composed of 2 --4 adhered rectangles of Viola. Split Rectangle feature has less di- verged operation than the Haar-like feaze. It also requires less oper- ation because the stun of pixels requires ordy two rectangles. Split Rectangle feature provides various and fast features to the Adaboost, which produrces the strong classifier with increased accuracy and speed. In the experiment, the system had 5.92 ms performance speed and 84 %--94 % accuracy by leaming 5 facial expressions, neutral, happiness, sadness, anger and surprise with the use of the Adaboost based on the Split Rectangle feature.

An Integrated Face Tracking and Facial Expression Recognition System

This article proposes a feature extraction method for an integrated face tracking and facial expression recognition in real time video. The method proposed by Viola and Jones [1] is used to detect the face region in the first frame of the video. A rectangular bounding box is fitted over for the face region and the detected face is tracked in the successive frames using the cascaded Support vector machine (SVM) and cascaded Radial basis function neural network (RBFNN). The haar-like features are extracted from the detected face region and they are used to create a cascaded SVM and RBFNN classifiers. Each stage of the SVM classifier and RBFNN classifier rejects the non-face regions and pass the face regions to the next stage in the cascade thereby efficiently tracking the face. The performance of tracking is evaluated using one hour video data. The performance of the cascaded SVM is compared with the cascaded RBFNN. The experiment results show that the proposed cascaded SVM classifier method gives better performance over the RBFNN and also the methods described in the literature using single SVM classifier [2]. While the face is being tracked, features are extracted from the mouth region for expression recognition. The features are modelled using a multi-class SVM. The SVM finds an optimal hyperplane to distinguish different facial expressions with an accuracy of 96.0%.

Research on the Novel Facial Expression Modelling Method

In this paper, we conduct research on novel and new facial expression modelling method. Although human facial expression recognition ability is stronger, but the computer to implement is a lot of diffi culties and the displays in: establish facial expression model and sentiment classifi cation, and put them with the changes in the facial features and expressions. Face is a fl exible body instead of rigid body, it is diffi cult to relate facial movement and facial expression change, according to the characteristics of the face image sequence established dynamic expression model that is a complete description of the dynamic expression of the process. Under this condition, in this paper, we propose the novel perspectives of the issues that are meaningful and innovative.

纤维肌痛综合征生物标记物的筛选及免疫细胞浸润分析

背景:纤维肌痛综合征作为常见风湿病,其发病与中枢敏化及免疫异常有关,但具体过程尚未阐明,缺乏特异性诊断标志物,不断探索该病的发病机制具有重要的临床意义。目的:基于加权基因共表达网络分析(WGCNA)等生物信息学方法和机器学习算法筛选纤维肌痛综合征潜在的诊断相关标志基因,并分析其免疫细胞浸润特征。方法:对来自基因表达综合数据库(GEO)的纤维肌痛综合征数据集转录谱进行差异分析和WGCNA分析,整合筛选出差异共表达基因,进一步采用机器学习套索回归(LASSO)算法、支持向量机递归特征消除(SVM-RFE)机器学习算法来识别核心生物标志物,并绘制受试者工作特征(ROC)曲线以评估诊断价值。最后,采用单样本基因集富集分析(ssGSEA)和基因集富集分析(GSEA)评估纤维肌痛综合征的免疫细胞浸润情况及通路富集。结果与结论:①对GSE67311数据集按照log2|(FC)|>0,P<0.05的条件进行差异分析后获得8个下调的差异表达基因;进行WGCNA分析后获得正相关性最高(r=0.22,P=0.04)的模块(MEdarkviolet)内含基因497个,负相关性最高(r=-0.41,P=6×10-5)的模块(MEsalmon2)内含基因19个;将差异表达基因与WGCNA的2个高相关性模块基因取交集,获得7个基因。②对上述7个基因进行LASSO回归算法筛选出4个基因,进行SVM-RFE机器学习算法筛选出5个基因,两者取交集后确定了3个核心基因,分别为重组1号染色体开放阅读框150蛋白(germinal center associated signaling and motility like,GCSAML)、整合素β8(Integrin beta-8,ITGB8)和羧肽酶A3(carboxypeptidase A3,CPA3);绘制3个核心基因的ROC曲线下面积分别为0.744,0.739,0.734,提示均具有很好的诊断价值,可作为纤维肌痛综合征的生物标志物。③免疫浸润分析结果显示,与对照组相比纤维肌痛综合征患者记忆B细胞、CD56 bright NK细胞和肥大细胞显著下调(P<0.05),且与上述3个生物标志物显著正相关(P<0.05)。④富集分析结果提示,纤维肌痛综合征的富集途径包括9条,主要与嗅觉传导、神经活性配体-受体相互作用及感染等通路密切相关。⑤上述结果显示,纤维肌痛综合征的发生发展与多基因参与、免疫调节异常及多个通路失调有关,但这些基因与免疫细胞之间的相互作用,以及它们与各通路之间的关系尚需进一步研究。

Fusion of visible and thermal images for facial expression recognition

Most present research into facial expression recognition focuses on the visible spectrum, which is sen- sitive to illumination change. In this paper, we focus on in- tegrating thermal infrared data with visible spectrum images for spontaneous facial expression recognition. First, the ac- tive appearance model AAM parameters and three defined head motion features are extracted from visible spectrum im- ages, and several thermal statistical features are extracted from infrared （IR） images. Second, feature selection is per- formed using the F-test statistic. Third, Bayesian networks BNs and support vector machines SVMs are proposed for both decision-level and feature-level fusion. Experiments on the natural visible and infrared facial expression （NVIE） spontaneous database show the effectiveness of the proposed methods, and demonstrate thermal 1R images＇ supplementary role for visible facial expression recognition.

Person-independent expression recognition based on person-similarity weighted expression feature

A new method to extract person-independent expression feature based on higher-order singular value decomposition （HOSVD） is proposed for facial expression recognition. Based on the assumption that similar persons have similar facial expression appearance and shape, the person-similarity weighted expression feature is proposed to estimate the expression feature of test persons. As a result, the estimated expression feature can reduce the influence of individuals caused by insufficient training data, and hence become less person-dependent. The proposed method is tested on Cohn-Kanade facial expression database and Japanese female facial expression （JAFFE） database. Person-independent experimental results show the superiority of the proposed method over the existing methods.

A Novel Efficient Patient Monitoring FER System Using Optimal DL-Features

Automated Facial Expression Recognition(FER)serves as the backbone of patient monitoring systems,security,and surveillance systems.Real-time FER is a challenging task,due to the uncontrolled nature of the environment and poor quality of input frames.In this paper,a novel FER framework has been proposed for patient monitoring.Preprocessing is performed using contrast-limited adaptive enhancement and the dataset is balanced using augmentation.Two lightweight efficient Convolution Neural Network(CNN)models MobileNetV2 and Neural search Architecture Network Mobile(NasNetMobile)are trained,and feature vectors are extracted.The Whale Optimization Algorithm(WOA)is utilized to remove irrelevant features from these vectors.Finally,the optimized features are serially fused to pass them to the classifier.A comprehensive set of experiments were carried out for the evaluation of real-time image datasets FER-2013,MMA,and CK+to report performance based on various metrics.Accuracy results show that the proposed model has achieved 82.5%accuracy and performed better in comparison to the state-of-the-art classification techniques in terms of accuracy.We would like to highlight that the proposed technique has achieved better accuracy by using 2.8 times lesser number of features.

Video expression recognition based on frame-level attention mechanism

Facial expression recognition(FER) in video has attracted the increasing interest and many approaches have been made.The crucial problem of classifying a given video sequence into several basic emotions is how to fuse facial features of individual frames.In this paper, a frame-level attention module is integrated into an improved VGG-based frame work and a lightweight facial expression recognition method is proposed.The proposed network takes a sub video cut from an experimental video sequence as its input and generates a fixed-dimension representation.The VGG-based network with an enhanced branch embeds face images into feature vectors.The frame-level attention module learns weights which are used to adaptively aggregate the feature vectors to form a single discriminative video representation.Finally, a regression module outputs the classification results.The experimental results on CK+and AFEW databases show that the recognition rates of the proposed method can achieve the state-of-the-art performance.

基于关键区域遮挡与重建的人脸表情识别

为了解决自然场景下人脸表情识别任务中的无用信息干扰和遮挡对识别性能的影响问题,提出一种基于关键区域遮挡与重建的人脸表情识别模型。利用多尺度特征提取网络,提取人脸图像的全局特征。根据68个人脸关键点划分出68个关键区域,并通过插值法提取68个关键区域的特征,同时采用注意力机制学习关键区域特征之间的先验关系。设计自监督的遮挡与重建模块,对关键区域特征进行随机遮挡,并利用已知区域信息来预测和重建被遮挡区域的特征,从而提高模型在自然场景下的表情识别性能。设计多个实验验证了该模型的泛化能力,并通过消融实验验证了模型中每个模块的有效性。实验结果表明,该模型在真实世界的情感面孔数据集(RAF-DB)和Occlusion-RAF-DB数据集上分别达到了88.44%和86.09%的识别准确率,相比于视觉Transformer(Vi T)等模型有效地提升了自然场景下人脸表情识别的性能。

融合rPPG和人脸三维法向量的非接触情绪感知技术

近年来,基于深度学习的情绪识别技术取得了显著进展。然而,现有方法主要集中在面部图像或视频上,忽略了其他模态信息,导致鲁棒性和稳定性不足。为了解决这一问题,提出了一种融合多模态信息的面部表情识别方法。首先将输入的人脸视频进行远程光电容积脉搏波(Remote Photo plethysmography,rPPG)信号和人脸三维法向量的提取,其次将这两种模态的信息输入其相应的情绪特征提取子网络,提取出对应的情绪特征向量。然后,将这两种模态提取出的情绪特征向量进行融合,生成一个丰富的特征向量,最后将其输入分类器进行情绪分类任务。通过这种多模态信息融合的方式,提高了面部表情识别的准确性和稳定性。对所提方法在不同数据集上进行了验证,实验结果表明,该方法在多样化面部表情识别中的表现优于当前先进的情感识别方法,具有更高的鲁棒性和稳定性。

基于改进残差网络的驾驶员情绪监测方法

针对驾驶员情绪实时监测问题,提出一种基于改进残差网络(ResNet)的表情识别方法,并设计了一种驾驶员情绪监测系统。系统首先对摄像头获取的驾驶员面部图像进行人脸检测与预处理,包括定位和白化处理,然后通过ResNet提取表情特征;其次,选用支持向量机(support vector machine,SVM)改进ResNet中的分类器,将表情特征输入SVM中进行情绪分类;最终,采用交叉熵损失函数,使用Fer2013数据集进行训练和实验,并利用迁移学习加速模型训练。实验结果表明,所提方法的识别准确率达到72.6%。与其他方法的对比分析验证了所提方法在驾驶员情绪监测中的有效性和实用性。

基于人脸表情特征的高校课堂教学质量在线评估模型

针对高校课堂教学质量在线评估模型识别率较低,评估过程主观性较强的问题,提出基于人脸表情特征的高校课堂教学质量在线评估模型。提取高校课堂人脸表情特征,利用图像层、S1层、C1层、S2层和C2层进行特征匹配和评选,使用贝叶斯分类模型对特征图像的平滑参数进行优化,确定使用率先验概率,判断学生的状态,评估课堂质量。实验结果表明,提出评估模型的评估率优于传统评估模型,在10~30 min内,学生的听课率最高,因此可以将重点问题和难点问题在第10~30 min内讲解,提高教学质量。

基于图像融合与深度学习的人脸表情识别

针对纹理特征提取方法单一及深度学习不能有效提取图像局部特征的问题,提出一种基于图像融合与深度学习的人脸表情识别方法。首先,对人脸表情图像分别提取局部二值模式(LBP)图像与韦伯局部描述符(WLD)图像;然后,将2种纹理图像进行融合作为输入图像送入改进后的残差神经网络(Res-Net)提取表情特征;将ResNet中的卷积核替换为空洞卷积,并在网络中添加改进后的注意力机制,使模型更加关注有效特征;最后,使用SoftMax进行表情分类。在JAFFE和CK+数据集上进行实验,准确率分别为97.0%与99.3%。实验结果表明,该方法能有效提高人脸表情识别的准确率。