Methodical Analysis of Western-Caucasian and East-Asian Basic Facial Expressions of Emotions Based on Specific Facial Regions

Facial expressions are the straight link for showing human emotions. Psychologists have established the universality of six prototypic basic facial expressions of emotions which they believe are consistent among cultures and races. However, some recent cross-cultural studies have questioned and to some degree refuted this cultural universality. Therefore, in order to contribute to the theory of cultural specificity of basic expressions, from a composite viewpoint of psychology and HCI (Human Computer Interaction), this paper presents a methodical analysis of Western-Caucasian and East-Asian prototypic expressions focused on four facial regions: forehead, eyes-eyebrows, mouth and nose. Our analysis is based on facial expression recognition and visual analysis of facial expression images of two datasets composed by four standard databases CK+, JAFFE, TFEID and JACFEE. A hybrid feature extraction method based on Fourier coefficients is proposed for the recognition analysis. In addition, we present a cross-cultural human study applied to 40 subjects as a baseline, as well as one comparison of facial expression recognition performance between the previous cross-cultural tests from the literature. With this work, it is possible to clarify the prior considerations for working with multicultural facial expression recognition and contribute to identifying the specific facial expression differences between Western-Caucasian and East-Asian basic expressions of emotions.

Facial expression recognition based on fuzzy-LDA/CCA

A novel fuzzy linear discriminant analysis method by the canonical correlation analysis （fuzzy-LDA/CCA）is presented and applied to the facial expression recognition. The fuzzy method is used to evaluate the degree of the class membership to which each training sample belongs. CCA is then used to establish the relationship between each facial image and the corresponding class membership vector, and the class membership vector of a test image is estimated using this relationship. Moreover, the fuzzy-LDA/CCA method is also generalized to deal with nonlinear discriminant analysis problems via kernel method. The performance of the proposed method is demonstrated using real data.

Facial Expression Recognition Using Enhanced Convolution Neural Network with Attention Mechanism

Facial Expression Recognition(FER)has been an interesting area of research in places where there is human-computer interaction.Human psychol-ogy,emotions and behaviors can be analyzed in FER.Classifiers used in FER have been perfect on normal faces but have been found to be constrained in occluded faces.Recently,Deep Learning Techniques(DLT)have gained popular-ity in applications of real-world problems including recognition of human emo-tions.The human face reflects emotional states and human intentions.An expression is the most natural and powerful way of communicating non-verbally.Systems which form communications between the two are termed Human Machine Interaction(HMI)systems.FER can improve HMI systems as human expressions convey useful information to an observer.This paper proposes a FER scheme called EECNN(Enhanced Convolution Neural Network with Atten-tion mechanism)to recognize seven types of human emotions with satisfying results in its experiments.Proposed EECNN achieved 89.8%accuracy in classi-fying the images.

Facial Expression Recognition Model Depending on Optimized Support Vector Machine

In computer vision,emotion recognition using facial expression images is considered an important research issue.Deep learning advances in recent years have aided in attaining improved results in this issue.According to recent studies,multiple facial expressions may be included in facial photographs representing a particular type of emotion.It is feasible and useful to convert face photos into collections of visual words and carry out global expression recognition.The main contribution of this paper is to propose a facial expression recognitionmodel(FERM)depending on an optimized Support Vector Machine(SVM).To test the performance of the proposed model(FERM),AffectNet is used.AffectNet uses 1250 emotion-related keywords in six different languages to search three major search engines and get over 1,000,000 facial photos online.The FERM is composed of three main phases:(i)the Data preparation phase,(ii)Applying grid search for optimization,and(iii)the categorization phase.Linear discriminant analysis(LDA)is used to categorize the data into eight labels(neutral,happy,sad,surprised,fear,disgust,angry,and contempt).Due to using LDA,the performance of categorization via SVM has been obviously enhanced.Grid search is used to find the optimal values for hyperparameters of SVM(C and gamma).The proposed optimized SVM algorithm has achieved an accuracy of 99%and a 98%F1 score.

An Analysis Model of Learners’ Online Learning Status Based on Deep Neural Network and Multi-Dimensional Information Fusion

The learning status of learners directly affects the quality of learning.Compared with offline teachers,it is difficult for online teachers to capture the learning status of students in the whole class,and it is even more difficult to continue to pay attention to studentswhile teaching.Therefore,this paper proposes an online learning state analysis model based on a convolutional neural network and multi-dimensional information fusion.Specifically,a facial expression recognition model and an eye state recognition model are constructed to detect students’emotions and fatigue,respectively.By integrating the detected data with the homework test score data after online learning,an analysis model of students’online learning status is constructed.According to the PAD model,the learning state is expressed as three dimensions of students’understanding,engagement and interest,and then analyzed from multiple perspectives.Finally,the proposed model is applied to actual teaching,and procedural analysis of 5 different types of online classroom learners is carried out,and the validity of the model is verified by comparing with the results of the manual analysis.

Vision-Based Learning Status Monitoring on Color and Depth Live Facial Videos

This study analyzes live facial videos for recognizing nonverbal learning-related facial movements and head poses to discover the learning status of students. First, color and depth facial videos captured by a Kinect are analyzed for face tracking using a three-dimensional （3D） active appearance model （AAM）. Second, the facial feature vector sequences are used to train hidden Markov models （HMMs） to recognize seven learning-related facial movements （smile, blink, frown, shake, nod, yawn, and talk）. The final stage involves the analysis of the facial movement vector sequence to evaluate three status scores （understanding, interaction, and consciousness）, each represents the learning status of a student and is helpful to both teachers and students for improving teaching and learning. Five teaching activities demonstrate that the proposed learning status analysis system promotes the interpersonal communication between teachers and students.

Sentiment Analysis on Social Media Using Genetic Algorithm with CNN

There are various intense forces causing customers to use evaluated data when using social media platforms and microblogging sites.Today,customers throughout the world share their points of view on all kinds of topics through these sources.The massive volume of data created by these customers makes it impossible to analyze such data manually.Therefore,an efficient and intelligent method for evaluating social media data and their divergence needs to be developed.Today,various types of equipment and techniques are available for automatically estimating the classification of sentiments.Sentiment analysis involves determining people’s emotions using facial expressions.Sentiment analysis can be performed for any individual based on specific incidents.The present study describes the analysis of an image dataset using CNNswithPCA intended to detect people’s sentiments(specifically,whether a person is happy or sad).This process is optimized using a genetic algorithm to get better results.Further,a comparative analysis has been conducted between the different models generated by changing the mutation factor,performing batch normalization,and applying feature reduction using PCA.These steps are carried out across five experiments using theKaggledataset.The maximum accuracy obtained is 96.984%,which is associated with the Happy and Sad sentiments.

Analysis of cDNA sequence,protein structure and expression of parotid secretory protein in pig

Parotid secretory protein (PSP) secreted abundantly in saliva, whose function is related with the anti-bacterial effect. The PSP cDNA has been isolated from pig parotid glands by 3′ and 5′ rapid amplification of cDNA end (RACE), based on the conserved signal peptide region among the known mammalian PSP. The result of homologous comparison shows that pig PSP and human PSP shares the high identity at the level of the primary, secondary and tertiary protein structure. A search for functionally significant protein motifs revealed a unique amino acid sequence pattern consisting of the residues Leu-X(6)-Leu-X(6)-Leu- X(7)-Leu-X(6)-Leu-X(6)-Leu near the amino-terminal portion of the protein, which is important to its function. RT-PCR, Dot blot and Northern blot analysis demonstrated that PSP was strongly expressed in parotid glands, but not in other tissues.

Game Outlier Behavior Detection System Based on Dynamic Time Warp Algorithm

This paper proposes a methodology for using multi-modal data in gameplay to detect outlier behavior.The proposedmethodology collects,synchronizes,and quantifies time-series data fromwebcams,mouses,and keyboards.Facial expressions are varied on a one-dimensional pleasure axis,and changes in expression in the mouth and eye areas are detected separately.Furthermore,the keyboard and mouse input frequencies are tracked to determine the interaction intensity of users.Then,we apply a dynamic time warp algorithm to detect outlier behavior.The detected outlier behavior graph patterns were the play patterns that the game designer did not intend or play patterns that differed greatly from those of other users.These outlier patterns can provide game designers with feedback on the actual play experiences of users of the game.Our results can be applied to the game industry as game user experience analysis,enabling a quantitative evaluation of the excitement of a game.

Microarray Analysis Using Rank Order Statistics for ARCH Residual Empirical Process

Statistical two-group comparisons are widely used to identify the significant differentially expressed (DE) signatures against a therapy response for microarray data analysis. We applied a rank order statistics based on an Autoregressive Conditional Heteroskedasticity (ARCH) residual empirical process to DE analysis. This approach was considered for simulation data and publicly available datasets, and was compared with two-group comparison by original data and Auto-regressive (AR) residual. The significant DE genes by the ARCH and AR residuals were reduced by about 20% - 30% to these genes by the original data. Almost 100% of the genes by ARCH are covered by the genes by the original data unlike the genes by AR residuals. GO enrichment and Pathway analyses indicate the consistent biological characteristics between genes by ARCH residuals and original data. ARCH residuals array data might contribute to refining the number of significant DE genes to detect the biological feature as well as ordinal microarray data.

How Facial Expressions of Recipients Influence Online Prosocial Behaviors?-Evidence from Big Data Analysis on Tencent Gongyi Platform

Cyberspace has significantly influenced people’s perceptions of social interactions and communication.As a result,the conventional theories of kin selection and reciprocal altruism fall short in completely elucidating online prosocial behavior.Based on the social information processing model,we propose an analytical framework to explain the donation behaviors on online platform.Through collecting textual and visual data from Tencent Gongyi platform pertaining to disease relief projects,and employing techniques encompassing text analysis,image analysis,and propensity score matching,we investigate the impact of both internal emotional cues and external contextual cues on donation behaviors.It is found that positive emotions tend to attract a larger number of donations,while negative emotions tend to result in higher per capita donation amounts.Furthermore,these effects manifest differently under distinct external contextual conditions.

干红葡萄酒涩感质量的多维表征及其效果分析

【目的】综合考虑涩感的时间依赖性、口腔感觉的亚品质属性以及品评员即时面部表情,设计干红葡萄酒涩感分析方法,为涩感质量的多维表征提供方法学支撑。【方法】干红葡萄酒的涩感时间依赖性采用时间强度法表征涩感的最大强度(Imax)、感知速率(Vi)、消解速率(Vd)、整体响应强度(AUC)和感知持续时间(Ttot);涩感的干燥、粗糙和褶皱等亚品质通过选择合适项分析法(CATA)和动态主导型感官属性法(TDS)表达;饮用愉悦度试验基于面部表情分析技术表征。以甘肃、宁夏和新疆产区27款‘赤霞珠’干红葡萄酒样品为试材,表征涩感的强度、亚品质特征以及感知愉悦度差异。【结果】3个产区酒样的涩感差异主要体现在涩感Imax、Vd以及AUC上,相比甘肃和新疆,宁夏酒样的涩感Imax、AUC和Vd最大。干燥、麻木、粗糙、褶皱、生青和颗粒6种亚品质的选择频率超过50%,是供试酒样涩感的主要属性。涩感多元指标的相关性分析得出,3个产区的干红葡萄酒涩感亚品质差异主要体现在粗糙、褶皱和干燥感上。干红葡萄酒涩感过高的Vi、Vd,过高的粗糙和麻木感会降低品评员的积极情绪,而颗粒感通常会导致高兴和惊讶的表情出现。供试酒样涩感多元数据的主成分分析得出,本研究设计的涩感多维表征技术方法具有较强的酒样涩感区分能力。宁夏酒样有较高的Imax、AUC、Vi和Vd,粗糙、褶皱、干燥和麻木感更明显;新疆酒样的颗粒感较强,Imax、AUC、生青和麻木感较弱,饮用的舒适度较好;甘肃酒样拥有最强的生青感和最弱的粗糙优势率。【结论】本研究设计的干红葡萄酒多维表征方法能够具象、细致地表征产品涩感的多样性,较为科学地评价涩感的差异,具有应用推广价值。

微表情面部肌电跨模态分析及标注算法

长久以来,微表情的小样本问题始终制约着微表情分析的发展,而小样本问题归根到底是因为微表情的数据标注十分困难。本研究希望借助面部肌电作为技术手段,从微表情数据自动标注、半自动标注和无标注三个方面各提出一套解决方案。对于自动标注,提出基于面部远端肌电的微表情自动标注方案;对于半自动标注,提出基于单帧标注的微表情起止帧自动标注;对于无标注,提出了基于肌电信号的跨模态自监督学习算法。同时,本研究还希望借助肌电模态,对微表情的呈现时间和幅度等机理特征进行拓展研究。

Pose-robust feature learning for facial expression recognition

Automatic facial expression recognition （FER） from non-frontal views is a challenging research topic which has recently started to attract the attention of the research community. Pose variations are difficult to tackle and many face analysis methods require the use of sophisticated nor- malization and initialization procedures. Thus head-pose in- variant facial expression recognition continues to be an is- sue to traditional methods. In this paper, we propose a novel approach for pose-invariant FER based on pose-robust fea- tures which are learned by deep learning methods -- prin- cipal component analysis network （PCANet） and convolu- tional neural networks （CNN） （PRP-CNN）. In the first stage, unlabeled frontal face images are used to learn features by PCANet. The features, in the second stage, are used as the tar- get of CNN to learn a feature mapping between frontal faces and non-frontal faces. We then describe the non-frontal face images using the novel descriptions generated by the maps, and get unified descriptors for arbitrary face images. Finally, the pose-robust features are used to train a single classifier for FER instead of training multiple models for each spe- cific pose. Our method, on the whole, does not require pose/ landmark annotation and can recognize facial expression in a wide range of orientations. Extensive experiments on two public databases show that our framework yields dramatic improvements in facial expression analysis.

学生课堂表情识别系统的设计与实现

在智慧教育领域,实时监测学生课堂专注度对于提升教学质量具有十分重要的意义。本文开发一款基于表情识别的学生课堂专注度分析系统。系统基于C/S架构,采用Python编程语言和Pyside6图形界面框架,并使用Fer2013数据集对Mini-Xception网络模型进行训练和测试。测试结果表明,系统能够较为准确地辨识出学生课堂听讲期间的面部表情,从而可以辅助教师掌握学生的学习专注状态,为个性化教学的开展提供基础数据支撑。

表情对大学生志愿行为的预测作用

目的探讨厌恶情绪对志愿行为倾向的预测作用。方法采用2(测量阶段:前测、后测)×3(组别:集体道德视频组、个人道德视频组、控制组)的混合实验设计,通过道德提升感启动任务,考察大学生在3种视频观看任务中产生的厌恶情绪,及其与志愿行为倾向的关系。结果在观看个人道德行为视频和风景视频时,厌恶情绪负向预测志愿行为倾向(F=4.37,P<0.01;F=9.94,P<0.001);在观看集体道德行为视频时,厌恶情绪对志愿行为倾向预测作用不显著。结论道德提升感启动过程中伴随的厌恶情绪对志愿行为倾向有预测效应,未来研究者可以运用表情分析技术,进一步探索志愿行为的心理机制。

基于面部表情识别的学生课堂状态分析系统

近年来,随着计算机视觉的发展,人机交互技术也逐渐应用于教育领域,助力推动教育数字化转型。针对传统课堂中教师无法照顾到每一位学生对知识的掌握程度,制约教学质量的问题,利用Jetson Nano开发套件设计了一种基于面部表情识别的学生课堂状态分析系统,该系统实时检测每个学生面部表情的变化信息并及时反馈给教师,进而帮助教师了解学生的课堂状态,及时调整授课方式,提高教学质量。该系统结构包括摄像头实时采集模块、人脸检测模块、基于深度学习的面部表情识别模型等。所设计的学生课堂状态分析系统识别准确率高、实时性好,对于建设智慧课堂具有很高的应用和推广价值。

智慧学习环境中基于面部表情的情感分析

情感与认知加工之间存在着密不可分的联系,学习过程中的情感状态对学习效果有一定的影响。在智慧学习环境中实现学习者情感分析,有利于促进智慧学习的发生。表情作为人类情感表达的主要方式,其中蕴含了大量有关内心情感变化的信息,通过面部表情人们可以推断内心微妙的情感状态。目前,人脸检测技术已经实现了从复杂背景中定位人脸,分类算法也相对成熟,因此表情识别的研究工作主要集中在表情特征提取上,而现有研究基本上都是基于人脸与表情的混合特征进行的识别,这产生了较大的干扰。在表情识别时,理想情况是将个体相关的人脸特征和与个体无关的表情特征相分离。依据心理学家Ekman提出的FACS(面部表情编码系统)构建的智慧学习环境下基于面部表情识别的情感分析框架,通过特征分解将个体特征及表情特征分解到不同的子空间,在表情子空间中进行表情识别,从而排除个体特征对表情识别的干扰。经JAFFE表情库的验证,表情识别结果比较理想,已在三维虚拟学习平台Magic Learning的师生情感交互子系统上实现了基于面部表情的学习者情感识别及情感干预。