Rational and Continuous Measurement of the Emotional Decision Making in Visual Recognition of Facial Emotional Expressions with M.A.R.I.E.: First Half

Context: The advent of Artificial Intelligence (AI) requires modeling prior to its implementation in algorithms for most human skills. This observation requires us to have a detailed and precise understanding of the interfaces of verbal and emotional communications. The progress of AI is significant on the verbal level but modest in terms of the recognition of facial emotions even if this functionality is one of the oldest in humans and is omnipresent in our daily lives. Dysfunction in the ability for facial emotional expressions is present in many brain pathologies encountered by psychiatrists, neurologists, psychotherapists, mental health professionals including social workers. It cannot be objectively verified and measured due to a lack of reliable tools that are valid and consistently sensitive. Indeed, the articles in the scientific literature dealing with Visual-Facial-Emotions-Recognition (ViFaEmRe), suffer from the absence of 1) consensual and rational tools for continuous quantified measurement, 2) operational concepts. We have invented a software that can use computer-morphing attempting to respond to these two obstacles. It is identified as the Method of Analysis and Research of the Integration of Emotions (M.A.R.I.E.). Our primary goal is to use M.A.R.I.E. to understand the physiology of ViFaEmRe in normal healthy subjects by standardizing the measurements. Then, it will allow us to focus on subjects manifesting abnormalities in this ability. Our second goal is to make our contribution to the progress of AI hoping to add the dimension of recognition of facial emotional expressions. Objective: To study: 1) categorical vs dimensional aspects of recognition of ViFaEmRe, 2) universality vs idiosyncrasy, 3) immediate vs ambivalent Emotional-Decision-Making, 4) the Emotional-Fingerprint of a face and 5) creation of population references data. Methods: With M.A.R.I.E. enable a rational quantified measurement of Emotional-Visual-Acuity (EVA) of 1) a) an individual observer, b) in a population aged 20 to 70 years old, 2) measure the range and intensity of expressed emotions by 3 Face-Tests, 3) quantify the performance of a sample of 204 observers with hyper normal measures of cognition, “thymia,” (ibid. defined elsewhere) and low levels of anxiety 4) analysis of the 6 primary emotions. Results: We have individualized the following continuous parameters: 1) “Emotional-Visual-Acuity”, 2) “Visual-Emotional-Feeling”, 3) “Emotional-Quotient”, 4) “Emotional-Deci-sion-Making”, 5) “Emotional-Decision-Making Graph” or “Individual-Gun-Trigger”6) “Emotional-Fingerprint” or “Key-graph”, 7) “Emotional-Finger-print-Graph”, 8) detecting “misunderstanding” and 9) detecting “error”. This allowed us a taxonomy with coding of the face-emotion pair. Each face has specific measurements and graphics. The EVA improves from ages of 20 to 55 years, then decreases. It does not depend on the sex of the observer, nor the face studied. In addition, 1% of people endowed with normal intelligence do not recognize emotions. The categorical dimension is a variable for everyone. The range and intensity of ViFaEmRe is idiosyncratic and not universally uniform. The recognition of emotions is purely categorical for a single individual. It is dimensional for a population sample. Conclusions: Firstly, M.A.R.I.E. has made possible to bring out new concepts and new continuous measurements variables. The comparison between healthy and abnormal individuals makes it possible to take into consideration the significance of this line of study. From now on, these new functional parameters will allow us to identify and name “emotional” disorders or illnesses which can give additional dimension to behavioral disorders in all pathologies that affect the brain. Secondly, the ViFaEmRe is idiosyncratic, categorical, and a function of the identity of the observer and of the observed face. These findings stack up against Artificial Intelligence, which cannot have a globalist or regionalist algorithm that can be programmed into a robot, nor can AI compete with human abilities and judgment in this domain. *Here “Emotional disorders” refers to disorders of emotional expressions and recognition.

Faster Region Convolutional Neural Network(FRCNN)Based Facial Emotion Recognition

Facial emotion recognition(FER)has become a focal point of research due to its widespread applications,ranging from human-computer interaction to affective computing.While traditional FER techniques have relied on handcrafted features and classification models trained on image or video datasets,recent strides in artificial intelligence and deep learning(DL)have ushered in more sophisticated approaches.The research aims to develop a FER system using a Faster Region Convolutional Neural Network(FRCNN)and design a specialized FRCNN architecture tailored for facial emotion recognition,leveraging its ability to capture spatial hierarchies within localized regions of facial features.The proposed work enhances the accuracy and efficiency of facial emotion recognition.The proposed work comprises twomajor key components:Inception V3-based feature extraction and FRCNN-based emotion categorization.Extensive experimentation on Kaggle datasets validates the effectiveness of the proposed strategy,showcasing the FRCNN approach’s resilience and accuracy in identifying and categorizing facial expressions.The model’s overall performance metrics are compelling,with an accuracy of 98.4%,precision of 97.2%,and recall of 96.31%.This work introduces a perceptive deep learning-based FER method,contributing to the evolving landscape of emotion recognition technologies.The high accuracy and resilience demonstrated by the FRCNN approach underscore its potential for real-world applications.This research advances the field of FER and presents a compelling case for the practicality and efficacy of deep learning models in automating the understanding of facial emotions.

Multi-Objective Equilibrium Optimizer for Feature Selection in High-Dimensional English Speech Emotion Recognition

Speech emotion recognition(SER)uses acoustic analysis to find features for emotion recognition and examines variations in voice that are caused by emotions.The number of features acquired with acoustic analysis is extremely high,so we introduce a hybrid filter-wrapper feature selection algorithm based on an improved equilibrium optimizer for constructing an emotion recognition system.The proposed algorithm implements multi-objective emotion recognition with the minimum number of selected features and maximum accuracy.First,we use the information gain and Fisher Score to sort the features extracted from signals.Then,we employ a multi-objective ranking method to evaluate these features and assign different importance to them.Features with high rankings have a large probability of being selected.Finally,we propose a repair strategy to address the problem of duplicate solutions in multi-objective feature selection,which can improve the diversity of solutions and avoid falling into local traps.Using random forest and K-nearest neighbor classifiers,four English speech emotion datasets are employed to test the proposed algorithm(MBEO)as well as other multi-objective emotion identification techniques.The results illustrate that it performs well in inverted generational distance,hypervolume,Pareto solutions,and execution time,and MBEO is appropriate for high-dimensional English SER.

E2E-MFERC:AMulti-Face Expression Recognition Model for Group Emotion Assessment

In smart classrooms, conducting multi-face expression recognition based on existing hardware devices to assessstudents’ group emotions can provide educators with more comprehensive and intuitive classroom effect analysis,thereby continuouslypromotingthe improvementof teaching quality.However,most existingmulti-face expressionrecognition methods adopt a multi-stage approach, with an overall complex process, poor real-time performance,and insufficient generalization ability. In addition, the existing facial expression datasets are mostly single faceimages, which are of low quality and lack specificity, also restricting the development of this research. This paperaims to propose an end-to-end high-performance multi-face expression recognition algorithm model suitable forsmart classrooms, construct a high-quality multi-face expression dataset to support algorithm research, and applythe model to group emotion assessment to expand its application value. To this end, we propose an end-to-endmulti-face expression recognition algorithm model for smart classrooms (E2E-MFERC). In order to provide highqualityand highly targeted data support for model research, we constructed a multi-face expression dataset inreal classrooms (MFED), containing 2,385 images and a total of 18,712 expression labels, collected from smartclassrooms. In constructing E2E-MFERC, by introducing Re-parameterization visual geometry group (RepVGG)block and symmetric positive definite convolution (SPD-Conv) modules to enhance representational capability;combined with the cross stage partial network fusion module optimized by attention mechanism (C2f_Attention),it strengthens the ability to extract key information;adopts asymptotic feature pyramid network (AFPN) featurefusion tailored to classroomscenes and optimizes the head prediction output size;achieves high-performance endto-end multi-face expression detection. Finally, we apply the model to smart classroom group emotion assessmentand provide design references for classroom effect analysis evaluation metrics. Experiments based on MFED showthat the mAP and F1-score of E2E-MFERC on classroom evaluation data reach 83.6% and 0.77, respectively,improving the mAP of same-scale You Only Look Once version 5 (YOLOv5) and You Only Look Once version8 (YOLOv8) by 6.8% and 2.5%, respectively, and the F1-score by 0.06 and 0.04, respectively. E2E-MFERC modelhas obvious advantages in both detection speed and accuracy, which can meet the practical needs of real-timemulti-face expression analysis in classrooms, and serve the application of teaching effect assessment very well.

Exploring Sequential Feature Selection in Deep Bi-LSTM Models for Speech Emotion Recognition

Machine Learning(ML)algorithms play a pivotal role in Speech Emotion Recognition(SER),although they encounter a formidable obstacle in accurately discerning a speaker’s emotional state.The examination of the emotional states of speakers holds significant importance in a range of real-time applications,including but not limited to virtual reality,human-robot interaction,emergency centers,and human behavior assessment.Accurately identifying emotions in the SER process relies on extracting relevant information from audio inputs.Previous studies on SER have predominantly utilized short-time characteristics such as Mel Frequency Cepstral Coefficients(MFCCs)due to their ability to capture the periodic nature of audio signals effectively.Although these traits may improve their ability to perceive and interpret emotional depictions appropriately,MFCCS has some limitations.So this study aims to tackle the aforementioned issue by systematically picking multiple audio cues,enhancing the classifier model’s efficacy in accurately discerning human emotions.The utilized dataset is taken from the EMO-DB database,preprocessing input speech is done using a 2D Convolution Neural Network(CNN)involves applying convolutional operations to spectrograms as they afford a visual representation of the way the audio signal frequency content changes over time.The next step is the spectrogram data normalization which is crucial for Neural Network(NN)training as it aids in faster convergence.Then the five auditory features MFCCs,Chroma,Mel-Spectrogram,Contrast,and Tonnetz are extracted from the spectrogram sequentially.The attitude of feature selection is to retain only dominant features by excluding the irrelevant ones.In this paper,the Sequential Forward Selection(SFS)and Sequential Backward Selection(SBS)techniques were employed for multiple audio cues features selection.Finally,the feature sets composed from the hybrid feature extraction methods are fed into the deep Bidirectional Long Short Term Memory(Bi-LSTM)network to discern emotions.Since the deep Bi-LSTM can hierarchically learn complex features and increases model capacity by achieving more robust temporal modeling,it is more effective than a shallow Bi-LSTM in capturing the intricate tones of emotional content existent in speech signals.The effectiveness and resilience of the proposed SER model were evaluated by experiments,comparing it to state-of-the-art SER techniques.The results indicated that the model achieved accuracy rates of 90.92%,93%,and 92%over the Ryerson Audio-Visual Database of Emotional Speech and Song(RAVDESS),Berlin Database of Emotional Speech(EMO-DB),and The Interactive Emotional Dyadic Motion Capture(IEMOCAP)datasets,respectively.These findings signify a prominent enhancement in the ability to emotional depictions identification in speech,showcasing the potential of the proposed model in advancing the SER field.

Impact of propofol and sevoflurane anesthesia on cognition and emotion in gastric cancer patients undergoing radical resection

BACKGROUND Propofol and sevoflurane are commonly used anesthetic agents for maintenance anesthesia during radical resection of gastric cancer.However,there is a debate concerning their differential effects on cognitive function,anxiety,and depression in patients undergoing this procedure.AIM To compare the effects of propofol and sevoflurane anesthesia on postoperative cognitive function,anxiety,depression,and organ function in patients undergoing radical resection of gastric cancer.METHODS A total of 80 patients were involved in this research.The subjects were divided into two groups:Propofol group and sevoflurane group.The evaluation scale for cognitive function was the Loewenstein occupational therapy cognitive assessment(LOTCA),and anxiety and depression were assessed with the aid of the self-rating anxiety scale(SAS)and self-rating depression scale(SDS).Hemodynamic indicators,oxidative stress levels,and pulmonary function were also measured.RESULTS The LOTCA score at 1 d after surgery was significantly lower in the propofol group than in the sevoflurane group.Additionally,the SAS and SDS scores of the sevoflurane group were significantly lower than those of the propofol group.The sevoflurane group showed greater stability in heart rate as well as the mean arterial pressure compared to the propofol group.Moreover,the sevoflurane group displayed better pulmonary function and less lung injury than the propofol group.CONCLUSION Both propofol and sevoflurane could be utilized as maintenance anesthesia during radical resection of gastric cancer.Propofol anesthesia has a minimal effect on patients'pulmonary function,consequently enhancing their postoperative recovery.Sevoflurane anesthesia causes less impairment on patients'cognitive function and mitigates negative emotions,leading to an improved postoperative mental state.Therefore,the selection of anesthetic agents should be based on the individual patient's specific circumstances.

Impaired implicit emotion regulation in patients with panic disorder:An event-related potential study on affect labeling

BACKGROUND Panic disorder(PD)involves emotion dysregulation,but its underlying mechanisms remain poorly understood.Previous research suggests that implicit emotion regulation may play a central role in PD-related emotion dysregulation and symptom maintenance.However,there is a lack of studies exploring the neural mechanisms of implicit emotion regulation in PD using neurophysiological indicators.AIM To study the neural mechanisms of implicit emotion regulation in PD with eventrelated potentials(ERP).METHODS A total of 25 PD patients and 20 healthy controls(HC)underwent clinical evaluations.The study utilized a case-control design with random sampling,selecting participants for the case group from March to December 2018.Participants performed an affect labeling task,using affect labeling as the experimental condition and gender labeling as the control condition.ERP and behavioral data were recorded to compare the late positive potential(LPP)within and between the groups.RESULTS Both PD and HC groups showed longer reaction times and decreased accuracy under the affect labeling.In the HC group,late LPP amplitudes exhibited a dynamic pattern of initial increase followed by decrease.Importantly,a significant group×condition interaction effect was observed.Simple effect analysis revealed a reduction in the differences of late LPP amplitudes between the affect labeling and gender labeling conditions in the PD group compared to the HC group.Furthermore,among PD patients under the affect labeling,the late LPP was negatively correlated with disease severity,symptom frequency,and intensity.CONCLUSION PD patients demonstrate abnormalities in implicit emotion regulation,hampering their ability to mobilize cognitive resources for downregulating negative emotions.The late LPP amplitude in response to affect labeling may serve as a potentially valuable clinical indicator of PD severity.

Sleep Quality and Emotional Adaptation among Freshmen in Elite Chinese Universities during Prolonged COVID-19 Lockdown:The Mediating Role of Anxiety Symptoms

Under the effects of COVID-19 and a number of ongoing lockdown tactics,anxiety symptoms and poor sleep quality have become common mental health issues among college freshmen and are intimately related to their emotional adaptation.To explore this connection,this study gathered data from a sample of 256 freshmen enrolled in an elite university in China in September 2022.The association between sleep quality,anxiety symptoms,and emotional adaptation was clarified using correlation analysis.Additionally,the mediating function of anxiety symptoms between sleep quality and emotional adaptation was investigated using a structural equation model.The results reveal that:(1)Chinese elite university freshmen who were subjected to prolonged lockdown had poor sleep quality and mild anxiety symptoms;(2)a significant positive correlation between poor sleep quality and anxiety symptoms was identified;(3)anxiety symptoms were found to have a significant negative impact on emotional adaptation;(4)poor sleep quality had a negative impact on emotional adaptation through anxiety symptoms.This research makes a valuable contribution by offering insights into the intricate relationship between sleep quality and emotional adaptation among freshmen in elite Chinese universities during prolonged lockdown conditions,and it is beneficial for schools and educators to further improve school schedules and psychological health initiatives.

Effects of psychological intervention on negative emotions and psychological resilience in breast cancer patients after radical mastectomy

Breast cancer(BC)is the most common malignant tumor in women,and the treatment process not only results in physical pain but also significant psychological distress in patients.Psychological intervention(PI)has been recognized as an important approach in treating postoperative psychological disorders in BC patients.It has been proven that PI has a significant therapeutic effect on postoperative psychological disorders,improving patients'negative emotions,enhancing their psychological resilience,and effectively enhancing their quality of life and treatment compliance.

Sepsis one-hour bundle management combined with psychological intervention on negative emotion and sleep quality in patients with sepsis

BACKGROUND Sepsis is a serious infectious disease caused by various systemic inflammatory responses and is ultimately life-threatening.Patients usually experience depression and anxiety,which affect their sleep quality and post-traumatic growth levels.AIM To investigate the effects of sepsis,a one-hour bundle(H1B)management was combined with psychological intervention in patients with sepsis.METHODS This retrospective analysis included 300 patients with sepsis who were admitted to Henan Provincial People’s Hospital between June 2022 and June 2023.According to different intervention methods,the participants were divided into a simple group(SG,n=150)and combined group(CG,n=150).H1B management was used in the SG and H1B management combined with psychological intervention was used in the CG.The changes of negative emotion,sleep quality and post-traumatic growth and prognosis were compared between the two groups before(T0)and after(T1)intervention.RESULTS After intervention(T1),the scores of the Hamilton Anxiety scale and Hamilton Depression scale in the CG were significantly lower than those in the SG(P<0.001).Sleep time,sleep quality,sleep efficiency,daytime dysfunction,sleep disturbance dimension score,and the total score in the CG were significantly lower than those in the SG(P<0.001).The appreciation of life,mental changes,relationship with others,personal strength dimension score,and total score of the CG were significantly higher than those of the SG(P<0.001).The scores for mental health,general health status,physiological function,emotional function,physical pain,social function,energy,and physiological function in the CG were significantly higher than those in the SG(P<0.001).The mechanical ventilation time,intensive care unit stay time,and 28-d mortality of the CG were significantly lower than those of the SG(P<0.05).CONCLUSION H1B management combined with psychological intervention can effectively alleviate the negative emotions of patients with sepsis and increase their quality of sleep and life.

The Emotional Intelligence and the Associated Factors among Nursing Students

Introduction: Emotional intelligence, or the capacity to cope one’s emotions, makes it simpler to form good connections with others and do caring duties. Nursing students can enroll a health team in a helpful and beneficial way with the use of emotional intelligence. Nurses who can identify, control, and interpret both their own emotions and those of their patients provide better patient care. The purpose of this study was to assess the emotional intelligence and to investigate the relationship and differences between emotional intelligence and demographic characteristics of nursing students. Methods: A cross-sectional study was carried out on 381 nursing students. Data collection was completed by “Schutte Self Report Emotional Intelligence Test”. Data were analyzed with the Statistical Package for Social Science. An independent t test, ANOVA, and Pearson correlation, multiple linear regression were used. Results: The results revealed that the emotional intelligence mean was 143.1 ± 21.6 (ranging from 33 to 165), which is high. Also, the analysis revealed that most of the participants 348 (91.3%) had higher emotional intelligence level. This finding suggests that nursing students are emotionally intelligent and may be able to notice, analyze, control, manage, and harness emotion in an adaptive manner. Also, academic year of nursing students was a predictor of emotional intelligence. Furthermore, there was positive relationship between the age and emotional intelligence (p < 0.05). The students’ ability to use their EI increased as they rose through the nursing grades. Conclusion: This study confirmed that the emotional intelligence score of the nursing students was high. Also, academic year of nursing students was a predictor of emotional intelligence. In addition, a positive relationship was confirmed between the emotional intelligence and age of nursing students. .

Film and Television Website Scores Authenticity Verification Based on the Emotional Analysis

Sentiment analysis is a method to identify and understand the emotion in the text through NLP and text analysis. In the era of information technology, there is often a certain error between the comments on the movie website and the actual score of the movie, and sentiment analysis technology provides a new way to solve this problem. In this paper, Python is used to obtain the movie review data from the Douban platform, and the model is constructed and trained by using naive Bayes and Bi-LSTM. According to the index, a better Bi-LSTM model is selected to classify the emotion of users’ movie reviews, and the classification results are scored according to the classification results, and compared with the real ratings on the website. According to the error of the final comparison results, the feasibility of this technology in the scoring direction of film reviews is being verified. By applying this technology, the phenomenon of film rating distortion in the information age can be prevented and the rights and interests of film and television works can be safeguarded.

Analysis of the Effect of Outpatient Nursing Intervention on Hypoglycemic Treatment Effect and Psychological Emotions of Diabetic Patients

Objective:To explore the effect of outpatient nursing interventions on the hypoglycemic treatment and psychological emotions of diabetic patients.Methods:148 patients who came to our hospital for outpatient treatment from February 2022 to October 2023 were selected and divided into a control group and an observation group,with 74 cases per group,according to the random number table method.The control group received routine nursing intervention,and the observation group received outpatient nursing intervention based on the control group.The two groups were observed for their effects of hypoglycemic treatment and psychological and emotional improvement before and after outpatient nursing intervention.Results:The health behavior scores of the control group were lower than that of the observation group;the post-intervention fasting blood glucose,2h postprandial blood glucose,anxiety self-rating scale(SAS),and the depression self-rating scale(SDS)of the control group were significantly higher than that of the observation group,and the difference was statistically significant(P<0.01).Conclusion:Outpatient nursing intervention encouraged patients to comply with healthy behaviors and helped control blood sugar levels.Patients’anxiety,depression,and other adverse psychological states were also improved hence the outpatient nursing intervention is worthy of further promotion.

Patterns of Interactions of the Complex City System:Emotional Urban Objects as Triggering Agents-A Secondary Publication

This article presents an analysis of the patterns of interactions resulting from the positive and negative emotional events that occur in cities,considering them as complex systems.It explores,from the imaginaries,how certain urban objects can act as emotional agents and how these events affect the urban system as a whole.An adaptive complex systems perspective is used to analyze these patterns.The results show patterns in the processes and dynamics that occur in cities based on the objects that affect the emotions of the people who live there.These patterns depend on the characteristics of the emotional charge of urban objects,but they can be generalized in the following process:(1)immediate reaction by some individuals;(2)emotions are generated at the individual level which begins to generalize,permuting to a collective emotion;(3)a process of reflection is detonated in some individuals from the reading of collective emotions;(4)integration/significance in the community both at the individual and collective level,on the concepts,roles and/or functions that give rise to the process in the system.Therefore,it is clear that emotions play a significant role in the development of cities and these aspects should be considered in the design strategies of all kinds of projects for the city.Future extensions of this work could include a deeper analysis of specific emotional events in urban environments,as well as possible implications for urban policy and decision making.

Correlation between critical thinking and emotional intelligence:a national crosssectional study on operating room nursing students in Iran

Objective:According to the World Federation of Medical Education,critical thinking should be part of the training of medical and paramedical students.Professionals can improve the quality of care of patients after surgery by having or acquiring this skill in health care.Also,Emotional intelligence is introduced as an impor tant and effective factor on the professional performance and mental health of healthcare professionals.Thus,the present study was designed and implemented to determine the relationship between emotional intelligence and critical thinking among operating room nursing students of medical sciences universities in Iran.Methods:This cross-sectional study was done on 420 operating room students in 10 top medical sciences universities of Iran in 2022.The sampling method in this research was multistage sampling.The data collection instruments included demographic characteristics,Rickett's critical thinking,and Bradberry-Greaves'emotional intelligence questionnaires.After receiving the ethics code,data collection was done for 2 months.For data analysis,descriptive and inferential analyses including independent t-tests,analysis of variance,and Pearson correlation were used.The collected data were analyzed by SPSS 18(IBM Corporation,Armonk,New York,United States).P-value<0.05 was considered significant.Results:The mean age of the students participating in this study was 23.02±3.70 years,with women constituting 67.4%of them.The results of data analysis indicated that the mean total score of critical thinking and emotional intelligence was 124.10±37.52 and 114.12±43.63,respectively.A direct significant correlation between critical thinking and emotional intelligence(r=0.459,P-value<0.001)and a significant relationship between gender and emotional intelligence(P-value=0.028)were found.Conclusions:Based on the present study results,educational managers in the Ministry of Health are suggested to consider suitable educational programs for improving critical thinking and emotional intelligence to enhance the quality of care provided by students in operating rooms.

Text Augmentation-Based Model for Emotion Recognition Using Transformers

Emotion Recognition in Conversations(ERC)is fundamental in creating emotionally intelligentmachines.Graph-BasedNetwork(GBN)models have gained popularity in detecting conversational contexts for ERC tasks.However,their limited ability to collect and acquire contextual information hinders their effectiveness.We propose a Text Augmentation-based computational model for recognizing emotions using transformers(TA-MERT)to address this.The proposed model uses the Multimodal Emotion Lines Dataset(MELD),which ensures a balanced representation for recognizing human emotions.Themodel used text augmentation techniques to producemore training data,improving the proposed model’s accuracy.Transformer encoders train the deep neural network(DNN)model,especially Bidirectional Encoder(BE)representations that capture both forward and backward contextual information.This integration improves the accuracy and robustness of the proposed model.Furthermore,we present a method for balancing the training dataset by creating enhanced samples from the original dataset.By balancing the dataset across all emotion categories,we can lessen the adverse effects of data imbalance on the accuracy of the proposed model.Experimental results on the MELD dataset show that TA-MERT outperforms earlier methods,achieving a weighted F1 score of 62.60%and an accuracy of 64.36%.Overall,the proposed TA-MERT model solves the GBN models’weaknesses in obtaining contextual data for ERC.TA-MERT model recognizes human emotions more accurately by employing text augmentation and transformer-based encoding.The balanced dataset and the additional training samples also enhance its resilience.These findings highlight the significance of transformer-based approaches for special emotion recognition in conversations.

Knowledge-enriched joint-learning model for implicit emotion cause extraction

Emotion cause extraction(ECE)task that aims at extracting potential trigger events of certain emotions has attracted extensive attention recently.However,current work neglects the implicit emotion expressed without any explicit emotional keywords,which appears more frequently in application scenarios.The lack of explicit emotion information makes it extremely hard to extract emotion causes only with the local context.Moreover,an entire event is usually across multiple clauses,while existing work merely extracts cause events at clause level and cannot effectively capture complete cause event information.To address these issues,the events are first redefined at the tuple level and a span-based tuple-level algorithm is proposed to extract events from different clauses.Based on it,a corpus for implicit emotion cause extraction that tries to extract causes of implicit emotions is constructed.The authors propose a knowledge-enriched jointlearning model of implicit emotion recognition and implicit emotion cause extraction tasks(KJ-IECE),which leverages commonsense knowledge from ConceptNet and NRC_VAD to better capture connections between emotion and corresponding cause events.Experiments on both implicit and explicit emotion cause extraction datasets demonstrate the effectiveness of the proposed model.

Image Emotion Classification Network Based on Multilayer Attentional Interaction,Adaptive Feature Aggregation

The image emotion classification task aims to use the model to automatically predict the emotional response of people when they see the image.Studies have shown that certain local regions are more likely to inspire an emotional response than the whole image.However,existing methods perform poorly in predicting the details of emotional regions and are prone to overfitting during training due to the small size of the dataset.Therefore,this study proposes an image emotion classification network based on multilayer attentional interaction and adaptive feature aggregation.To perform more accurate emotional region prediction,this study designs a multilayer attentional interaction module.The module calculates spatial attention maps for higher-layer semantic features and fusion features through amultilayer shuffle attention module.Through layer-by-layer up-sampling and gating operations,the higher-layer features guide the lower-layer features to learn,eventually achieving sentiment region prediction at the optimal scale.To complement the important information lost by layer-by-layer fusion,this study not only adds an intra-layer fusion to the multilayer attention interaction module but also designs an adaptive feature aggregation module.The module uses global average pooling to compress spatial information and connect channel information from all layers.Then,the module adaptively generates a set of aggregated weights through two fully connected layers to augment the original features of each layer.Eventually,the semantics and details of the different layers are aggregated through gating operations and residual connectivity to complement the lost information.To reduce overfitting on small datasets,the network is pre-trained on the FI dataset,and further weight fine-tuning is performed on the small dataset.The experimental results on the FI,Twitter I and Emotion ROI(Region of Interest)datasets show that the proposed network exceeds existing image emotion classification methods,with accuracies of 90.27%,84.66%and 84.96%.

Design of Hierarchical Classifier to Improve Speech Emotion Recognition

Automatic Speech Emotion Recognition(SER)is used to recognize emotion from speech automatically.Speech Emotion recognition is working well in a laboratory environment but real-time emotion recognition has been influenced by the variations in gender,age,the cultural and acoustical background of the speaker.The acoustical resemblance between emotional expressions further increases the complexity of recognition.Many recent research works are concentrated to address these effects individually.Instead of addressing every influencing attribute individually,we would like to design a system,which reduces the effect that arises on any factor.We propose a two-level Hierarchical classifier named Interpreter of responses(IR).Thefirst level of IR has been realized using Support Vector Machine(SVM)and Gaussian Mixer Model(GMM)classifiers.In the second level of IR,a discriminative SVM classifier has been trained and tested with meta information offirst-level classifiers along with the input acoustical feature vector which is used in primary classifiers.To train the system with a corpus of versatile nature,an integrated emotion corpus has been composed using emotion samples of 5 speech corpora,namely;EMO-DB,IITKGP-SESC,SAVEE Corpus,Spanish emotion corpus,CMU's Woogle corpus.The hierarchical classifier has been trained and tested using MFCC and Low-Level Descriptors(LLD).The empirical analysis shows that the proposed classifier outperforms the traditional classifiers.The proposed ensemble design is very generic and can be adapted even when the number and nature of features change.Thefirst-level classifiers GMM or SVM may be replaced with any other learning algorithm.

Leveraging hierarchical semantic‐emotional memory in emotional conversation generation

Handling emotions in human‐computer dialogues has emerged as a challenging task which requires artificial intelligence systems to generate emotional responses by jointly perceiving the emotion involved in the input posts and incorporating it into the gener-ation of semantically coherent and emotionally reasonable responses.However,most previous works generate emotional responses solely from input posts,which do not take full advantage of the training corpus and suffer from generating generic responses.In this study,we introduce a hierarchical semantic‐emotional memory module for emotional conversation generation(called HSEMEC),which can learn abstract semantic conver-sation patterns and emotional information from the large training corpus.The learnt semantic and emotional knowledge helps to enrich the post representation and assist the emotional conversation generation.Comprehensive experiments on a large real‐world conversation corpus show that HSEMEC can outperform the strong baselines on both automatic and manual evaluation.For reproducibility,we release the code and data publicly at:https://github.com/siat‐nlp/HSEMEC‐code‐data.