E2E-MFERC:AMulti-Face Expression Recognition Model for Group Emotion Assessment

In smart classrooms, conducting multi-face expression recognition based on existing hardware devices to assessstudents’ group emotions can provide educators with more comprehensive and intuitive classroom effect analysis,thereby continuouslypromotingthe improvementof teaching quality.However,most existingmulti-face expressionrecognition methods adopt a multi-stage approach, with an overall complex process, poor real-time performance,and insufficient generalization ability. In addition, the existing facial expression datasets are mostly single faceimages, which are of low quality and lack specificity, also restricting the development of this research. This paperaims to propose an end-to-end high-performance multi-face expression recognition algorithm model suitable forsmart classrooms, construct a high-quality multi-face expression dataset to support algorithm research, and applythe model to group emotion assessment to expand its application value. To this end, we propose an end-to-endmulti-face expression recognition algorithm model for smart classrooms (E2E-MFERC). In order to provide highqualityand highly targeted data support for model research, we constructed a multi-face expression dataset inreal classrooms (MFED), containing 2,385 images and a total of 18,712 expression labels, collected from smartclassrooms. In constructing E2E-MFERC, by introducing Re-parameterization visual geometry group (RepVGG)block and symmetric positive definite convolution (SPD-Conv) modules to enhance representational capability;combined with the cross stage partial network fusion module optimized by attention mechanism (C2f_Attention),it strengthens the ability to extract key information;adopts asymptotic feature pyramid network (AFPN) featurefusion tailored to classroomscenes and optimizes the head prediction output size;achieves high-performance endto-end multi-face expression detection. Finally, we apply the model to smart classroom group emotion assessmentand provide design references for classroom effect analysis evaluation metrics. Experiments based on MFED showthat the mAP and F1-score of E2E-MFERC on classroom evaluation data reach 83.6% and 0.77, respectively,improving the mAP of same-scale You Only Look Once version 5 (YOLOv5) and You Only Look Once version8 (YOLOv8) by 6.8% and 2.5%, respectively, and the F1-score by 0.06 and 0.04, respectively. E2E-MFERC modelhas obvious advantages in both detection speed and accuracy, which can meet the practical needs of real-timemulti-face expression analysis in classrooms, and serve the application of teaching effect assessment very well.

A Preliminary Discussion on Watercolor Creation and Emotional Expression of Female Themes in the 20th Century

The history of watercolor painting spans over three centuries,and the study of watercolor creations focusing on female themes have been a hot topic in the subfield of art studies.Watercolor originated in the Age of Exploration,initially serving as a practical tool for natural history,and later evolving into an art form for the aristocracy.In the 20th century,creations featuring female themes flourished and reached their peak.Compared to early works portraying female theses in mythology and religion,these paintings exhibit significant differences in expression,color,and texture,reflecting the gradual rise of women’s status and the artists’reflections on modern society.This article delves into the development,emotional expression,creation,and inspiration of watercolor paintings with female themes.By analyzing representative works,we explore the artists’portrayal of women and the reflection of women’s living conditions,social status,and ideological consciousness in contemporary society.This article also examines the profound influence and artistic value of these works on contemporary female-themed paintings from the perspective of art studies.

Exploration of the Expression of Color Emotion in Oil Painting Sketch Creation

This paper deeply analyzes the expression of color emotion in oil painting sketch creation.Starting with the three basic attributes of color theory(hue,lightness,and purity),this paper discusses its emotional symbolic significance and the relationship between contrast and harmony.By interpreting the works of artists such as Van Gogh’s Sunflower,Monet’s Rouen Cathedral,and Cézanne’s Mont Sainte-Victoire,this paper shows the unique charm of different colors in conveying emotions,creating atmosphere,and expressing themes.At the same time,it is expounded that in the creation of an oil painting sketch,the effective expression of color emotion can be realized by observing nature,using subjective colors,and reasonable composition and layout of colors,so as to enhance the artistic value of the works.

A Facial Expression Emotion Recognition Based Human-robot Interaction System

A facial expression emotion recognition based human-robot interaction(FEER-HRI) system is proposed, for which a four-layer system framework is designed. The FEERHRI system enables the robots not only to recognize human emotions, but also to generate facial expression for adapting to human emotions. A facial emotion recognition method based on2D-Gabor, uniform local binary pattern(LBP) operator, and multiclass extreme learning machine(ELM) classifier is presented,which is applied to real-time facial expression recognition for robots. Facial expressions of robots are represented by simple cartoon symbols and displayed by a LED screen equipped in the robots, which can be easily understood by human. Four scenarios,i.e., guiding, entertainment, home service and scene simulation are performed in the human-robot interaction experiment, in which smooth communication is realized by facial expression recognition of humans and facial expression generation of robots within 2 seconds. As a few prospective applications, the FEERHRI system can be applied in home service, smart home, safe driving, and so on.

Methodical Analysis of Western-Caucasian and East-Asian Basic Facial Expressions of Emotions Based on Specific Facial Regions

Facial expressions are the straight link for showing human emotions. Psychologists have established the universality of six prototypic basic facial expressions of emotions which they believe are consistent among cultures and races. However, some recent cross-cultural studies have questioned and to some degree refuted this cultural universality. Therefore, in order to contribute to the theory of cultural specificity of basic expressions, from a composite viewpoint of psychology and HCI (Human Computer Interaction), this paper presents a methodical analysis of Western-Caucasian and East-Asian prototypic expressions focused on four facial regions: forehead, eyes-eyebrows, mouth and nose. Our analysis is based on facial expression recognition and visual analysis of facial expression images of two datasets composed by four standard databases CK+, JAFFE, TFEID and JACFEE. A hybrid feature extraction method based on Fourier coefficients is proposed for the recognition analysis. In addition, we present a cross-cultural human study applied to 40 subjects as a baseline, as well as one comparison of facial expression recognition performance between the previous cross-cultural tests from the literature. With this work, it is possible to clarify the prior considerations for working with multicultural facial expression recognition and contribute to identifying the specific facial expression differences between Western-Caucasian and East-Asian basic expressions of emotions.

Multimodal Expression—Synthesis of Facial Emotion,Mouth Movement and Voice

This paper presents a multimodal system for synthesis of continuous voice and corresponding images of facial emotions. In the emotion synthesis, a general 2 D face model is established and mapped to a particular face by locating some key points of the facial image. The edges of eyes and mouth are approximated by Hough transformation on the proposed models, which has significant advantage over other methods of edge extraction of facial organs, such as deformable templates. A synthesized subsystem of text driven speech and mouth movement is obtained by using the method of emotion synthesis. The parameters for mouth movement are considered as the functions of original mouth shape input to meet the difference of mouth movements among different persons. The method of wave editing is used to synthesize speech, in which Chinese syllables are taken as the basic units to save time. Automatic transformation of mouth shape parameters, automatic synchronism of voice and mouth movement, and realtime synthesis ability are the three major features of this subsystem. The present system can synthesize continuous speech consisting of words in first and second standard Chinese word tables and the corresponding mouth movements.

EEG Mapping of Cortical Activation Related to Emotional Stroop with Facial Expressions: A TREFACE Study

TREFACE (Test for Recognition of Facial Expressions with Emotional Conflict) is a computerized model for investigating the emotional factor in executive functions based on the Stroop paradigm, for the recognition of emotional expressions in human faces. To investigate the influence of the emotional component at the cortical level, the electroencephalographic (EEG) recording technique was used to measure the involvement of cortical areas during the execution of certain tasks. Thirty Brazilian native Portuguese-speaking graduate students were evaluated on their anxiety and depression levels and on their well-being at the time of the session. The EEG recording was performed in 19 channels during the execution of the TREFACE test in the 3 stages established by the model-guided training, reading, and recognition—both with congruent conditions, when the image corresponds to the word shown, and incongruent condition, when there is no correspondence. The results showed better performance in the reading stage and in congruent conditions, while greater intensity of cortical activation in the recognition stage and in incongruent conditions. In a complementary way, specific frontal activations were observed: intense theta frequency activation in the left extension representing the frontal recruitment of posterior regions in information processing;also, activation in alpha frequency in the right frontotemporal line, illustrating the executive processing in the control of attention, in addition to the dorsal manifestation of the prefrontal side, for emotional performance. Activations in beta and gamma frequencies were displayed in a more intensely distributed way in the recognition stage. The results of this mapping of cortical activity in our study can help to understand how words and images of faces can be regulated in everyday life and in clinical contexts, suggesting an integrated model that includes the neural bases of the regulation strategy.

Color Expression of Architectural Emotion Language from the Perspective of Public Art

Against the background of "architecturalization" of public art, the significance of architectural color design lies not only in the visual element of architectural environment, but also in the emotion expression of city culture and spirit of the time. Architectural colors should be constructed from the 6 aspects, namely, conveying city spirit, coordinating the environment, showing regional characteristics, reflecting public concept, obtaining public sympathy, combining cultural connotations of color and functions of space. Moreover, the impact of regional culture on architectural color should be properly handled, the emotions of architecture for the city reflected on the surface of architecture, which can reverse the convergence of city images to some extent, and promote the diversity of regional humanistic architectural landscapes.

Material Expression of Architectural Emotion From the Perspective of Public Art

Architectural emotion, to a large extent, is reflected in the emotionalization of building materials. The exploration of the vitality and spiritual connotation of the building materials is a new direction of expressing architectural feelings. This paper explores the visual features and connotation of various building materials from the perspective of public art, with a view to understanding the characteristics of building materials and creating unique buildings with a strong character.

Multimodal Emotion Recognition Based on Facial Expression and ECG Signal

As a key link in human-computer interaction,emotion recognition can enable robots to correctly perceive user emotions and provide dynamic and adjustable services according to the emotional needs of different users,which is the key to improve the cognitive level of robot service.Emotion recognition based on facial expression and electrocardiogram has numerous industrial applications.First,three-dimensional convolutional neural network deep learning architecture is utilized to extract the spatial and temporal features from facial expression video data and electrocardiogram(ECG)data,and emotion classification is carried out.Then two modalities are fused in the data level and the decision level,respectively,and the emotion recognition results are then given.Finally,the emotion recognition results of single-modality and multi-modality are compared and analyzed.Through the comparative analysis of the experimental results of single-modality and multi-modality under the two fusion methods,it is concluded that the accuracy rate of multi-modal emotion recognition is greatly improved compared with that of single-modal emotion recognition,and decision-level fusion is easier to operate and more effective than data-level fusion.

Harmonies and Emotion Expression in Piano Works – Based on sigh in piano works by Joe Hisaishi

Piano is a type of western musical instrument. Since its creation, people have been deeply attracted by its melodious sound and elegant temperament. In order to express emotions better with the piano, a lot of musicians and music amateurs have made joint efforts to create many piano playing skills, wherein harmony is a typical representative. Enjoying high fame in the world, Japanese animation has become one representative culture of this country. Success of Japanese animation is attributed to many factors. Except for real emotional expression as well as abundant and varied styles, the exquisite background music has played a dominant role. Joe Hisaishi is a Japanese pianist and composer enjoying the high reputation. He cooperates with Miyazaki Hayao –a Japanese cartoon master very tacitly. Close relations have been built between them based on the win-win goals. Application of sigh in Joe Hisaishi’s piano works can accurately express ideals and emotions of ？ gures, forming a unique style in his music works. The paper researches harmony application and expression of ideas and emotions in piano works based on application of sigh in Joe Hisa ishi’s works. In order to realize in-depth study, the paper also introduces the cartoonist Miyazaki Hayao who is closely related with Joe Hisaishi. Through combination of them, readers can understand the research more easily.

Emotional Expression and Artistic Reconstruction in Musical Performance

The emotional expression brought by music is the self-realization and artistic reconstruction of the art of music in performance.From the perspective of the development of musical performance,artistic reconstruction is an important means of expression which does not only explores the emotional connotation of music but also endows music with stronger vitality through the performers'understanding and imagination,presenting them to the audience in a more three-dimensional way and stirring up deep resonance.Performers can also gradually develop their own style of performance.How to better integrate the interaction between the two is an important proposition in exploring musical performance.This paper focuses on the relationship between emotional expression and artistic reconstruction in addition to elaborating the important role of the two in musical performance to provide a useful reference for music creators and performers.

Human-Computer Interaction Using Deep Fusion Model-Based Facial Expression Recognition System

A deep fusion model is proposed for facial expression-based human-computer Interaction system.Initially,image preprocessing,i.e.,the extraction of the facial region from the input image is utilized.Thereafter,the extraction of more discriminative and distinctive deep learning features is achieved using extracted facial regions.To prevent overfitting,in-depth features of facial images are extracted and assigned to the proposed convolutional neural network(CNN)models.Various CNN models are then trained.Finally,the performance of each CNN model is fused to obtain the final decision for the seven basic classes of facial expressions,i.e.,fear,disgust,anger,surprise,sadness,happiness,neutral.For experimental purposes,three benchmark datasets,i.e.,SFEW,CK+,and KDEF are utilized.The performance of the proposed systemis compared with some state-of-the-artmethods concerning each dataset.Extensive performance analysis reveals that the proposed system outperforms the competitive methods in terms of various performance metrics.Finally,the proposed deep fusion model is being utilized to control a music player using the recognized emotions of the users.

Emotion Exploration in Autistic Children as an Early Biomarker through R-CNN

Autism Spectrum Disorder(ASD)is found to be a major concern among various occupational therapists.The foremost challenge of this neurodeve-lopmental disorder lies in the fact of analyzing and exploring various symptoms of the children at their early stage of development.Such early identification could prop up the therapists and clinicians to provide proper assistive support to make the children lead an independent life.Facial expressions and emotions perceived by the children could contribute to such early intervention of autism.In this regard,the paper implements in identifying basic facial expression and exploring their emotions upon a time-variant factor.The emotions are analyzed by incorporating the facial expression identified through Convolution Neural Network(CNN)using 68 landmark points plotted on the frontal face with a prediction network formed by Recurring Neural Network(RNN)known as the RCNN based Facial Expression Recommendation(FER)system.The paper adopts Recurring Convolution Neural Network(R-CNN)to take the advantage of increased accuracy and performance with decreased time complexity in predicting emotion as textual network analysis.The papers prove better accuracy in identifying the emotion in autistic children when compared over simple machine learning models built for such identifications contributing to autistic society.

A Multi-Modal Deep Learning Approach for Emotion Recognition

In recent years,research on facial expression recognition(FER)under mask is trending.Wearing a mask for protection from Covid 19 has become a compulsion and it hides the facial expressions that is why FER under the mask is a difficult task.The prevailing unimodal techniques for facial recognition are not up to the mark in terms of good results for the masked face,however,a multi-modal technique can be employed to generate better results.We proposed a multi-modal methodology based on deep learning for facial recognition under a masked face using facial and vocal expressions.The multimodal has been trained on a facial and vocal dataset.We have used two standard datasets,M-LFW for the masked dataset and CREMA-D and TESS dataset for vocal expressions.The vocal expressions are in the form of audio while the faces data is in image form that is why the data is heterogenous.In order to make the data homogeneous,the voice data is converted into images by taking spectrogram.A spectrogram embeds important features of the voice and it converts the audio format into the images.Later,the dataset is passed to the multimodal for training.neural network and the experimental results demonstrate that the proposed multimodal algorithm outsets unimodal methods and other state-of-the-art deep neural network models.

How do Patients with Schizophrenia Perceive Relatives’ Levels of Expressed Emotions? A Brief Patient Rated Scale

Background : Patients’ perspective on relatives’ attitude and behaviour towards them (Expressed emotion—EE) may be an important addition to the current focus on relatives’ perspective only, as measured by Camberwell Family Interview (CFI) or other methods. Based on the theory of EE, we have designed a brief, three-item questionnaire completed by patients, named Felt Expressed Emotion Rating Scale (FEERS). FEERS measures the patient’s experience of criticism (Cri) and emotional over involvement (i.e. worry (Wo), and control (Con). Aims: To investigate the test-retest reliability of the FEERS and associations between the FEERS and the CFI and to which extent FEERS scores were modified by severity of psychotic symptoms, cognitive function, patient mood and amount of face-to-face contact with relatives. Methods : Forty-five patients with schizophrenia and related psychoses admitted to a psychiatric hospital and 67 relatives were included. Assessments included FEERS, CFI and Positive and Negative Syndrome Scale (PANSS). Results : FEERS-Cri test-retest intra-class correlation (ICC1,1) was 0.71 among patients with low total PANSS scores, low cognitive impairment (0.59) and depression (0.63). For low levels of cognitive impairment, the ICCs of the FEERS-Wo and the FEERS-Con were 0.62 and 0.83, respectively. The FEERS-Cri and FEERSHowWo correlated significantly with CFI-CC and CFI-positive comments, respectively. Among the relatives that the patient deemed “not at all critical” (low FEERS-Cri scores), 94% had low CFI-CC levels. Conclusions : The FEERS may be a brief, time-saving alternative for identifying relatives with low levels of criticism. However, illness severity, cognitive function and mood influence FEERS test-retest reliability and link to CFI.

Bipolar I disorder and expressed emotion of families: A cohort study in Japan

The relationships between expressed emotion (EE) of the families and the course of bipolar disorder have been examined only in a limited number of cohort studies. No study has yet been reported from Asia. The subjects were 12 patients that had been diagnosed with bipolar I disorder according to DSM-IV and their 12 key family members. The families of the patients were interviewed using the Camberwell Family Interview (CFI) within 2 weeks of the admission of the patients, and their EE were evaluated. The patients were then followed up for 9 months after their discharge from the hospital. The patients were divided into a high-EE group and a low-EE group using the cut-off based on the number of critical comments (CC) and emotional overinvolvement (EOI), and the 9-month relapse risk was compared. When the subjects with 3 or more CC or an EOI score of 3 or more were regarded as the high-EE group, and the others as the low-EE group, the 9-month relapse risk was 100% (3/3) for the high EE group and 0% (0/9) for the low EE group. (Fisher’s exact test p = 0.005) EE based on the CFI appear to be correlated with relapse in bipolar I disorder in Japan.

Research on Improvement Strategies of Psychological Training and Stage Expressiveness in Percussion Performance

With the continuous development of music education,percussion,as an important form of performance,has led to growing attention to the psychological training of its performers.This study aims to explore how psychological factors in percussion performance impact stage expressiveness and to propose corresponding psychological training strategies.By analyzing relevant domestic and international literature,we found that psychological training not only enhances performers’confidence and alleviates performance anxiety but also contributes to an overall improvement in performance quality.This study shows that methods such as emotional management and cognitive restructuring exhibit promising application potential in practice.Therefore,exploring a systematic psychological training program is significant for improving the stage expressiveness of percussion performers.

Control Emotion Intensity for LSTM-Based Expressive Speech Synthesis

To improve the performance of human-computer interaction interfaces, emotion is considered to be one of the most important factors. The major objective of expressive speech synthesis is to inject various expressions reflecting different emotions to the synthesized speech. To effectively model and control the emotion, emotion intensity is introduced for expressive speech synthesis model to generate speech conveyed the delicate and complicate emotional states. The system was composed of an emotion analysis module with the goal of extracting control emotion intensity vector and a speech synthesis module responsible for mapping text characters to speech waveform. The proposed continuous variable “perception vector” is a data-driven approach of controlling the model to synthesize speech with different emotion intensities. Compared with the system using a one-hot vector to control emotion intensity, this model using perception vector is able to learn the high-level emotion information from low-level acoustic features. In terms of the model controllability and flexibility, both the objective and subjective evaluations demonstrate perception vector outperforms one-hot vector.