A Multimodal Analysis of Social Class in College English Textbook

As a required course at higher education level,English has permeated the everyday life of Chinese students from all so-cial backgrounds.Social class,deciding one’s access to various social and educational recourses,has a great influence on thelearning process of college students inside and outside the classroom.However,few studies have placed their focus on social classand college English textbooks.Thus,this paper attempts to adopt a multimodal analysis to find how the speaking tasks in collegeEnglish textbook represents the social identity of Chinese college students and to offer pedagogic implications.

A Robust Framework for Multimodal Sentiment Analysis with Noisy Labels Generated from Distributed Data Annotation

Multimodal sentiment analysis utilizes multimodal data such as text,facial expressions and voice to detect people’s attitudes.With the advent of distributed data collection and annotation,we can easily obtain and share such multimodal data.However,due to professional discrepancies among annotators and lax quality control,noisy labels might be introduced.Recent research suggests that deep neural networks(DNNs)will overfit noisy labels,leading to the poor performance of the DNNs.To address this challenging problem,we present a Multimodal Robust Meta Learning framework(MRML)for multimodal sentiment analysis to resist noisy labels and correlate distinct modalities simultaneously.Specifically,we propose a two-layer fusion net to deeply fuse different modalities and improve the quality of the multimodal data features for label correction and network training.Besides,a multiple meta-learner(label corrector)strategy is proposed to enhance the label correction approach and prevent models from overfitting to noisy labels.We conducted experiments on three popular multimodal datasets to verify the superiority of ourmethod by comparing it with four baselines.

A Multimodal Critical Discourse Analysis of Lingnan Cultural Promotional Videos on Social Media

In recent years,more and more directors of culture and tourism have taken part in the promotion of local cultural tourism by cross-dressing,talent shows,and pushing their limits on self-media platforms.This study investigates short videos of Lingnan culture promoted by directors general and deputy directors general of the Culture,Radio,Television,Tourism,and Sports Bureau of counties and cities in Guangdong Province on social media by the method of multimodal critical discourse analysis.The analysis of 33 videos shows that Lingnan culture is a domineering and confident culture,historical culture,graceful and elegant culture,and vibrant and active culture.Domineering and confident culture is embedded in the utterances and behaviors of the directors general or deputy directors general in the video.Historical culture is realized through the conversation with historical figures through time travel.Graceful and elegant culture is constructed in the depiction of sceneries and the depiction of characters’manners.Vibrant and active culture is represented in the depiction of the characters’actional process and analytical process.

Multimodal sentiment analysis for social media contents during public emergencies

Purpose:Nowadays,public opinions during public emergencies involve not only textual contents but also contain images.However,the existing works mainly focus on textual contents and they do not provide a satisfactory accuracy of sentiment analysis,lacking the combination of multimodal contents.In this paper,we propose to combine texts and images generated in the social media to perform sentiment analysis.Design/methodology/approach:We propose a Deep Multimodal Fusion Model(DMFM),which combines textual and visual sentiment analysis.We first train word2vec model on a large-scale public emergency corpus to obtain semantic-rich word vectors as the input of textual sentiment analysis.BiLSTM is employed to generate encoded textual embeddings.To fully excavate visual information from images,a modified pretrained VGG16-based sentiment analysis network is used with the best-performed fine-tuning strategy.A multimodal fusion method is implemented to fuse textual and visual embeddings completely,producing predicted labels.Findings:We performed extensive experiments on Weibo and Twitter public emergency datasets,to evaluate the performance of our proposed model.Experimental results demonstrate that the DMFM provides higher accuracy compared with baseline models.The introduction of images can boost the performance of sentiment analysis during public emergencies.Research limitations:In the future,we will test our model in a wider dataset.We will also consider a better way to learn the multimodal fusion information.Practical implications:We build an efficient multimodal sentiment analysis model for the social media contents during public emergencies.Originality/value:We consider the images posted by online users during public emergencies on social platforms.The proposed method can present a novel scope for sentiment analysis during public emergencies and provide the decision support for the government when formulating policies in public emergencies.

Leveraging Vision-Language Pre-Trained Model and Contrastive Learning for Enhanced Multimodal Sentiment Analysis

Multimodal sentiment analysis is an essential area of research in artificial intelligence that combines multiple modes,such as text and image,to accurately assess sentiment.However,conventional approaches that rely on unimodal pre-trained models for feature extraction from each modality often overlook the intrinsic connections of semantic information between modalities.This limitation is attributed to their training on unimodal data,and necessitates the use of complex fusion mechanisms for sentiment analysis.In this study,we present a novel approach that combines a vision-language pre-trained model with a proposed multimodal contrastive learning method.Our approach harnesses the power of transfer learning by utilizing a vision-language pre-trained model to extract both visual and textual representations in a unified framework.We employ a Transformer architecture to integrate these representations,thereby enabling the capture of rich semantic infor-mation in image-text pairs.To further enhance the representation learning of these pairs,we introduce our proposed multimodal contrastive learning method,which leads to improved performance in sentiment analysis tasks.Our approach is evaluated through extensive experiments on two publicly accessible datasets,where we demonstrate its effectiveness.We achieve a significant improvement in sentiment analysis accuracy,indicating the supe-riority of our approach over existing techniques.These results highlight the potential of multimodal sentiment analysis and underscore the importance of considering the intrinsic semantic connections between modalities for accurate sentiment assessment.

A multimodal analysis of Piggyvest's animated advertisement videos

This study examines the persuasive strategies in selected animated advertisement videos by Piggyvest,the first and one of the fastest-growing online savings platforms in Nigeria,whose modus operandi is quite different from those of the conventional banking system.Drawing insights from Kress and van Leeuwen's(2006.Reading images:The grammar of visual design,2nd edn.New York:Routledge)Multimodal Discourse Analytic framework,12 samples of Piggyvest's animated videos(for advertisements)were randomly sampled out of the 30 videos initially purposively sampled.Findings reveal metaphorisation of financial management and discipline,playing around gender-related ideas on finances,foregrounding of aspirations and financial decisions,depiction of cognitive strength and weakness on money matters,and striking a connection between money and relationships are persuasive strategies deployed in the animated advertisement videos by Piggyvest to appeal to their target audience.

On Multimodal Criticism of The Kitchen God’s Wife

Studying the covers of the three editions of The Kitchen God’s Wife by Amy Tan,a famous female Chinese American writer,the author finds Chinese cultural elements stood out in publishers’earliest recommendations,but later were replaced by immigrant mother-daughter relationship,and then were overwhelmed by the concern for human beings.It is interesting that this change fits the changing focuses of Chinese American writers in the 21st century.

Tobacco Control Intervention:A Comparative Multimodal Discourse Analysis of Video Advertisements in China and Australia

Significant differences in tobacco-control initiatives and achievements between Australia and China have been witnessed in the world,despite the fact that thecountries share the Asia-pacific region and have intimate relationships.The smoking rate in Australia has been on a downward trend since the 1980s and therefore Australia has been portrayed as a“dark market”by global tobacco industry companies.However in China,cigarette smoking is the chief killer that causes deadly health risks to both direct users of tobacco and people exposed to second-hand smoke.Non-communicable diseases due to this have caused great burden to the national medical resources.As a vital intervention tool of mass media campaign,public health advertisements on tobacco-control have played an important role in influencing the cognition and behavior of recipients.This study aims to compare the strategies adopted by the advertisements in Australia and China through corpus-driven multimodal discourse analysis to conduct quantitative analysis and qualitative case study.The results of comparative analysis indicate different pathways of tobacco-control advertisements constructions in the two countries that Australian tobacco-control advertisements achieved better in terms of health communication intervention to encourage cessation among smokers and reduce smoking initiation among potential smokers at the population level.To be compared,Chinese tobacco-control advertisements placed a heavier emphasis on the health hazards caused by second-hand smoking to protect people from the attributable morbidity and mortality.From the perspective of content theme,this study argues that Chinese tobacco-control advertisements should shift the focus to the health damage of smoking to smokers which is more targeted and fundamental.From the perspective of multimodal presentations of the video advertisements,a more prominent demonstration of the health damage is recommended.Moreover,this study argues that cultural-oriented strategies should be developed in the tobacco-control advertisements if considering the cigarette sharing and gifting social phenomenon in the Chinese society.

The Media Images of Old Influencers on TikTok:A Multimodal Critical Discourse Analysis

Drawing on the insights from multimodal critical discourse analysis,this study mainly focuses on two super-gerontic celebrities by selecting 60 videos to elucidate how their images are represented through linguistic(e.g.,captions,and hashtags)and other semiotic resources(e.g.,accessories,semiotic tools,and etc.).The findings suggest that images of the old celebrities are characterized as being attractive,wise,stylish,healthy and young,indicating that they are breaking up the stereotypical images of old people who are unattractive,dependent and senile.The phenomenon of old influencers making themselves visible on the social media suggests that they are moving towards successful aging with the backdrop of aging society in China.This present study suggests that multimodal construction of old celebrities can inform relevant practice to combat ageism in the media and helps to empower the old people in the media discourse.In doing so,the pedagogical implication of present study lies in its education about how we view aging in the aging society.

Multimodal Discourse Analysis-A Corpus-based Study

According to multimodal discourse analysis(MDA),different kinds of semiotic modes work together to conduct meaning.Previous studies on MDA only involve case analysis.Nowadays,scholars tend to establish multimodal corpora to provide a more fuller and comprehensive study on multimodal discourses.In this paper,the author will look at some grand thesis about multimodal corpora and summarize the development of multimodal corpora.

A Multimodal Discourse Analysis of the Cosmetic Advertisement: Takingthe Poster of STEEROREAR as an Example

Because of the high speed of the development of technology,the traditional way to transmit information through language is gradually replaced by the newly appeared multimodal way of analyzing.This paper takes one cosmetic advertisement of STEERO-REAR as an example,aiming to analyze those adopted modalities in the poster.After analyzing,it is found that the elements,like color and position of the participants,contribute to the meaning construction in an organic way.The meaning and purpose of the producer are realized through the construction of representational meaning,interactive meaning and compositional meaning.

Meaning Construction of 2019 Beijing International Horticultural Exposition Emblem from the Perspective of Multimodal Discourse Analysis

The multimodal discourse has been playing a significant role in modern society since its springing-up from the 1990s.Based on the theoretical framework of Visual Grammar(VG)put forward by Kress&van Leeuwen(1996),this study,from the per⁃spective of multimodal discourse analysis,aims at exploring the meaning construction in the emblem of 2019 Beijing International Horticultural Exposition.It is found that an emblem can be interpreted with other modes such as visual image,color and pattern de⁃sign apart from verbal language,which will not only work to participate in the overall meaning construction with mutual interaction,but also analyze how sociocultural background,local customs and humanity spirit are reflected through these forms.

The Subtitle Translation of Movie from the Perspective of Multimodal Discourse Analysis

With the development of science and technology,especially the development of digital technology,mankind has entered the age of multimedia,and the mode of human life and communication have undergone profound changes.As a single communicative mode,language has been gradually replaced by complex communicative mode composed of language,image and sound.Multimodal discourse analysis provides a new perspective for discourse analysis composed of a variety of symbols,which can help readers understand how symbols such as images and music work together and form meanings.Firm analysis is often analyzed from the perspective of psychology,aesthetics and other macro aspects,but seldom from the perspective of linguistics.The paper analyzes how the theory of multimodal discourse analysis affects the translation of film by discussing the interaction between film translation and multimodal modes in the film Pride and Prejudice.

A Multimodal Discourse Analysis of Meaning Construction of Real Estate Posters

Posters have been one of the main research subjects in the Systemic-Functional Linguistics. Nowadays, posters are full of various types of semiotic modes with their wide applications of multimodality. This phenomenon is reflected in real estate posters as well, therefore, this thesis hopes to explore their essence of using multimodality and discussing meaning construction of the real estate posters.

Modal Interactive Feature Encoder for Multimodal Sentiment Analysis

Multimodal Sentiment analysis refers to analyzing emotions in infor-mation carriers containing multiple modalities.To better analyze the features within and between modalities and solve the problem of incomplete multimodal feature fusion,this paper proposes a multimodal sentiment analysis model MIF(Modal Interactive Feature Encoder For Multimodal Sentiment Analysis).First,the global features of three modalities are obtained through unimodal feature extraction networks.Second,the inter-modal interactive feature encoder and the intra-modal interactive feature encoder extract similarity features between modal-ities and intra-modal special features separately.Finally,unimodal special features and the interaction information between modalities are decoded to get the fusion features and predict sentimental polarity results.We conduct extensive experi-ments on three public multimodal datasets,including one in Chinese and two in English.The results show that the performance of our approach is significantly improved compared with benchmark models.

A Multimodal Critical Study of Selected Political Rally Campaign Discourse of 2011 Elections in Southwestern Nigeria

Multimodal communication is at the heart of political campaigns.The 2011 electioneering campaigns in Nigeria witnessed an interesting use of multimodal resources for the expression of meaning.While Ademilokun and Olateju(2016)examined and interpreted some of such multimodal elements from a descriptive perspective,the present study extends engagement with the subject as it seeks to tease out critical meanings embedded in the multimodal ensembles.Data for the study were purposively obtained from the collection of multimodal resources deployed in the Southwestern part of Nigeria during the 2011 electioneering campaigns in Nigeria.The study is hinged on the theoretical provisions of Machin and Mayr(2012).The study demonstrates that the multimodal resources in the 2011 electioneering campaigns in Southwestern Nigeria are purveyors of gender,carnivalesque and communal ideologies which are used largely for negotiating and perpetuating political and class hegemony.The study further shows that cultural and political identities are strategically used for political marketing and legitimation through the political campaign visuals.Power is also enacted in the renegotiation of gender ideology and the hegemonic configuration of the rallies.The study concludes that visuality in political rally communication in Nigeria is a useful index for ideologies and identities drawn upon for political survival by politicians and political parties in the nation.

Cross-Modal Complementary Network with Hierarchical Fusion for Multimodal Sentiment Classification

Multimodal Sentiment Classification(MSC)uses multimodal data,such as images and texts,to identify the users'sentiment polarities from the information posted by users on the Internet.MSC has attracted considerable attention because of its wide applications in social computing and opinion mining.However,improper correlation strategies can cause erroneous fusion as the texts and the images that are unrelated to each other may integrate.Moreover,simply concatenating them modal by modal,even with true correlation,cannot fully capture the features within and between modals.To solve these problems,this paper proposes a Cross-Modal Complementary Network(CMCN)with hierarchical fusion for MSC.The CMCN is designed as a hierarchical structure with three key modules,namely,the feature extraction module to extract features from texts and images,the feature attention module to learn both text and image attention features generated by an image-text correlation generator,and the cross-modal hierarchical fusion module to fuse features within and between modals.Such a CMCN provides a hierarchical fusion framework that can fully integrate different modal features and helps reduce the risk of integrating unrelated modal features.Extensive experimental results on three public datasets show that the proposed approach significantly outperforms the state-of-the-art methods.

Oil as narcotic or as medicine:the DISEASE metaphor in political cartoons on energy crisis

The present paper explores how the issue of the current energy crisis in the wake of the Covid-19 and Ukraine war was constructed by political cartoons.Adopting Critical Multimodal Metaphor Scenario Analysis,this paper focuses on the DISEASE metaphor scenario,one of the most recurring scenarios in political cartoons on this topic,and specifically emphasizes how the METHOD OF TREATMENT,one of the structural elements in the scenario,is represented.The analysis reveals that two predominant scenarios cOnstitute the representations of the METHOD OF TREATMENT:NARCOTIC Scenario and MEDICINE scenario.They differ in entailments:one frames fossil energy as detrimental narcotic while the other frames it as therapeutic medicine.By means of the two scenarios,these cartoons convey strong criticism of the major involvers in the energy crisis,namely,the E.U.,Russia,and the U.S.The cartoons make full use of the dynamic interplay of visual and/or verbal metonymy,metaphor,and narrative to elicit associations,assumptions and evaluations in the viewers,helping facilitate understanding and constructing a view of the crisis reality.The present analysis sheds light on the way cartoonists reshape the public point of view in the framing of specific event(s).