DeBERTa-GRU: Sentiment Analysis for Large Language Model

Modern technological advancements have made social media an essential component of daily life.Social media allow individuals to share thoughts,emotions,and ideas.Sentiment analysis plays the function of evaluating whether the sentiment of the text is positive,negative,neutral,or any other personal emotion to understand the sentiment context of the text.Sentiment analysis is essential in business and society because it impacts strategic decision-making.Sentiment analysis involves challenges due to lexical variation,an unlabeled dataset,and text distance correlations.The execution time increases due to the sequential processing of the sequence models.However,the calculation times for the Transformer models are reduced because of the parallel processing.This study uses a hybrid deep learning strategy to combine the strengths of the Transformer and Sequence models while ignoring their limitations.In particular,the proposed model integrates the Decoding-enhanced with Bidirectional Encoder Representations from Transformers(BERT)attention(DeBERTa)and the Gated Recurrent Unit(GRU)for sentiment analysis.Using the Decoding-enhanced BERT technique,the words are mapped into a compact,semantic word embedding space,and the Gated Recurrent Unit model can capture the distance contextual semantics correctly.The proposed hybrid model achieves F1-scores of 97%on the Twitter Large Language Model(LLM)dataset,which is much higher than the performance of new techniques.

Sentiment Analysis of Low-Resource Language Literature Using Data Processing and Deep Learning

Sentiment analysis, a crucial task in discerning emotional tones within the text, plays a pivotal role in understandingpublic opinion and user sentiment across diverse languages.While numerous scholars conduct sentiment analysisin widely spoken languages such as English, Chinese, Arabic, Roman Arabic, and more, we come to grapplingwith resource-poor languages like Urdu literature which becomes a challenge. Urdu is a uniquely crafted language,characterized by a script that amalgamates elements from diverse languages, including Arabic, Parsi, Pashtu,Turkish, Punjabi, Saraiki, and more. As Urdu literature, characterized by distinct character sets and linguisticfeatures, presents an additional hurdle due to the lack of accessible datasets, rendering sentiment analysis aformidable undertaking. The limited availability of resources has fueled increased interest among researchers,prompting a deeper exploration into Urdu sentiment analysis. This research is dedicated to Urdu languagesentiment analysis, employing sophisticated deep learning models on an extensive dataset categorized into fivelabels: Positive, Negative, Neutral, Mixed, and Ambiguous. The primary objective is to discern sentiments andemotions within the Urdu language, despite the absence of well-curated datasets. To tackle this challenge, theinitial step involves the creation of a comprehensive Urdu dataset by aggregating data from various sources such asnewspapers, articles, and socialmedia comments. Subsequent to this data collection, a thorough process of cleaningand preprocessing is implemented to ensure the quality of the data. The study leverages two well-known deeplearningmodels, namely Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), for bothtraining and evaluating sentiment analysis performance. Additionally, the study explores hyperparameter tuning tooptimize the models’ efficacy. Evaluation metrics such as precision, recall, and the F1-score are employed to assessthe effectiveness of the models. The research findings reveal that RNN surpasses CNN in Urdu sentiment analysis,gaining a significantly higher accuracy rate of 91%. This result accentuates the exceptional performance of RNN,solidifying its status as a compelling option for conducting sentiment analysis tasks in the Urdu language.

RUSAS: Roman Urdu Sentiment Analysis System

Sentiment analysis, the meta field of Natural Language Processing (NLP), attempts to analyze and identify thesentiments in the opinionated text data. People share their judgments, reactions, and feedback on the internetusing various languages. Urdu is one of them, and it is frequently used worldwide. Urdu-speaking people prefer tocommunicate on social media in Roman Urdu (RU), an English scripting style with the Urdu language dialect.Researchers have developed versatile lexical resources for features-rich comprehensive languages, but limitedlinguistic resources are available to facilitate the sentiment classification of Roman Urdu. This effort encompassesextracting subjective expressions in Roman Urdu and determining the implied opinionated text polarity. Theprimary sources of the dataset are Daraz (an e-commerce platform), Google Maps, and the manual effort. Thecontributions of this study include a Bilingual Roman Urdu Language Detector (BRULD) and a Roman UrduSpelling Checker (RUSC). These integrated modules accept the user input, detect the text language, correct thespellings, categorize the sentiments, and return the input sentence’s orientation with a sentiment intensity score.The developed system gains strength with each input experience gradually. The results show that the languagedetector gives an accuracy of 97.1% on a close domain dataset, with an overall sentiment classification accuracy of94.3%.

Aspect-Level Sentiment Analysis Based on Deep Learning

In recent years,deep learning methods have developed rapidly and found application in many fields,including natural language processing.In the field of aspect-level sentiment analysis,deep learning methods can also greatly improve the performance of models.However,previous studies did not take into account the relationship between user feature extraction and contextual terms.To address this issue,we use data feature extraction and deep learning combined to develop an aspect-level sentiment analysis method.To be specific,we design user comment feature extraction(UCFE)to distill salient features from users’historical comments and transform them into representative user feature vectors.Then,the aspect-sentence graph convolutional neural network(ASGCN)is used to incorporate innovative techniques for calculating adjacency matrices;meanwhile,ASGCN emphasizes capturing nuanced semantics within relationships among aspect words and syntactic dependency types.Afterward,three embedding methods are devised to embed the user feature vector into the ASGCN model.The empirical validations verify the effectiveness of these models,consistently surpassing conventional benchmarks and reaffirming the indispensable role of deep learning in advancing sentiment analysis methodologies.

A Robust Framework for Multimodal Sentiment Analysis with Noisy Labels Generated from Distributed Data Annotation

Multimodal sentiment analysis utilizes multimodal data such as text,facial expressions and voice to detect people’s attitudes.With the advent of distributed data collection and annotation,we can easily obtain and share such multimodal data.However,due to professional discrepancies among annotators and lax quality control,noisy labels might be introduced.Recent research suggests that deep neural networks(DNNs)will overfit noisy labels,leading to the poor performance of the DNNs.To address this challenging problem,we present a Multimodal Robust Meta Learning framework(MRML)for multimodal sentiment analysis to resist noisy labels and correlate distinct modalities simultaneously.Specifically,we propose a two-layer fusion net to deeply fuse different modalities and improve the quality of the multimodal data features for label correction and network training.Besides,a multiple meta-learner(label corrector)strategy is proposed to enhance the label correction approach and prevent models from overfitting to noisy labels.We conducted experiments on three popular multimodal datasets to verify the superiority of ourmethod by comparing it with four baselines.

Analysis of Public Sentiment regarding COVID-19 Vaccines on the Social Media Platform Reddit

This study undertakes a thorough analysis of the sentiment within the r/Corona-virus subreddit community regarding COVID-19 vaccines on Reddit. We meticulously collected and processed 34,768 comments, spanning from November 20, 2020, to January 17, 2021, using sentiment calculation methods such as TextBlob and Twitter-RoBERTa-Base-sentiment to categorize comments into positive, negative, or neutral sentiments. The methodology involved the use of Count Vectorizer as a vectorization technique and the implementation of advanced ensemble algorithms like XGBoost and Random Forest, achieving an accuracy of approximately 80%. Furthermore, through the Dirichlet latent allocation, we identified 23 distinct reasons for vaccine distrust among negative comments. These findings are crucial for understanding the community’s attitudes towards vaccination and can guide targeted public health messaging. Our study not only provides insights into public opinion during a critical health crisis, but also demonstrates the effectiveness of combining natural language processing tools and ensemble algorithms in sentiment analysis.

Aspect-Level Sentiment Analysis Incorporating Semantic and Syntactic Information

Aiming at the problem that existing models in aspect-level sentiment analysis cannot fully and effectively utilize sentence semantic and syntactic structure information, this paper proposes a graph neural network-based aspect-level sentiment classification model. Self-attention, aspectual word multi-head attention and dependent syntactic relations are fused and the node representations are enhanced with graph convolutional networks to enable the model to fully learn the global semantic and syntactic structural information of sentences. Experimental results show that the model performs well on three public benchmark datasets Rest14, Lap14, and Twitter, improving the accuracy of sentiment classification.

Multimodal Sentiment Analysis Based on a Cross-Modal Multihead Attention Mechanism

Multimodal sentiment analysis aims to understand people’s emotions and opinions from diverse data.Concate-nating or multiplying various modalities is a traditional multi-modal sentiment analysis fusion method.This fusion method does not utilize the correlation information between modalities.To solve this problem,this paper proposes amodel based on amulti-head attention mechanism.First,after preprocessing the original data.Then,the feature representation is converted into a sequence of word vectors and positional encoding is introduced to better understand the semantic and sequential information in the input sequence.Next,the input coding sequence is fed into the transformer model for further processing and learning.At the transformer layer,a cross-modal attention consisting of a pair of multi-head attention modules is employed to reflect the correlation between modalities.Finally,the processed results are input into the feedforward neural network to obtain the emotional output through the classification layer.Through the above processing flow,the model can capture semantic information and contextual relationships and achieve good results in various natural language processing tasks.Our model was tested on the CMU Multimodal Opinion Sentiment and Emotion Intensity(CMU-MOSEI)and Multimodal EmotionLines Dataset(MELD),achieving an accuracy of 82.04% and F1 parameters reached 80.59% on the former dataset.

Lexicon and Deep Learning-Based Approaches in Sentiment Analysis on Short Texts

Social media is an essential component of our personal and professional lives. We use it extensively to share various things, including our opinions on daily topics and feelings about different subjects. This sharing of posts provides insights into someone’s current emotions. In artificial intelligence (AI) and deep learning (DL), researchers emphasize opinion mining and analysis of sentiment, particularly on social media platforms such as Twitter (currently known as X), which has a global user base. This research work revolves explicitly around a comparison between two popular approaches: Lexicon-based and Deep learning-based Approaches. To conduct this study, this study has used a Twitter dataset called sentiment140, which contains over 1.5 million data points. The primary focus was the Long Short-Term Memory (LSTM) deep learning sequence model. In the beginning, we used particular techniques to preprocess the data. The dataset is divided into training and test data. We evaluated the performance of our model using the test data. Simultaneously, we have applied the lexicon-based approach to the same test data and recorded the outputs. Finally, we compared the two approaches by creating confusion matrices based on their respective outputs. This allows us to assess their precision, recall, and F1-Score, enabling us to determine which approach yields better accuracy. This research achieved 98% model accuracy for deep learning algorithms and 95% model accuracy for the lexicon-based approach.

Improving Sentiment Analysis in Election-Based Conversations on Twitter with ElecBERT Language Model

Sentiment analysis plays a vital role in understanding public opinions and sentiments toward various topics.In recent years,the rise of social media platforms(SMPs)has provided a rich source of data for analyzing public opinions,particularly in the context of election-related conversations.Nevertheless,sentiment analysis of electionrelated tweets presents unique challenges due to the complex language used,including figurative expressions,sarcasm,and the spread of misinformation.To address these challenges,this paper proposes Election-focused Bidirectional Encoder Representations from Transformers(ElecBERT),a new model for sentiment analysis in the context of election-related tweets.Election-related tweets pose unique challenges for sentiment analysis due to their complex language,sarcasm,andmisinformation.ElecBERT is based on the Bidirectional Encoder Representations from Transformers(BERT)language model and is fine-tuned on two datasets:Election-Related Sentiment-Annotated Tweets(ElecSent)-Multi-Languages,containing 5.31 million labeled tweets in multiple languages,and ElecSent-English,containing 4.75million labeled tweets in English.Themodel outperforms othermachine learning models such as Support Vector Machines(SVM),Na飗e Bayes(NB),and eXtreme Gradient Boosting(XGBoost),with an accuracy of 0.9905 and F1-score of 0.9816 on ElecSent-Multi-Languages,and an accuracy of 0.9930 and F1-score of 0.9899 on ElecSent-English.The performance of differentmodels was compared using the 2020 United States(US)Presidential Election as a case study.The ElecBERT-English and ElecBERT-Multi-Languages models outperformed BERTweet,with the ElecBERT-English model achieving aMean Absolute Error(MAE)of 6.13.This paper presents a valuable contribution to sentiment analysis in the context of election-related tweets,with potential applications in political analysis,social media management,and policymaking.

Sine Cosine Optimization with Deep Learning-Based Applied Linguistics for Sentiment Analysis on COVID-19 Tweets

Applied linguistics is an interdisciplinary domain which identifies,investigates,and offers solutions to language-related real-life problems.The new coronavirus disease,otherwise known as Coronavirus disease(COVID-19),has severely affected the everyday life of people all over the world.Specifically,since there is insufficient access to vaccines and no straight or reliable treatment for coronavirus infection,the country has initiated the appropriate preventive measures(like lockdown,physical separation,and masking)for combating this extremely transmittable disease.So,individuals spent more time on online social media platforms(i.e.,Twitter,Facebook,Instagram,LinkedIn,and Reddit)and expressed their thoughts and feelings about coronavirus infection.Twitter has become one of the popular social media platforms and allows anyone to post tweets.This study proposes a sine cosine optimization with bidirectional gated recurrent unit-based senti-ment analysis(SCOBGRU-SA)on COVID-19 tweets.The SCOBGRU-SA technique aimed to detect and classify the various sentiments in Twitter data during the COVID-19 pandemic.The SCOBGRU-SA technique follows data pre-processing and the Fast-Text word embedding process to accomplish this.Moreover,the BGRU model is utilized to recognise and classify sen-timents present in the tweets.Furthermore,the SCO algorithm is exploited for tuning the BGRU method’s hyperparameter,which helps attain improved classification performance.The experimental validation of the SCOBGRU-SA technique takes place using a benchmark dataset,and the results signify its promising performance compared to other DL models.

Optimal Machine Learning Driven Sentiment Analysis on COVID-19 Twitter Data

The outbreak of the pandemic,caused by Coronavirus Disease 2019(COVID-19),has affected the daily activities of people across the globe.During COVID-19 outbreak and the successive lockdowns,Twitter was heavily used and the number of tweets regarding COVID-19 increased tremendously.Several studies used Sentiment Analysis(SA)to analyze the emotions expressed through tweets upon COVID-19.Therefore,in current study,a new Artificial Bee Colony(ABC)with Machine Learning-driven SA(ABCMLSA)model is developed for conducting Sentiment Analysis of COVID-19 Twitter data.The prime focus of the presented ABCML-SA model is to recognize the sentiments expressed in tweets made uponCOVID-19.It involves data pre-processing at the initial stage followed by n-gram based feature extraction to derive the feature vectors.For identification and classification of the sentiments,the Support Vector Machine(SVM)model is exploited.At last,the ABC algorithm is applied to fine tune the parameters involved in SVM.To demonstrate the improved performance of the proposed ABCML-SA model,a sequence of simulations was conducted.The comparative assessment results confirmed the effectual performance of the proposed ABCML-SA model over other approaches.

A Parallel Approach for Sentiment Analysis on Social Networks Using Spark

The public is increasingly using social media platforms such as Twitter and Facebook to express their views on a variety of topics.As a result,social media has emerged as the most effective and largest open source for obtaining public opinion.Single node computational methods are inefficient for sentiment analysis on such large datasets.Supercomputers or parallel or distributed proces-sing are two options for dealing with such large amounts of data.Most parallel programming frameworks,such as MPI(Message Processing Interface),are dif-ficult to use and scale in environments where supercomputers are expensive.Using the Apache Spark Parallel Model,this proposed work presents a scalable system for sentiment analysis on Twitter.A Spark-based Naive Bayes training technique is suggested for this purpose;unlike prior research,this algorithm does not need any disk access.Millions of tweets have been classified using the trained model.Experiments with various-sized clusters reveal that the suggested strategy is extremely scalable and cost-effective for larger data sets.It is nearly 12 times quicker than the Map Reduce-based model and nearly 21 times faster than the Naive Bayes Classifier in Apache Mahout.To evaluate the framework’s scalabil-ity,we gathered a large training corpus from Twitter.The accuracy of the classi-fier trained with this new dataset was more than 80%.

Sentiment Analysis with Tweets Behaviour in Twitter Streaming API

Twitter is a radiant platform with a quick and effective technique to analyze users’perceptions of activities on social media.Many researchers and industry experts show their attention to Twitter sentiment analysis to recognize the stakeholder group.The sentiment analysis needs an advanced level of approaches including adoption to encompass data sentiment analysis and various machine learning tools.An assessment of sentiment analysis in multiple fields that affect their elevations among the people in real-time by using Naive Bayes and Support Vector Machine(SVM).This paper focused on analysing the distinguished sentiment techniques in tweets behaviour datasets for various spheres such as healthcare,behaviour estimation,etc.In addition,the results in this work explore and validate the statistical machine learning classifiers that provide the accuracy percentages attained in terms of positive,negative and neutral tweets.In this work,we obligated Twitter Application Programming Interface(API)account and programmed in python for sentiment analysis approach for the computational measure of user’s perceptions that extract a massive number of tweets and provide market value to the Twitter account proprietor.To distinguish the results in terms of the performance evaluation,an error analysis investigates the features of various stakeholders comprising social media analytics researchers,Natural Language Processing(NLP)developers,engineering managers and experts involved to have a decision-making approach.

A Cloud Based Sentiment Analysis through Logistic Regression in AWS Platform

The use of Amazon Web Services is growing rapidly as more users are adopting the technology.It has various functionalities that can be used by large corporates and individuals as well.Sentiment analysis is used to build an intelligent system that can study the opinions of the people and help to classify those related emotions.In this research work,sentiment analysis is performed on the AWS Elastic Compute Cloud(EC2)through Twitter data.The data is managed to the EC2 by using elastic load balancing.The collected data is subjected to preprocessing approaches to clean the data,and then machine learning-based logistic regression is employed to categorize the sentiments into positive and negative sentiments.High accuracy of 94.17%is obtained through the proposed machine learning model which is higher than the other models that are developed using the existing algorithms.

Multimodal Sentiment Analysis Using BiGRU and Attention-Based Hybrid Fusion Strategy

Recently,multimodal sentiment analysis has increasingly attracted attention with the popularity of complementary data streams,which has great potential to surpass unimodal sentiment analysis.One challenge of multimodal sentiment analysis is how to design an efficient multimodal feature fusion strategy.Unfortunately,existing work always considers feature-level fusion or decision-level fusion,and few research works focus on hybrid fusion strategies that contain feature-level fusion and decision-level fusion.To improve the performance of multimodal sentiment analysis,we present a novel multimodal sentiment analysis model using BiGRU and attention-based hybrid fusion strategy(BAHFS).Firstly,we apply BiGRU to learn the unimodal features of text,audio and video.Then we fuse the unimodal features into bimodal features using the bimodal attention fusion module.Next,BAHFS feeds the unimodal features and bimodal features into the trimodal attention fusion module and the trimodal concatenation fusion module simultaneously to get two sets of trimodal features.Finally,BAHFS makes a classification with the two sets of trimodal features respectively and gets the final analysis results with decision-level fusion.Based on the CMU-MOSI and CMU-MOSEI datasets,extensive experiments have been carried out to verify BAHFS’s superiority.

Multimodal sentiment analysis for social media contents during public emergencies

Purpose:Nowadays,public opinions during public emergencies involve not only textual contents but also contain images.However,the existing works mainly focus on textual contents and they do not provide a satisfactory accuracy of sentiment analysis,lacking the combination of multimodal contents.In this paper,we propose to combine texts and images generated in the social media to perform sentiment analysis.Design/methodology/approach:We propose a Deep Multimodal Fusion Model(DMFM),which combines textual and visual sentiment analysis.We first train word2vec model on a large-scale public emergency corpus to obtain semantic-rich word vectors as the input of textual sentiment analysis.BiLSTM is employed to generate encoded textual embeddings.To fully excavate visual information from images,a modified pretrained VGG16-based sentiment analysis network is used with the best-performed fine-tuning strategy.A multimodal fusion method is implemented to fuse textual and visual embeddings completely,producing predicted labels.Findings:We performed extensive experiments on Weibo and Twitter public emergency datasets,to evaluate the performance of our proposed model.Experimental results demonstrate that the DMFM provides higher accuracy compared with baseline models.The introduction of images can boost the performance of sentiment analysis during public emergencies.Research limitations:In the future,we will test our model in a wider dataset.We will also consider a better way to learn the multimodal fusion information.Practical implications:We build an efficient multimodal sentiment analysis model for the social media contents during public emergencies.Originality/value:We consider the images posted by online users during public emergencies on social platforms.The proposed method can present a novel scope for sentiment analysis during public emergencies and provide the decision support for the government when formulating policies in public emergencies.

SA-Model:Multi-Feature Fusion Poetic Sentiment Analysis Based on a Hybrid Word Vector Model

Sentiment analysis in Chinese classical poetry has become a prominent topic in historical and cultural tracing,ancient literature research,etc.However,the existing research on sentiment analysis is relatively small.It does not effectively solve the problems such as the weak feature extraction ability of poetry text,which leads to the low performance of the model on sentiment analysis for Chinese classical poetry.In this research,we offer the SA-Model,a poetic sentiment analysis model.SA-Model firstly extracts text vector information and fuses it through Bidirectional encoder representation from transformers-Whole word masking-extension(BERT-wwmext)and Enhanced representation through knowledge integration(ERNIE)to enrich text vector information;Secondly,it incorporates numerous encoders to remove text features at multiple levels,thereby increasing text feature information,improving text semantics accuracy,and enhancing the model’s learning and generalization capabilities;finally,multi-feature fusion poetry sentiment analysis model is constructed.The feasibility and accuracy of the model are validated through the ancient poetry sentiment corpus.Compared with other baseline models,the experimental findings indicate that SA-Model may increase the accuracy of text semantics and hence improve the capability of poetry sentiment analysis.

Multi-Tier Sentiment Analysis of Social Media Text Using Supervised Machine Learning

Sentiment Analysis(SA)is often referred to as opinion mining.It is defined as the extraction,identification,or characterization of the sentiment from text.Generally,the sentiment of a textual document is classified into binary classes i.e.,positive and negative.However,fine-grained classification provides a better insight into the sentiments.The downside is that fine-grained classification is more challenging as compared to binary.On the contrary,performance deteriorates significantly in the case of multi-class classification.In this study,pre-processing techniques and machine learning models for the multi-class classification of sentiments were explored.To augment the performance,a multi-layer classification model has been proposed.Owing to similitude with social media text,the movie reviews dataset has been used for the implementation.Supervised machine learning models namely Decision Tree,Support Vector Machine,and Naive Bayes models have been implemented for the task of sentiment classification.We have compared the models of single-layer architecture with multi-tier model.The results of Multi-tier model have slight improvement over the single-layer architecture.Moreover,multi-tier models have better recall which allow our proposed model to learn more context.We have discussed certain shortcomings of the model that will help researchers to design multi-tier models with more contextual information.

LexDeep:Hybrid Lexicon and Deep Learning Sentiment Analysis Using Twitter for Unemployment-Related Discussions During COVID-19

The COVID-19 pandemic has spread globally,resulting in financialinstability in many countries and reductions in the per capita grossdomestic product.Sentiment analysis is a cost-effective method for acquiringsentiments based on household income loss,as expressed on social media.However,limited research has been conducted in this domain using theLexDeep approach.This study aimed to explore social trend analytics usingLexDeep,which is a hybrid sentiment analysis technique,on Twitter to capturethe risk of household income loss during the COVID-19 pandemic.First,tweet data were collected using Twint with relevant keywords before(9 March2019 to 17 March 2020)and during(18 March 2020 to 21 August 2021)thepandemic.Subsequently,the tweets were annotated using VADER(lexiconbased)and fed into deep learning classifiers,and experiments were conductedusing several embeddings,namely simple embedding,Global Vectors,andWord2Vec,to classify the sentiments expressed in the tweets.The performanceof each LexDeep model was evaluated and compared with that of a supportvector machine(SVM).Finally,the unemployment rates before and duringCOVID-19 were analysed to gain insights into the differences in unemploymentpercentages through social media input and analysis.The resultsdemonstrated that all LexDeep models with simple embedding outperformedthe SVM.This confirmed the superiority of the proposed LexDeep modelover a classical machine learning classifier in performing sentiment analysistasks for domain-specific sentiments.In terms of the risk of income loss,the unemployment issue is highly politicised on both the regional and globalscales;thus,if a country cannot combat this issue,the global economy will alsobe affected.Future research should develop a utility maximisation algorithmfor household welfare evaluation,given the percentage risk of income lossowing to COVID-19.