Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Since the 1950s,when the Turing Test was introduced,there has been notable progress in machine language intelligence.Language modeling,crucial for AI development,has evolved from statistical to neural models over the last two decades.Recently,transformer-based Pre-trained Language Models(PLM)have excelled in Natural Language Processing(NLP)tasks by leveraging large-scale training corpora.Increasing the scale of these models enhances performance significantly,introducing abilities like context learning that smaller models lack.The advancement in Large Language Models,exemplified by the development of ChatGPT,has made significant impacts both academically and industrially,capturing widespread societal interest.This survey provides an overview of the development and prospects from Large Language Models(LLM)to Large Multimodal Models(LMM).It first discusses the contributions and technological advancements of LLMs in the field of natural language processing,especially in text generation and language understanding.Then,it turns to the discussion of LMMs,which integrates various data modalities such as text,images,and sound,demonstrating advanced capabilities in understanding and generating cross-modal content,paving new pathways for the adaptability and flexibility of AI systems.Finally,the survey highlights the prospects of LMMs in terms of technological development and application potential,while also pointing out challenges in data integration,cross-modal understanding accuracy,providing a comprehensive perspective on the latest developments in this field.

Women health and microbiota:Different aspects of well-being

In this editorial,we comment on the article by Marano et al recently published in the World Journal of Gastroenterology 2023;29(45):5945-5952.We focus on the role of gut microbiota(GM)in women’s health,highlighting the need to thoroughly comprehend the sex differences in microbiota.Together,the host and GM support the host’s health.The microbiota components consist of viruses,bacteria,fungi,and archaea.This complex is an essential part of the host and is involved in neu-rological development,metabolic control,immune system dynamics,and host dynamic homeostasis.It has been shown that differences in the GM of males and females can contribute to chronic diseases,such as gastrointestinal,metabolic,neurological,cardiovascular,and respiratory illnesses.These differences can also result in some sex-specific changes in immunity.Every day,research on GM reveals new and more expansive frontiers,offering a wealth of innovative oppor-tunities for preventive and precision medicine.

Enhancing Communication Accessibility:UrSL-CNN Approach to Urdu Sign Language Translation for Hearing-Impaired Individuals

Deaf people or people facing hearing issues can communicate using sign language(SL),a visual language.Many works based on rich source language have been proposed;however,the work using poor resource language is still lacking.Unlike other SLs,the visuals of the Urdu Language are different.This study presents a novel approach to translating Urdu sign language(UrSL)using the UrSL-CNN model,a convolutional neural network(CNN)architecture specifically designed for this purpose.Unlike existingworks that primarily focus on languageswith rich resources,this study addresses the challenge of translating a sign language with limited resources.We conducted experiments using two datasets containing 1500 and 78,000 images,employing a methodology comprising four modules:data collection,pre-processing,categorization,and prediction.To enhance prediction accuracy,each sign image was transformed into a greyscale image and underwent noise filtering.Comparative analysis with machine learning baseline methods(support vectormachine,GaussianNaive Bayes,randomforest,and k-nearest neighbors’algorithm)on the UrSL alphabets dataset demonstrated the superiority of UrSL-CNN,achieving an accuracy of 0.95.Additionally,our model exhibited superior performance in Precision,Recall,and F1-score evaluations.This work not only contributes to advancing sign language translation but also holds promise for improving communication accessibility for individuals with hearing impairments.

Relationship Between Individuals’Epidemic Risk Perception Within Living Space and Subjective Well-Being:Empirical Evidence from China after the First Wave of COVID-19

It is common to observe the epidemic risk perception(ERP)and a decline in subjective well-being(SWB)in the context of public health events,such as Corona Virus Disease 2019(COVID-19).However,there have been few studies exploring the impact of individuals’ERP within living space on their SWB,especially from a geographical and daily activity perspective after the resumption of work and other activities following a wave of the pandemic.In this paper,we conducted a study with 789 participants in urban China,measuring their ERP within living space and examining its influence on their SWB using path analysis.The results indicated that individuals’ERP within their living space had a significant negative effect on their SWB.The density of certain types of facilities within their living space,such as bus stops,subway stations,restaurants,fast food shops,convenience shops,hospitals,and public toilets,had a significantly negative impact on their SWB,mediated by their ERP within living space.Additionally,participation in out-of-home work and other activities not only increased individuals’ERP within living space,but also strengthened its negative effect on their SWB.

Systematizing Teacher Development:A Review of Foreign Language Teacher Learning

Foreign language teaching practice is developing rapidly,but research on foreign language teacher learning is currently relatively fragmented and unstructured.The book Foreign Language Teacher Learning,written by Professor Kang Yan from Capital Normal University,published in September 2022,makes a systematic introduction to foreign language teacher learning,which to some extent makes up for this shortcoming.Her book presents the lineage of foreign language teacher learning research at home and abroad,analyzes both theoretical and practical aspects,reviews the cuttingedge research results,and foresees the future development trend,painting a complete research picture for researchers in the field of foreign language teaching and teacher education as well as front-line teachers interested in foreign language teacher learning.This is an important inspiration for conducting foreign language teacher learning research in the future.And this paper makes a review of the book from aspects such as its content,major characteristics,contributions and limitations.

Self-Compassion Moderates the Effect of Contingent Self-Esteem on Well-Being:Evidence from Cross-Sectional Survey and Experiment

Contingent self-esteem captures the fragile nature of self-esteem and is often regarded as suboptimal to psychological functioning.Self-compassion is another important self-related concept assumed to promote mental health and well-being.However,research on the relation of self-compassion to contingent self-esteem is lacking.Two studies were conducted to explore the role of selfcompassion,either as a personal characteristic or an induced mindset,in influencing the effects of contingent self-esteem on well-being.Study 1 recruited 256 Chinese college students(30.4%male,mean age=21.72 years)who filled out measures of contingent self-esteem,self-compassion,and well-being.The results found that self-compassion moderated the effect of contingent self-esteem on well-being.In Study 2,a sample of 90 Chinese college students(34%male,mean age=18.39 years)were randomly assigned to either a control or self-compassion group.They completed baseline trait measures of contingent self-esteem,self-compassion,and self-esteem.Then,they were led to have a 12-min break(control group)or listen to a 12-min self-compassion audio(self-compassion group),followed by a social stress task and outcome measures.The results demonstrated the effectiveness of the brief self-compassion training and its moderating role in influencing the effects of contingent self-esteem on negative affects after the social stress task.This research provides implications that to equip with a self-compassionate mindset could lower the risk of the impairment of well-being associated with elements of contingent selfesteem,which involves a fragile sense of self-worth.It may also provide insights into the development of an“optimal selfesteem”and the improvement of well-being.

The Effects of Childhood Trauma on College Students’Depressive Symptoms:The Mediation Role of Subjective Well-Being and the Moderation Role of Resilience

Background:The enduring and detrimental impact of childhood trauma on later health and well-being is now well established.However,research on the relationship between childhood trauma and depressive symptoms,along with the potential risk and protective factors,is insufficient in the context of Chinese college student population.Methods:Data on childhood trauma,depressive symptoms,resilience,and subjective well-being were collected through surveys conducted with 367 Chinese university students.The data collected in this study were analyzed using SPSS 26.0 and PROCESS 3.5.Results:The results revealed that subjective well-being mediated the relationship between childhood trauma and depressive symptoms among college students,with direct and indirect effects accounting for 59.46%and 40.54%of the total effect,respectively.The pathway process between subjective well-being and depressive symptoms was moderated by resilience,whereby an increase in resilience levels corresponded to a gradual escalation in the predictive power of subjective well-being on depressive symptoms.Conclusion:The study indicates that childhood trauma significantly and positively predicts depressive symptoms among college students,and it can also directly predict depressive symptoms through the mediating effect of subjective well-being.Elevating levels of psychological resilience and subjective well-being among college students can mitigate depression and promote psychological well-being.From the perspective of positive psychology,the present study provides a new perspective for the prevention and intervention of depressive symptoms among college students.

Disparities in the impact of economic well-being on self-esteem in adulthood:Race and ethnicity

BACKGROUND Most studies have defined economic well-being as socioeconomic status,with little attention given to whether other indicators influence self-esteem.Little is known about racial/ethnic disparities in the relationship between economic wellbeing and self-esteem during adulthood.AIM To explore the impact of economic well-being on self-esteem in adulthood and differences in the association across race/ethnicity.METHODS The current study used data from the National Longitudinal Survey of Youth 1979.The final sample consisted of 2267 African Americans,1425 Hispanics,and 3678 non-Hispanic Whites.Ordinary linear regression analyses and logistic regression analyses were conducted.RESULTS African Americans and Hispanics were more likely to be in poverty in comparison with non-Hispanic Whites.More African Americans were unemployed than Whites.Those who received fringe benefits,were more satisfied with jobs,and were employed were more likely to have higher levels of self-esteem.Poverty was negatively associated with self-esteem.Interaction effects were found between African Americans and job satisfaction predicting self-esteem.CONCLUSION The role of employers is important in cultivating employees’self-esteem.Satisfactory outcomes or feelings of happiness from the workplace may be more important to non-Hispanic Whites compared to African Americans and Hispanics.

Exploring well-being disparities between urban and rural areas:A case study in the Stavropol Territory,Russia

Rural areas are crucial for a country’s sustainable economy.New strategies are needed to develop rural areas to improve the well-being of rural population and generate new job opportunities.This is especially important in countries where agricultural production accounts for a significant share of the gross product,such as Russia.In this study,we identified the key indicators of satisfaction and differences between rural and urban citizens based on their social,economic,and environmental backgrounds,and determined whether there are well-being disparities between rural and urban areas in the Stavropol Territory,Russia.We collected primary data through a survey based on the European Social Survey framework to investigate the potential differences between rural and urban areas.By computing the regional well-being index using principal component analysis,we found that there was no statistically significant difference in well-being between rural and urban areas.Results of key indicators showed that rural residents felt psychologically more comfortable and safer,assessed their family relationships better,and adhered more to traditions and customs.However,urban residents showed better economic and social conditions(e.g.,infrastructures,medical care,education,and Internet access).The results of this study imply that we can better understand the local needs,advantages,and unique qualities,thereby gaining insight into the effectiveness of government programs.Policy-makers and local authorities can consider targeted interventions based on the findings of this study and strive to enhance the well-being of both urban and rural residents.

Literature classification and its applications in condensed matter physics and materials science by natural language processing

The exponential growth of literature is constraining researchers’access to comprehensive information in related fields.While natural language processing(NLP)may offer an effective solution to literature classification,it remains hindered by the lack of labelled dataset.In this article,we introduce a novel method for generating literature classification models through semi-supervised learning,which can generate labelled dataset iteratively with limited human input.We apply this method to train NLP models for classifying literatures related to several research directions,i.e.,battery,superconductor,topological material,and artificial intelligence(AI)in materials science.The trained NLP‘battery’model applied on a larger dataset different from the training and testing dataset can achieve F1 score of 0.738,which indicates the accuracy and reliability of this scheme.Furthermore,our approach demonstrates that even with insufficient data,the not-well-trained model in the first few cycles can identify the relationships among different research fields and facilitate the discovery and understanding of interdisciplinary directions.

Researching the Impact of the Quality of Village Governance on Farmers’Well-Being:From the Perspectives of Income and Occupational Differentiation

The goal of village governance is to improve the well-being of farmers,so this study aims to measure the impact the quality of village governance on the well-being of farmers.It also examines the heterogeneity of this impact across different farmer groups from the perspectives of income levels and occupational differentiation.To this end,this study developed an indicator system based on survey data collected from 1,442 farmers in the Sichuan,Shaanxi,and Gansu provinces,as well as the Ningxia Hui autonomous region.Multiple linear regression models were then used to analyze this data,and the findings revealed that improvements in the quality of village governance significantly increased the well-being of farmers.Specifically,primary-level empowerment and capacity building were shown to contribute the most to the enhancement of the farmers’well-being,followed by social inclusion,and social cohesion was found to have only a minimal effect.In terms of income levels,improving the quality of village governance benefited middle-income farmers the most,followed by low-income farmers,and it had the least effect on high-income farmers.In terms of occupations,full-time farmers gained the most from improvements in the quality of village governance,followed by off-farm farmers,with part-time farmers benefiting the least.Based on these findings,this study suggests that policymakers should improve the quality of village governance to enhance the well-being of farmers,focusing on the impact that level of income and occupational differentiation have on village governance.

Automatic Generation of Attribute-Based Access Control Policies from Natural Language Documents

In response to the challenges of generating Attribute-Based Access Control(ABAC)policies,this paper proposes a deep learning-based method to automatically generate ABAC policies from natural language documents.This method is aimed at organizations such as companies and schools that are transitioning from traditional access control models to the ABAC model.The manual retrieval and analysis involved in this transition are inefficient,prone to errors,and costly.Most organizations have high-level specifications defined for security policies that include a set of access control policies,which often exist in the form of natural language documents.Utilizing this rich source of information,our method effectively identifies and extracts the necessary attributes and rules for access control from natural language documents,thereby constructing and optimizing access control policies.This work transforms the problem of policy automation generation into two tasks:extraction of access control statements andmining of access control attributes.First,the Chat General Language Model(ChatGLM)isemployed to extract access control-related statements from a wide range of natural language documents by constructing unique prompts and leveraging the model’s In-Context Learning to contextualize the statements.Then,the Iterated Dilated-Convolutions-Conditional Random Field(ID-CNN-CRF)model is used to annotate access control attributes within these extracted statements,including subject attributes,object attributes,and action attributes,thus reassembling new access control policies.Experimental results show that our method,compared to baseline methods,achieved the highest F1 score of 0.961,confirming the model’s effectiveness and accuracy.

Visualizing Language and Aging From 2013-2022

Declining cognitive abilities can be a concomitant of advanced age.As language is closely associated with cognitive abilities,changes in language abilities can be an important marker of changes in cognitive abilities.The current study is to review cognitive studies of language and aging by first identifying and exploring the major clusters and pivotal articles and then detecting emerging trends.Data of 3,266 articles on language and aging from 2013 to 2022 were collected from the Web of Science Core Collection database.Adopting Document Co-citation Analysis,Freeman’s betweenness centrality metric(Freeman,2002)and Kleinberg’s burst detection algorithm(Kleinberg,2002),we explored major clusters,pivotal articles and emerging trends in this field.Cognition appears to be the most remarkable cluster.Bilingualism,speech production,listening effort,and reading comprehension are other major active clusters in a certain period.The most recent active cluster concerns the studies of Alzheimer’s disease.Articles serving as pivotal points concentrate on cognitive studies of the Framework for Understanding Effortful Listening(FUEL),the new Ease of Language Understanding model(EUL)and a hierarchical multi-representational generative framework of language comprehension.The progress in statistical methods,the relationship between language and cognitive impairment and the relationship between language abilities and cognition are the emerging trends.These emerging trends will provide some insights into how cognitive abilities influence language abilities in aging.

Comparing Fine-Tuning, Zero and Few-Shot Strategies with Large Language Models in Hate Speech Detection in English

Large Language Models(LLMs)are increasingly demonstrating their ability to understand natural language and solve complex tasks,especially through text generation.One of the relevant capabilities is contextual learning,which involves the ability to receive instructions in natural language or task demonstrations to generate expected outputs for test instances without the need for additional training or gradient updates.In recent years,the popularity of social networking has provided a medium through which some users can engage in offensive and harmful online behavior.In this study,we investigate the ability of different LLMs,ranging from zero-shot and few-shot learning to fine-tuning.Our experiments show that LLMs can identify sexist and hateful online texts using zero-shot and few-shot approaches through information retrieval.Furthermore,it is found that the encoder-decoder model called Zephyr achieves the best results with the fine-tuning approach,scoring 86.811%on the Explainable Detection of Online Sexism(EDOS)test-set and 57.453%on the Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter(HatEval)test-set.Finally,it is confirmed that the evaluated models perform well in hate text detection,as they beat the best result in the HatEval task leaderboard.The error analysis shows that contextual learning had difficulty distinguishing between types of hate speech and figurative language.However,the fine-tuned approach tends to produce many false positives.

DeBERTa-GRU: Sentiment Analysis for Large Language Model

Modern technological advancements have made social media an essential component of daily life.Social media allow individuals to share thoughts,emotions,and ideas.Sentiment analysis plays the function of evaluating whether the sentiment of the text is positive,negative,neutral,or any other personal emotion to understand the sentiment context of the text.Sentiment analysis is essential in business and society because it impacts strategic decision-making.Sentiment analysis involves challenges due to lexical variation,an unlabeled dataset,and text distance correlations.The execution time increases due to the sequential processing of the sequence models.However,the calculation times for the Transformer models are reduced because of the parallel processing.This study uses a hybrid deep learning strategy to combine the strengths of the Transformer and Sequence models while ignoring their limitations.In particular,the proposed model integrates the Decoding-enhanced with Bidirectional Encoder Representations from Transformers(BERT)attention(DeBERTa)and the Gated Recurrent Unit(GRU)for sentiment analysis.Using the Decoding-enhanced BERT technique,the words are mapped into a compact,semantic word embedding space,and the Gated Recurrent Unit model can capture the distance contextual semantics correctly.The proposed hybrid model achieves F1-scores of 97%on the Twitter Large Language Model(LLM)dataset,which is much higher than the performance of new techniques.

Enhancing Orthopedic Knowledge Assessments:The Performance of Specialized Generative Language Model Optimization

Objective This study aimed to evaluate and compare the effectiveness of knowledge base-optimized and unoptimized large language models(LLMs)in the field of orthopedics to explore optimization strategies for the application of LLMs in specific fields.Methods This research constructed a specialized knowledge base using clinical guidelines from the American Academy of Orthopaedic Surgeons(AAOS)and authoritative orthopedic publications.A total of 30 orthopedic-related questions covering aspects such as anatomical knowledge,disease diagnosis,fracture classification,treatment options,and surgical techniques were input into both the knowledge base-optimized and unoptimized versions of the GPT-4,ChatGLM,and Spark LLM,with their generated responses recorded.The overall quality,accuracy,and comprehensiveness of these responses were evaluated by 3 experienced orthopedic surgeons.Results Compared with their unoptimized LLMs,the optimized version of GPT-4 showed improvements of 15.3%in overall quality,12.5%in accuracy,and 12.8%in comprehensiveness;ChatGLM showed improvements of 24.8%,16.1%,and 19.6%,respectively;and Spark LLM showed improvements of 6.5%,14.5%,and 24.7%,respectively.Conclusion The optimization of knowledge bases significantly enhances the quality,accuracy,and comprehensiveness of the responses provided by the 3 models in the orthopedic field.Therefore,knowledge base optimization is an effective method for improving the performance of LLMs in specific fields.

Recent Advances on Deep Learning for Sign Language Recognition

Sign language,a visual-gestural language used by the deaf and hard-of-hearing community,plays a crucial role in facilitating communication and promoting inclusivity.Sign language recognition(SLR),the process of automatically recognizing and interpreting sign language gestures,has gained significant attention in recent years due to its potential to bridge the communication gap between the hearing impaired and the hearing world.The emergence and continuous development of deep learning techniques have provided inspiration and momentum for advancing SLR.This paper presents a comprehensive and up-to-date analysis of the advancements,challenges,and opportunities in deep learning-based sign language recognition,focusing on the past five years of research.We explore various aspects of SLR,including sign data acquisition technologies,sign language datasets,evaluation methods,and different types of neural networks.Convolutional Neural Networks(CNN)and Recurrent Neural Networks(RNN)have shown promising results in fingerspelling and isolated sign recognition.However,the continuous nature of sign language poses challenges,leading to the exploration of advanced neural network models such as the Transformer model for continuous sign language recognition(CSLR).Despite significant advancements,several challenges remain in the field of SLR.These challenges include expanding sign language datasets,achieving user independence in recognition systems,exploring different input modalities,effectively fusing features,modeling co-articulation,and improving semantic and syntactic understanding.Additionally,developing lightweight network architectures for mobile applications is crucial for practical implementation.By addressing these challenges,we can further advance the field of deep learning for sign language recognition and improve communication for the hearing-impaired community.

LKPNR: Large Language Models and Knowledge Graph for Personalized News Recommendation Framework

Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.

A large language model-powered literature review for high-angle annular dark field imaging

High-angle annular dark field(HAADF)imaging in scanning transmission electron microscopy(STEM)has become an indispensable tool in materials science due to its ability to offer sub-°A resolution and provide chemical information through Z-contrast.This study leverages large language models(LLMs)to conduct a comprehensive bibliometric analysis of a large amount of HAADF-related literature(more than 41000 papers).By using LLMs,specifically ChatGPT,we were able to extract detailed information on applications,sample preparation methods,instruments used,and study conclusions.The findings highlight the capability of LLMs to provide a new perspective into HAADF imaging,underscoring its increasingly important role in materials science.Moreover,the rich information extracted from these publications can be harnessed to develop AI models that enhance the automation and intelligence of electron microscopes.

Large language models in laparoscopic surgery: A transformative opportunity

This opinion paper explores the transformative potential of large language models(LLMs)in laparoscopic surgery and argues for their integration to enhance surgical education,decision support,reporting,and patient care.LLMs can revolutionize surgical education by providing personalized learning experiences and accelerating skill acquisition.Intelligent decision support systems powered by LLMs can assist surgeons in making complex decisions,optimizing surgical workflows,and improving patient outcomes.Moreover,LLMs can automate surgical reporting and generate personalized patient education materials,streamlining documentation and improving patient engagement.However,challenges such as data scarcity,surgical semantic capture,real-time inference,and integration with existing systems need to be addressed for successful LLM integration.The future of laparoscopic surgery lies in the seamless integration of LLMs,enabling autonomous robotic surgery,predictive surgical planning,intraoperative decision support,virtual surgical assistants,and continuous learning.By harnessing the power of LLMs,laparoscopic surgery can be transformed,empowering surgeons and ultimately benefiting patients.