Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter

Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

Generativity of Self-Organizing Processes and Their Correlative Description in Terms of a Formal Language of Meta-Ordinal Generative Nature, in the Light of the Maximum Ordinality Principle and the Explicit Solution to the “Three-Body Problem”

The main objective of this paper is to demonstrate that the internal processes of Self-Organizing Systems represent a unique and singular process, characterized by their specific generativity. This process can be modeled using the Maximum Ordinality Principle and its associated formal language, known as the “Incipient” Differential Calculus (IDC).

Six-Writings multimodal processing with pictophonetic coding to enhance Chinese language models

While large language models(LLMs)have made significant strides in natural language processing(NLP),they continue to face challenges in adequately addressing the intricacies of the Chinese language in certain scenarios.We propose a framework called Six-Writings multimodal processing(SWMP)to enable direct integration of Chinese NLP(CNLP)with morphological and semantic elements.The first part of SWMP,known as Six-Writings pictophonetic coding(SWPC),is introduced with a suitable level of granularity for radicals and components,enabling effective representation of Chinese characters and words.We conduct several experimental scenarios,including the following:(1)We establish an experimental database consisting of images and SWPC for Chinese characters,enabling dual-mode processing and matrix generation for CNLP.(2)We characterize various generative modes of Chinese words,such as thousands of Chinese idioms,used as question-and-answer(Q&A)prompt functions,facilitating analogies by SWPC.The experiments achieve 100%accuracy in answering all questions in the Chinese morphological data set(CA8-Mor-10177).(3)A fine-tuning mechanism is proposed to refine word embedding results using SWPC,resulting in an average relative error of≤25%for 39.37%of the questions in the Chinese wOrd Similarity data set(COS960).The results demonstrate that SWMP/SWPC methods effectively capture the distinctive features of Chinese and offer a promising mechanism to enhance CNLP with better efficiency.

Vision Enhanced Generative Pre-trained Language Model for Multimodal Sentence Summarization

Multimodal sentence summarization(MMSS)is a new yet challenging task that aims to generate a concise summary of a long sentence and its corresponding image.Although existing methods have gained promising success in MMSS,they overlook the powerful generation ability of generative pre-trained language models(GPLMs),which have shown to be effective in many text generation tasks.To fill this research gap,we propose to using GPLMs to promote the performance of MMSS.Notably,adopting GPLMs to solve MMSS inevitably faces two challenges:1)What fusion strategy should we use to inject visual information into GPLMs properly?2)How to keep the GPLM′s generation ability intact to the utmost extent when the visual feature is injected into the GPLM.To address these two challenges,we propose a vision enhanced generative pre-trained language model for MMSS,dubbed as Vision-GPLM.In Vision-GPLM,we obtain features of visual and textual modalities with two separate encoders and utilize a text decoder to produce a summary.In particular,we utilize multi-head attention to fuse the features extracted from visual and textual modalities to inject the visual feature into the GPLM.Meanwhile,we train Vision-GPLM in two stages:the vision-oriented pre-training stage and fine-tuning stage.In the vision-oriented pre-training stage,we particularly train the visual encoder by the masked language model task while the other components are frozen,aiming to obtain homogeneous representations of text and image.In the fine-tuning stage,we train all the components of Vision-GPLM by the MMSS task.Extensive experiments on a public MMSS dataset verify the superiority of our model over existing baselines.

An Approach to Modelling and Analysing Reliability of Breeze/ADL-based Software Architecture

Breeze/architecture description language（ADL）, is an eX tensible markup language（XML） based architecture description language which is used to model software systems at the architecture level. Though Breeze/ADL provides an appropriate basis for architecture modelling, it can neither analyse nor evaluate the architecture reliability. In this paper, we propose a Breeze/ADL based strategy which, by combining generalized stochastic Petri net（GSPN） and tools for reliability analysis, supports architecture reliability modelling and evaluation. This work expands the idea in three directions： Firstly, we give a Breeze/ADL reliability model in which we add error attributes to Breeze/ADL error model for capturing architecture error information, and at the same time perform the system error state transition through the Breeze/ADL production. Secondly, we present how to map a Breeze/ADL reliability model to a GSPN model, which in turn can be used for reliability analysis. The other task is to develop a Breeze/ADL reliability analysis modelling tool–EXGSPN（Breeze/ADL reliability analysis modelling tool）, and combine it with platform independent petri net editor 2（PIPE2） to carry out a reliability assessment.Abstract： Breeze/architecture description language （ADL）, is an eXtensible markup language （XML） based architecture description language which is used to model software systems at the architecture level. Though Breeze/ADL provides an appropriate basis for architecture modelling, it can neither analyse nor evaluate the architecture reliability. In this paper, we propose a Breeze/ADL based strategy which, by combining generalized stochastic Petri net （GSPN） and tools for reliability analysis, supports architecture reliability modelling and evaluation. This work expands the idea in three directions： Firstly, we give a Breeze/ADL reliability model in which we add error attributes to Breeze/ADL error model for capturing architecture error information, and at the same time perform the system error state transition through the Breeze/ADL production. Secondly, we present how to map a Breeze/ADL reliability model to a GSPN model, which in turn can be used for reliability analysis. The other task is to develop a Breeze/ADL reliability analysis modelling tool-EXGSPN （Breeze/ADL reliability analysis modelling tool）, and combine it with platform independent petri net editor 2 （PIPE2） to carry out a reliability assessment.