Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter

Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

Sentence及其结构研究——以古典主义时期作品为例

文章以sentence为研究对象,一方面对国内外曲式学教材及相关文献展开概念比较、辨析与理论梳理,阐明其基本概念与内部结构;另一方面则以古典主义时期作品为分析实例,讨论这种特定主题(句法)的“结构范型”、并进一步对其“结构变形”的情况进行归类。

英语with复合结构赏析助力思辨性读写探索

作为英语中常见的一类复合结构,with结构聚合多重信息,体现英语长难句的特征。具有包容性、能动性和体验性的思辨性读写能力提升离不开对with复合结构的识别、推敲、加工和赏析。文中用具体实例分析with复合结构在表达复杂内容、传递多层含义等传情达意方面的独特功能,并且阐释英语with复合结构赏析助力思辨性读写的逻辑关系。为了能够精准地思辨性理解英语长难句,学习者需要捕捉英语语言微妙之处,品味语言魅力,养成语言输出时使用with复合结构的习惯,实现英语读写彼此促进、相得益彰。

Next Words Prediction and Sentence Completion in Bangla Language Using GRU-Based RNN on N-Gram Language Model

We use a lot of devices in our daily life to communicate with others. In this modern world, people use email, Facebook, Twitter, and many other social network sites for exchanging information. People lose their valuable time misspelling and retyping, and some people are not happy to type large sentences because they face unnecessary words or grammatical issues. So, for this reason, word predictive systems help to exchange textual information more quickly, easier, and comfortably for all people. These systems predict the next most probable words and give users to choose of the needed word from these suggested words. Word prediction can help the writer by predicting the next word and helping complete the sentence correctly. This research aims to forecast the most suitable next word to complete a sentence for any given context. In this research, we have worked on the Bangla language. We have presented a process that can expect the next maximum probable and proper words and suggest a complete sentence using predicted words. In this research, GRU-based RNN has been used on the N-gram dataset to develop the proposed model. We collected a large dataset using multiple sources in the Bangla language and also compared it to the other approaches that have been used such as LSTM, and Naive Bayes. But this suggested approach provides excellent exactness than others. Here, the Unigram model provides 88.22%, Bi-gram model is 99.24%, Tri-gram model is 97.69%, and 4-gram and 5-gram models provide 99.43% and 99.78% on average accurateness. We think that our proposed method profound impression on Bangla search engines.

基于Sentence-BERT的专利技术主题聚类研究——以人工智能领域为例

[研究目的]将Sentence-BERT模型应用于专利技术主题聚类,解决专利文献为突出新颖性,常使用独特技术术语造成词汇向量语义特征稀疏的问题。[研究方法]以人工智能领域2015年-2019年的22370篇专利为实验数据。首先,采用Sentence-BERT算法对专利文献摘要文本进行向量化表示;其次,对向量化矩阵进行数据降维,利用HDBSCAN方式寻找原始数据中的高密度簇;最后,识别类簇文本集合中的主题特征,并完成主题呈现。[研究结论]对比LDA主题模型、K-means、doc2vec等方法,本文的实验结果提高了主题划分的细粒度和精确度,获得了较好的主题一致性。如何采用fine-tune策略进一步提升模型的效果,是未来该方法进一步深入探索的方向。

A Study of Nominal Predicate Sentences Under the Framework of Cognitive Grammar

Cognitive grammar,as a linguistic theory that attaches importance to the relationship between language and thinking,provides us with a more comprehensive way to understand the structure,semantics and cognitive processing of noun predicate sentences.Therefore,under the framework of cognitive grammar,this paper tries to analyze the semantic connection and cognitive process in noun predicate sentences from the semantic perspective and the method of example theory,and discusses the motivation of the formation of this construction,so as to provide references for in-depth analysis of the cognitive laws behind noun predicate sentences.

Review of Research on English Translation of Chinese Running Sentences

In order to convey complete meanings,there is a phenomenon in Chinese of using multiple running sentences.Xu Jingning(2023,p.66)states,“In communication,a complete expression of meaning often requires more than one clause,which is common in human languages.”Domestic research on running sentences includes discussions on defining the concept and structural features of running sentences,sentence properties,sentence pattern classifications and their criteria,as well as issues related to translating running sentences into English.This article primarily focuses on scholarly research into the English translation of running sentences in China,highlighting recent achievements and identifying existing issues in the study of running sentence translation.However,by reviewing literature on the translation of running sentences,it is found that current research in the academic community on non-core running sentences is limited.Therefore,this paper proposes relevant strategies to address this issue.

An Analysis Into Ba Sentences Under Image Schema Theory——A Case Study of Words of Fire——Poems by Jidi Majia

The Ba sentence is unique existence in Chinese. There is no corresponding sentence pattern in English. Thus, the translation of Ba sentences is a challenge for translators. Even with the myriad analyses on the translation of Ba sentences, small quantity of them demonstrates the correctness and the reading effect of the translation by using a specific theory. And the image schema theory reflects the projects between the source domain and the target domain, while in the translation analysis there are schemata in the source language and the target language. So the paper does comparisons of schemata between the source text and the target text of Ba sentences, which are chosen from the English translation of Words of Fire—Poems by Jidi Majia translated by Denis Mair. After the demonstration, the following conclusions are found: First, the schema of the sentence is decided by verbs of the Ba sentences, rather than by the sentence structures;second, the image schema is a feasible tool to check correctness of Ba sentences translation.

Cross the chasm from the immediate execution of death penalty to the death sentence with a reprieve --- Interpretation of the ＂not to execute immediately＂

Retaining the death penalty and strict restricting the application of the death penalty is now a basic criminal policy in China, and from the judicial level, the key to the restriction of the death penalty is to study what lenient sentencing discretion the criminal has to constitute ＂not to execute immediately＂ when he has reached the standard of the immediate execution of the death penalty, to cross the chasm from the immediate execution of the death penalty to the death sentence with a reprieve. The basic process of the sentencing is to establish a baseline punishment on the basis of the social harmfulness of the activities of the criminal, and then measure the profits and losses according to the offender＇s personal danger. Therefore, although the social harmfulness of the activities of the criminal reaches the standard of the ＂most heinous crimes＂, due to the existence of the fault of the victim, active compensation for the victim, and the motives of the small blames and other lenient sentencing discretions, the criminal＇s danger has not reached the degree of ＂flagrance＂. Apply the death sentence with a two-year reprieve and even the life imprisonment generally. If there are some strict sentencing discretions, such as ＂the crime means is extremely cruel＂, carefully consider the use of the immediate execution of the death penalty. Under the circumstances of the concurrence of the sentencing, carry on the overall consideration based on the comprehensive measurement of various circumstances of the sentencing.

融合Sentence-BERT和LDA的评论文本主题识别

[目的/意义]为了解决评论文本主题识别时语义描述不充分以及学习到的主题语义连贯性不强等问题。本文将Sentence-BERT句子嵌入模型和LDA模型相结合,提升评论文本主题的语义性。[方法/过程]采用Sentence-BERT模型获取评论文本句子层面的向量特征,同时,采用LDA模型获取评论文本的概率主题向量,随后使用自动编码器连接两组向量,运用K-means算法对潜在空间向量进行聚类,从类簇中获取上下文主题信息。[结果/结论]通过对评论文本数据集的实验,本文方法可以较好地获得具有语义信息的主题词。Sentence-BERT模型与LDA结合,增加了模型的复杂性。通过对比,本文方法获得的主题一致性指标(Coherence)优于目前常见的评论文本主题识别方法。

On Sentence Complexity in THE TIMES: A Comparative Study of SentenceLength and Sentence Complexity in the News Section and the Sports Section

My investigation will serve two purposes. First, I shall investigate the function of the subclauses in the corpus in relation to their complexity, and I shall establish whether there is a correlation between sentence length and sentence complexity.Second, I shall analyse the complexity of the subclauses collected from the two sections and compare the results from these sections, focusing on finite subclauses and non-finite subclauses. I hope to be able to point out some differences in style between the news and sports sections concerning the use of subordinate clauses in various syntactic functions in order to examine how the choice of linguistic structures differs in different sections of The Times.

Markedness and UG in Chinese Children's Acquisition of One-word and Negative Sentences

Thepresentstudyisaninvestigationandanalysisoftherelationshipbetweenmarkednessandfirstlanguageacquisitionsequence,asshowninthecasesofone-wordandnegativesentences.Hereourobjectivesaretoargueforthepriorityofunmarkednessovermarkednessintheacquisitionsequ...

The Contrasts of English Sentences and Chinese Sentences and Translation Skills

Since the opening-up policy was carried out in 1979, every facility of social modernized construction has developed at high speed; meanwhile, the need of English is increased year by year. The occasion and scope of using it are expanded with the communications among countries. Therefore, English has become the generally international language in our country; in particular, translation plays an important and irreplaceable part in English to convey information. This paper aims to introduce the contrasts of English sentences and Chinese sentences and discuss some skills of translating each other. Though it is not complete and authoritative, yet it may help some people to understand the differences between two languages and to grasp some practical skills.

Information mining and similarity computation for semi-/un-structured sentences from the social data

In recent years,with the development of the social Internet of Things(IoT),all kinds of data accumulated on the network.These data,which contain a lot of social information and opinions.However,these data are rarely fully analyzed,which is a major obstacle to the intelligent development of the social IoT.In this paper,we propose a sentence similarity analysis model to analyze the similarity in people’s opinions on hot topics in social media and news pages.Most of these data are unstructured or semi-structured sentences,so the accuracy of sentence similarity analysis largely determines the model’s performance.For the purpose of improving accuracy,we propose a novel method of sentence similarity computation to extract the syntactic and semantic information of the semi-structured and unstructured sentences.We mainly consider the subjects,predicates and objects of sentence pairs and use Stanford Parser to classify the dependency relation triples to calculate the syntactic and semantic similarity between two sentences.Finally,we verify the performance of the model with the Microsoft Research Paraphrase Corpus(MRPC),which consists of 4076 pairs of training sentences and 1725 pairs of test sentences,and most of the data came from the news of social data.Extensive simulations demonstrate that our method outperforms other state-of-the-art methods regarding the correlation coefficient and the mean deviation.

Sentence Similarity Measurement with Convolutional Neural Networks Using Semantic and Syntactic Features

Calculating the semantic similarity of two sentences is an extremely challenging problem.We propose a solution based on convolutional neural networks(CNN)using semantic and syntactic features of sentences.The similarity score between two sentences is computed as follows.First,given a sentence,two matrices are constructed accordingly,which are called the syntax model input matrix and the semantic model input matrix;one records some syntax features,and the other records some semantic features.By experimenting with different arrangements of representing the syntactic and semantic features of the sentences in the matrices,we adopt the most effective way of constructing the matrices.Second,these two matrices are given to two neural networks,which are called the sentence model and the semantic model,respectively.The convolution process of the neural networks of the two models is carried out in multiple perspectives.The outputs of the two models are combined as a vector,which is the representation of the sentence.Third,given the representation vectors of two sentences,the similarity score of these representations is computed by a layer in the CNN.Experiment results show that our algorithm(SSCNN)surpasses the performance MPCPP,which noticeably the best recent work of using CNN for sentence similarity computation.Comparing with MPCNN,the convolution computation in SSCNN is considerably simpler.Based on the results of this work,we suggest that by further utilization of semantic and syntactic features,the performance of sentence similarity measurements has considerable potentials to be improved in the future.

An Abstractive Summarization Technique with Variable Length Keywords as per Document Diversity

Text Summarization is an essential area in text mining,which has procedures for text extraction.In natural language processing,text summarization maps the documents to a representative set of descriptive words.Therefore,the objective of text extraction is to attain reduced expressive contents from the text documents.Text summarization has two main areas such as abstractive,and extractive summarization.Extractive text summarization has further two approaches,in which the first approach applies the sentence score algorithm,and the second approach follows the word embedding principles.All such text extractions have limitations in providing the basic theme of the underlying documents.In this paper,we have employed text summarization by TF-IDF with PageRank keywords,sentence score algorithm,and Word2Vec word embedding.The study compared these forms of the text summarizations with the actual text,by calculating cosine similarities.Furthermore,TF-IDF based PageRank keywords are extracted from the other two extractive summarizations.An intersection over these three types of TD-IDF keywords to generate the more representative set of keywords for each text document is performed.This technique generates variable-length keywords as per document diversity instead of selecting fixedlength keywords for each document.This form of abstractive summarization improves metadata similarity to the original text compared to all other forms of summarized text.It also solves the issue of deciding the number of representative keywords for a specific text document.To evaluate the technique,the study used a sample of more than eighteen hundred text documents.The abstractive summarization follows the principles of deep learning to create uniform similarity of extracted words with actual text and all other forms of text summarization.The proposed technique provides a stable measure of similarity as compared to existing forms of text summarization.

Windowing and Gapping in Imperative Sentences: on the Basis of Talmy's"Causal-chain Windowing"Approach

This paper intends to analyze the six types of English imperative sentences proposed by Chen (1984) from a perspective of causal-chain windowing. It comes to the conclusions that Talmy's causal-chain windowing approach as well as the cognitive underpinnings of causal windowing and gapping is proved to be applicable in English imperative structures, and that generally speaking, the final portion of an imperative sentence is always windowed while the intermediate portions gapped.

Syntactic Study on Existential Sentences and Inspiration for the Teaching

As a special construction,existential sentences(ES) are wildly used in English.However,the Chinese,having different grammatical system,master this construction with much attention,occurring different levels of mistakes during each period of learning.In this paper,the author tries to study on ES from the syntactic perspective and provide some help for English teaching through the result of an experiment.

Teaching the Conditional Sentences to ESL Learners

This paper aims to explore how learners of English as a second language(ESL)acquire English conditional sentences and what causes their difficulties,especially focusing on how their native languages affect their acquisition of the conditional sentences.Interviews were carried out with four undergraduate ESL students of University of Central Oklahoma in the United States who are respectively Chinese,Korean,French,and Greek.By conducting interviews with them,the participants’perceptions of acquiring English conditional sentences will be collected and analyzed.There will be some typical errors of constructing conditional sentences demonstrated.Moreover,some pedagogical implications will also be provided,which will help students have a better command of the conditional sentences.

Evidence for the Hierarchical Structure of Sentences Revisited

To reveal invariant properties of human languages is one of the central goals of modern generative grammar.Hierarchical feature of sentence construction, one of the key notions in this regard, reflects one of the most widely accepted invariant properties of human languages. However,researchers also challenge this point and argue that languages differ fundamentally from one another so that it is very hard to find any single structural property which they share. This paper revisits the evidence for the hierarchical structure of sentences from the principle of structure dependence in first and second language acquisition and argues that the recognition of hierarchical structure of sentences is essential to any linguistic exploration.