Classification of Conversational Sentences Using an Ensemble Pre-Trained Language Model with the Fine-Tuned Parameter

Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.

Sentence及其结构研究——以古典主义时期作品为例

文章以sentence为研究对象,一方面对国内外曲式学教材及相关文献展开概念比较、辨析与理论梳理,阐明其基本概念与内部结构;另一方面则以古典主义时期作品为分析实例,讨论这种特定主题(句法)的“结构范型”、并进一步对其“结构变形”的情况进行归类。

A Study of Nominal Predicate Sentences Under the Framework of Cognitive Grammar

Cognitive grammar,as a linguistic theory that attaches importance to the relationship between language and thinking,provides us with a more comprehensive way to understand the structure,semantics and cognitive processing of noun predicate sentences.Therefore,under the framework of cognitive grammar,this paper tries to analyze the semantic connection and cognitive process in noun predicate sentences from the semantic perspective and the method of example theory,and discusses the motivation of the formation of this construction,so as to provide references for in-depth analysis of the cognitive laws behind noun predicate sentences.

基于Sentence-BERT的专利技术主题聚类研究——以人工智能领域为例

[研究目的]将Sentence-BERT模型应用于专利技术主题聚类,解决专利文献为突出新颖性,常使用独特技术术语造成词汇向量语义特征稀疏的问题。[研究方法]以人工智能领域2015年-2019年的22370篇专利为实验数据。首先,采用Sentence-BERT算法对专利文献摘要文本进行向量化表示;其次,对向量化矩阵进行数据降维,利用HDBSCAN方式寻找原始数据中的高密度簇;最后,识别类簇文本集合中的主题特征,并完成主题呈现。[研究结论]对比LDA主题模型、K-means、doc2vec等方法,本文的实验结果提高了主题划分的细粒度和精确度,获得了较好的主题一致性。如何采用fine-tune策略进一步提升模型的效果,是未来该方法进一步深入探索的方向。

融合Sentence-BERT和LDA的评论文本主题识别

[目的/意义]为了解决评论文本主题识别时语义描述不充分以及学习到的主题语义连贯性不强等问题。本文将Sentence-BERT句子嵌入模型和LDA模型相结合,提升评论文本主题的语义性。[方法/过程]采用Sentence-BERT模型获取评论文本句子层面的向量特征,同时,采用LDA模型获取评论文本的概率主题向量,随后使用自动编码器连接两组向量,运用K-means算法对潜在空间向量进行聚类,从类簇中获取上下文主题信息。[结果/结论]通过对评论文本数据集的实验,本文方法可以较好地获得具有语义信息的主题词。Sentence-BERT模型与LDA结合,增加了模型的复杂性。通过对比,本文方法获得的主题一致性指标(Coherence)优于目前常见的评论文本主题识别方法。

CiteOpinion: Evidence-based Evaluation Tool for Academic Contributions of Research Papers Based on Citing Sentences

Purpose:To uncover the evaluation information on the academic contribution of research papers cited by peers based on the content cited by citing papers,and to provide an evidencebased tool for evaluating the academic value of cited papers.Design/methodology/approach:CiteOpinion uses a deep learning model to automatically extract citing sentences from representative citing papers;it starts with an analysis on the citing sentences,then it identifies major academic contribution points of the cited paper,positive/negative evaluations from citing authors and the changes in the subjects of subsequent citing authors by means of Recognizing Categories of Moves(problems,methods,conclusions,etc.),and sentiment analysis and topic clustering.Findings:Citing sentences in a citing paper contain substantial evidences useful for academic evaluation.They can also be used to objectively and authentically reveal the nature and degree of contribution of the cited paper reflected by citation,beyond simple citation statistics.Practical implications:The evidence-based evaluation tool CiteOpinion can provide an objective and in-depth academic value evaluation basis for the representative papers of scientific researchers,research teams,and institutions.Originality/value:No other similar practical tool is found in papers retrieved.Research limitations:There are difficulties in acquiring full text of citing papers.There is a need to refine the calculation based on the sentiment scores of citing sentences.Currently,the tool is only used for academic contribution evaluation,while its value in policy studies,technical application,and promotion of science is not yet tested.

Markedness and UG in Chinese Children's Acquisition of One-word and Negative Sentences

Thepresentstudyisaninvestigationandanalysisoftherelationshipbetweenmarkednessandfirstlanguageacquisitionsequence,asshowninthecasesofone-wordandnegativesentences.Hereourobjectivesaretoargueforthepriorityofunmarkednessovermarkednessintheacquisitionsequ...

The Contrasts of English Sentences and Chinese Sentences and Translation Skills

Since the opening-up policy was carried out in 1979, every facility of social modernized construction has developed at high speed; meanwhile, the need of English is increased year by year. The occasion and scope of using it are expanded with the communications among countries. Therefore, English has become the generally international language in our country; in particular, translation plays an important and irreplaceable part in English to convey information. This paper aims to introduce the contrasts of English sentences and Chinese sentences and discuss some skills of translating each other. Though it is not complete and authoritative, yet it may help some people to understand the differences between two languages and to grasp some practical skills.

A New Method for Calculating Similarity between Sentences and Application on Automatic Abstracting

Sentence similarity computing plays an important role in machine question-answering systems, machine-translation systems, information retrieval and automatic abstracting systems. This article firstly sums up several methods for calculating similarity between sentences, and brings out a new method which takes all factors into consideration including critical words, semantic information, sentential form and sen-tence length. And on this basis, a automatic abstracting system based on LexRank algorithm is implemented. We made several improvements in both sentence weight computing and redundancy resolution. The system described in this article could deal with single or multi-document summarization both in English and Chinese. With evaluations on two corpuses, our system could produce better summaries to a certain degree. We also show that our system is quite insensitive to the noise in the data that may result from an imperfect topical clustering of documents. And in the end, existing problem and the developing trend of automatic summariza-tion technology are discussed.

Information mining and similarity computation for semi-/un-structured sentences from the social data

In recent years,with the development of the social Internet of Things(IoT),all kinds of data accumulated on the network.These data,which contain a lot of social information and opinions.However,these data are rarely fully analyzed,which is a major obstacle to the intelligent development of the social IoT.In this paper,we propose a sentence similarity analysis model to analyze the similarity in people’s opinions on hot topics in social media and news pages.Most of these data are unstructured or semi-structured sentences,so the accuracy of sentence similarity analysis largely determines the model’s performance.For the purpose of improving accuracy,we propose a novel method of sentence similarity computation to extract the syntactic and semantic information of the semi-structured and unstructured sentences.We mainly consider the subjects,predicates and objects of sentence pairs and use Stanford Parser to classify the dependency relation triples to calculate the syntactic and semantic similarity between two sentences.Finally,we verify the performance of the model with the Microsoft Research Paraphrase Corpus(MRPC),which consists of 4076 pairs of training sentences and 1725 pairs of test sentences,and most of the data came from the news of social data.Extensive simulations demonstrate that our method outperforms other state-of-the-art methods regarding the correlation coefficient and the mean deviation.

Windowing and Gapping in Imperative Sentences: on the Basis of Talmy's"Causal-chain Windowing"Approach

This paper intends to analyze the six types of English imperative sentences proposed by Chen (1984) from a perspective of causal-chain windowing. It comes to the conclusions that Talmy's causal-chain windowing approach as well as the cognitive underpinnings of causal windowing and gapping is proved to be applicable in English imperative structures, and that generally speaking, the final portion of an imperative sentence is always windowed while the intermediate portions gapped.

Syntactic Study on Existential Sentences and Inspiration for the Teaching

As a special construction,existential sentences(ES) are wildly used in English.However,the Chinese,having different grammatical system,master this construction with much attention,occurring different levels of mistakes during each period of learning.In this paper,the author tries to study on ES from the syntactic perspective and provide some help for English teaching through the result of an experiment.

Studying English Ambiguous Sentences from Syntax Perspective

In daily communication, people often make misunderstanding because of the ambiguous phenomenon. However, by reason of using the ambiguous well, some people have endless remarks. To avoiding misunderstanding and making the ambiguous sentences play the positive effect better, this paper will analyze ambiguous sentences according to the types of ambiguous sentences from the Syntax perspective.

Teaching the Conditional Sentences to ESL Learners

This paper aims to explore how learners of English as a second language(ESL)acquire English conditional sentences and what causes their difficulties,especially focusing on how their native languages affect their acquisition of the conditional sentences.Interviews were carried out with four undergraduate ESL students of University of Central Oklahoma in the United States who are respectively Chinese,Korean,French,and Greek.By conducting interviews with them,the participants’perceptions of acquiring English conditional sentences will be collected and analyzed.There will be some typical errors of constructing conditional sentences demonstrated.Moreover,some pedagogical implications will also be provided,which will help students have a better command of the conditional sentences.

Evidence for the Hierarchical Structure of Sentences Revisited

To reveal invariant properties of human languages is one of the central goals of modern generative grammar.Hierarchical feature of sentence construction, one of the key notions in this regard, reflects one of the most widely accepted invariant properties of human languages. However,researchers also challenge this point and argue that languages differ fundamentally from one another so that it is very hard to find any single structural property which they share. This paper revisits the evidence for the hierarchical structure of sentences from the principle of structure dependence in first and second language acquisition and argues that the recognition of hierarchical structure of sentences is essential to any linguistic exploration.

An Analysis of Translating Techniques of Long and Complex English Sentences

Long and complex English sentences, just as the name implies, are long sentences with complicated structures. The most typical features of this kind of sentences are the comprehensive use of different subordinate clauses and the flexible use of all kinds of prepositional phrases and participle phrases, which lead to complex sentence structures and varied constituent relationships, and bring great difficulties to English-Chinese translation practice. The paper concludes that, in the translation of long and complex English sentences, the translator should first figure out the logic relationship of the original text, and then, according to the Chinese expression customs, make a flexible use of various translation methods and techniques, and finally he can get a faithful and smooth translated text.

Real-time prediction of mechanical behaviors of underwater shield tunnel structure using machine learning method based on structural health monitoring data

Predicting the mechanical behaviors of structure and perceiving the anomalies in advance are essential to ensuring the safe operation of infrastructures in the long run.In addition to the incomplete consideration of influencing factors,the prediction time scale of existing studies is rough.Therefore,this study focuses on the development of a real-time prediction model by coupling the spatio-temporal correlation with external load through autoencoder network(ATENet)based on structural health monitoring(SHM)data.An autoencoder mechanism is performed to acquire the high-level representation of raw monitoring data at different spatial positions,and the recurrent neural network is applied to understanding the temporal correlation from the time series.Then,the obtained temporal-spatial information is coupled with dynamic loads through a fully connected layer to predict structural performance in next 12 h.As a case study,the proposed model is formulated on the SHM data collected from a representative underwater shield tunnel.The robustness study is carried out to verify the reliability and the prediction capability of the proposed model.Finally,the ATENet model is compared with some typical models,and the results indicate that it has the best performance.ATENet model is of great value to predict the realtime evolution trend of tunnel structure.

A Combinatorial Analysis of Tree-Like Sentences

A sentence over a finite alphabet A, is a finite sequence of non-empty words over A. More generally, we define a graphical sentence over A by attaching a non-empty word over A to each arrow and each loop of a connected directed graph (digraph, for short). Each word is written according to the direction of its corresponding arrow or loop. Graphical sentences can be used to encode sets of sentences in a compact way: the readable sentences of a graphical sentence being the sentences corresponding to directed paths in the digraph. We apply combinatorial equations on enriched trees and rooted trees, in the context of combinatorial species and Pólya theories, to analyze parameters in classes of tree-like sentences. These are graphical sentences constructed on tree-like digraphs.

A Study of Characteristics and Translation of Existential Sentences in Chinese and English

The syntactic characteristics of existential sentences (ES) in Chinese and English are systematically discussed. Chinese existential sentences have three major components: A, B, and C, i.e., the locative phrase, the existential verb, and the existential noun phrase. English existential sentences consist of the expletive “there”, existential verb, existential noun or noun phrase and locative phrase. In terms of syntactic characteristics, similarities and dissimilarities coexist. When translating ES in Chinese and English, some important skills are needed, such as replacement of locative phrase, addition or deletion of “there”, adjustment of verbs, and the change of determiners before the existential noun.

On Translation of Long Sentences Based on the Novel-Of Time and the River

Of Time and the River is a famous novel written by American novelist–Thomas Wolfe who was a major American novelist of the early 20th century.His books are well received among Chinese readers.In the novel—Of Time and the River,we can find Wolfe’s frequent use of long sentences.In this paper,the skills of translating long sentences and the ways of analyzing sentence structures are discussed by several examples selected from the novel.Based on my own translation,we can find the importance of sentence structure analysis in translating long sentences.