A Natural Language Generation Algorithm for Greek by Using Hole Semantics and a Systemic Grammatical Formalism

This work is about the progress of previous related work based on an experiment to improve the intelligence of robotic systems,with the aim of achieving more linguistic communication capabilities between humans and robots.In this paper,the authors attempt an algorithmic approach to natural language generation through hole semantics and by applying the OMAS-III computational model as a grammatical formalism.In the original work,a technical language is used,while in the later works,this has been replaced by a limited Greek natural language dictionary.This particular effort was made to give the evolving system the ability to ask questions,as well as the authors developed an initial dialogue system using these techniques.The results show that the use of these techniques the authors apply can give us a more sophisticated dialogue system in the future.

Communication Mediated through Natural Language Generation in Big Data Environments: The Case of Nomao

Along with the development of big data, various Natural Language Generation systems (NLGs) have recently been developed by different companies. The aim of this paper is to propose a better understanding of how these systems are designed and used. We propose to study in details one of them which is the NLGs developed by the company Nomao. First, we show the development of this NLGs underlies strong economic stakes since the business model of Nomao partly depends on it. Then, thanks to an eye movement analysis conducted with 28 participants, we show that the texts generated by Nomao’s NLGs contain syntactic and semantic structures that are easy to read but lack socio-semantic coherence which would improve their understanding. From a scientific perspective, our research results highlight the importance of socio-semantic coherence in text-based communication produced by NLGs.

Automatic Generation of Artificial Space Weather Forecast Product Based on Sequence-to-sequence Model

Both analyzing a large amount of space weather observed data and alleviating personal experience bias are significant challenges in generating artificial space weather forecast products.With the use of natural language generation methods based on the sequence-to-sequence model,space weather forecast texts can be automatically generated.To conduct our generation tasks at a fine-grained level,a taxonomy of space weather phenomena based on descriptions is presented.Then,our MDH(Multi-Domain Hybrid)model is proposed for generating space weather summaries in two stages.This model is composed of three sequence-to-sequence-based deep neural network sub-models(one Bidirectional Auto-Regressive Transformers pre-trained model and two Transformer models).Then,to evaluate how well MDH performs,quality evaluation metrics based on two prevalent automatic metrics and our innovative human metric are presented.The comprehensive scores of the three summaries generating tasks on testing datasets are 70.87,93.50,and 92.69,respectively.The results suggest that MDH can generate space weather summaries with high accuracy and coherence,as well as suitable length,which can assist forecasters in generating high-quality space weather forecast products,despite the data being starved.

STUDY ON NATURAL LANGUAGE INTERFACE OF NETWORK FAULT DIAGNOSIS EXPERT SYSTEM

The expert system is an important field of the artificial intelligence. The traditional interface of the expert system is the command, menu and window at present. It limits the application of the expert system and embarrasses the enthusiasm of using expert system. Combining with the study on the expert system of network fault diagnosis, the natural language interface of the expert system has been discussed in this article. This interface can understand and generate Chinese sentences. Using this interface, the user and field experts can use the expert system to diagnose the fault of network conveniently. In the article, first, the extended production rule has been proposed. Then the methods of Chinese sentence generation from conceptual graphs and the model of expert system are introduced in detail. Using this model, the network fault diagnosis expert system and its natural language interface have been developed with Prolog.

基于Schema和RST的自然语言生成混合规划方法

在自然语言生成系统的内容规划阶段，主要的手段是基于Schema的规划方法和修辞结构理论RST方法，Schema方法只适用于模式固定、细节略有变化的文本的生成，而RST方法则比较灵活，但是它的数据结构、规则库比较难建立。把这两种方法集成起来，并对Schema方法进行改造，使之能与RST有机地结合起来。在规划文本时首先用Schema方法规划全局的组织，然后用RST方法控制局部话语的连贯。这种结构可弥补采用单一方法的不足，具有双方的优点。

Cyber Deception Using NLP

Cyber security addresses the protection of information systems in cyberspace. These systems face multiple attacks on a daily basis, with the level of complication getting increasingly challenging. Despite the existence of multiple solutions, attackers are still quite successful at identifying vulnerabilities to exploit. This is why cyber deception is increasingly being used to divert attackers’ attention and, therefore, enhance the security of information systems. To be effective, deception environments need fake data. This is where Natural Language (NLP) Processing comes in. Many cyber security models have used NLP for vulnerability detection in information systems, email classification, fake citation detection, and many others. Although it is used for text generation, existing models seem to be unsuitable for data generation in a deception environment. Our goal is to use text generation in NLP to generate data in the deception context that will be used to build multi-level deception in information systems. Our model consists of three (3) components, including the connection component, the deception component, composed of several states in which an attacker may be, depending on whether he is malicious or not, and the text generation component. The text generation component considers as input the real data of the information system and allows the production of several texts as output, which are usable at different deception levels.

Automated Test Case Generation from Requirements: A Systematic Literature Review

Software testing is an important and cost intensive activity in software development.The major contribution in cost is due to test case generations.Requirement-based testing is an approach in which test cases are derivative from requirements without considering the implementation’s internal structure.Requirement-based testing includes functional and nonfunctional requirements.The objective of this study is to explore the approaches that generate test cases from requirements.A systematic literature review based on two research questions and extensive quality assessment criteria includes studies.The study identies 30 primary studies from 410 studies spanned from 2000 to 2018.The review’s nding shows that 53%of journal papers,42%of conference papers,and 5%of book chapters’address requirementsbased testing.Most of the studies use UML,activity,and use case diagrams for test case generation from requirements.One of the signicant lessons learned is that most software testing errors are traced back to errors in natural language requirements.A substantial amount of work focuses on UML diagrams for test case generations,which cannot capture all the system’s developed attributes.Furthermore,there is a lack of UML-based models that can generate test cases from natural language requirements by rening them in context.Coverage criteria indicate how efciently the testing has been performed 12.37%of studies use requirements coverage,20%of studies cover path coverage,and 17%study basic coverage.

Intrinsic and Extrinsic Automatic Evaluation Strategies for Paraphrase Generation Systems

Paraphrase is an expression of a text with alternative words and orders to achieve a better clarity. Paraphrases have been found vital for augmenting training dataset, which aid to enhance performance of machine learning models that intended for various natural language processing (NLP) tasks. Thus, recently, automatic paraphrase generation has received increasing attention. However, evaluating quality of generated paraphrases is technically challenging. In the literature, the importance of generated paraphrases is tended to be determined by their impact on the performance of other NLP tasks. This kind of evaluation is referred as extrinsic evaluation, which requires high computational resources to train and test the models. So far, very little attention has been paid to the role of intrinsic evaluation in which quality of generated paraphrase is judged against predefined ground truth (reference paraphrases). In fact, it is also very challenging to find ideal and complete reference paraphrases. Therefore, in this study, we propose semantic or meaning oriented automatic evaluation metric that helps to evaluate quality of generated paraphrases against the original text, which is an intrinsic evaluation approach. Further, we evaluate quality of the paraphrases by assessing their impact on other NLP tasks, which is an extrinsic evaluation method. The goal is to explore the relationship between intrinsic and extrinsic evaluation methods. To ensure the effectiveness of proposed evaluation methods, extensive experiments are done on different publicly available datasets. The experimental results demonstrate that our proposed intrinsic and extrinsic evaluation strategies are promising. The results further reveal that there is a significant correlation between intrinsic and extrinsic evaluation approaches.

Automatic Generation of UML Class Diagrams for Object-oriented Design and Programing Course

UML Class diagram generation from textual requirements is an important task in object-oriented design and programing course.This study proposes a method for automatically generating class diagrams from Chinese textual requirements on the basis of Natural Language Processing(NLP)and mapping rules for sentence pattern matching.First,classes are identified through entity recognition rules and candidate class pruning rules using NLP from requirements.Second,class attributes and relationships between classes are extracted using mapping rules for sentence pattern matching on the basis of NLP.Third,we developed an assistant tool integrated into a precision micro classroom system for automatic generation of class diagram,to effectively assist the teaching of object-oriented design and programing course.Results are evaluated with precision,accuracy and recall from eight requirements of object-oriented design and programing course using truth values created by teachers.Our research should benefit beginners of object-oriented design and programing course,who may be students or software developers.It helps them to create correct domain models represented in the UML class diagram.

Advances and challenges in artificial intelligence text generation

Text generation is an essential research area in artificial intelligence(AI)technology and natural language processing and provides key technical support for the rapid development of AI-generated content(AIGC).It is based on technologies such as natural language processing,machine learning,and deep learning,which enable learning language rules through training models to automatically generate text that meets grammatical and semantic requirements.In this paper,we sort and systematically summarize the main research progress in text generation and review recent text generation papers,focusing on presenting a detailed understanding of the technical models.In addition,several typical text generation application systems are presented.Finally,we address some challenges and future directions in AI text generation.We conclude that improving the quality,quantity,interactivity,and adaptability of generated text can help fundamentally advance AI text generation development.

冬奥气象服务文本自动生成模型研究

目前的冬奥气象服务文本依赖手工撰写,存在生成效率不高、内容提取不够精准和英文自然语言表达不足的问题。在深入分析冬奥服务文本特征基础上,提出以自然语言生成理论为指导,研究冬奥数据和知识构建特征引擎,为文本生成提供内容规划;结合功能合一语法(Functional unification grammar,FUG),开展递归运算,实现从短语到句式生成;利用XML Schema对文本篇章、段落结构进行组织管理。经模型自动生成的冬奥气象服务文本在2019年至2020年现场服务团队的冬训工作得以应用,为其他大型冬季赛事气象服务保障提供了技术方案。

面向新领域的事件抽取研究综述

在当前互联网时代,大量新领域下的非结构文本数据中蕴含了海量信息。面向新领域的事件抽取方法研究能快速地构建领域知识库,用于支撑基于知识的下游应用。但现有事件抽取系统的领域限定性强,在新领域中从零构建会极度依赖事件体系和标注数据的质量及规模,需要大量人力和专家知识来定制模板和标注语料。而且数据集中常见在相同的上下文中出现多个相关联的事件实例,对事件抽取和真实性检测产生了极大阻碍。本文针对面向新领域的事件抽取这一新兴研究领域进行综述,从事件模板推导、多实例联合事件抽取、事件真实性检测三个研究方向介绍了相关工作的研究现状,并对目前存在的重点和难点问题进行了讨论,指出了下一步需要开展的研究工作。

百科知识以图式形式在话语理解过程中的作用

Sperber & Wilson的关联理论认为话语理解的过程实质上是认知的推理过程,这种推理是非论证性的。百科知识是这一过程的重要参与因素,它以图式的形式储存在人脑长时记忆中。受话者百科知识的结构制约着受话者认知语境的选择和构建,从而制约着话语的理解。文章先对图式和话语理解的一般过程及参与因素进行简要概述,然后重点分析话语理解者的百科知识结构在选择和构建最佳相关语境过程中的重要作用,最后阐述百科知识以图式形式对话语理解的作用对外语教学的启示。

自然语言生成系统的实现技术分析

自然语言生成（NaturalLanguageGeneration）是自然语言处理两大域之一，国外许多学者都在致力于NLG技术的研究。本文主要介绍有关文本自动生成器的实现技术。首先简单的阐述文本自动生成的三大主要任务；其次，具体描述四种常用的生成器实现技术及其优缺点；最后，文章谈到了一个具体实例的实现模型。

Vision Enhanced Generative Pre-trained Language Model for Multimodal Sentence Summarization

Multimodal sentence summarization(MMSS)is a new yet challenging task that aims to generate a concise summary of a long sentence and its corresponding image.Although existing methods have gained promising success in MMSS,they overlook the powerful generation ability of generative pre-trained language models(GPLMs),which have shown to be effective in many text generation tasks.To fill this research gap,we propose to using GPLMs to promote the performance of MMSS.Notably,adopting GPLMs to solve MMSS inevitably faces two challenges:1)What fusion strategy should we use to inject visual information into GPLMs properly?2)How to keep the GPLM′s generation ability intact to the utmost extent when the visual feature is injected into the GPLM.To address these two challenges,we propose a vision enhanced generative pre-trained language model for MMSS,dubbed as Vision-GPLM.In Vision-GPLM,we obtain features of visual and textual modalities with two separate encoders and utilize a text decoder to produce a summary.In particular,we utilize multi-head attention to fuse the features extracted from visual and textual modalities to inject the visual feature into the GPLM.Meanwhile,we train Vision-GPLM in two stages:the vision-oriented pre-training stage and fine-tuning stage.In the vision-oriented pre-training stage,we particularly train the visual encoder by the masked language model task while the other components are frozen,aiming to obtain homogeneous representations of text and image.In the fine-tuning stage,we train all the components of Vision-GPLM by the MMSS task.Extensive experiments on a public MMSS dataset verify the superiority of our model over existing baselines.

DLBT:Deep Learning-Based Transformer to Generate Pseudo-Code from Source Code

Understanding the content of the source code and its regular expression is very difficult when they are written in an unfamiliar language.Pseudo-code explains and describes the content of the code without using syntax or programming language technologies.However,writing Pseudo-code to each code instruction is laborious.Recently,neural machine translation is used to generate textual descriptions for the source code.In this paper,a novel deep learning-based transformer(DLBT)model is proposed for automatic Pseudo-code generation from the source code.The proposed model uses deep learning which is based on Neural Machine Translation(NMT)to work as a language translator.The DLBT is based on the transformer which is an encoder-decoder structure.There are three major components:tokenizer and embeddings,transformer,and post-processing.Each code line is tokenized to dense vector.Then transformer captures the relatedness between the source code and the matching Pseudo-code without the need of Recurrent Neural Network(RNN).At the post-processing step,the generated Pseudo-code is optimized.The proposed model is assessed using a real Python dataset,which contains more than 18,800 lines of a source code written in Python.The experiments show promising performance results compared with other machine translation methods such as Recurrent Neural Network(RNN).The proposed DLBT records 47.32,68.49 accuracy and BLEU performance measures,respectively.

Design and Implementation of Speech Generation and Demonstration Research Based on Deep Learning

Aiming at complex and changeable factors such as speech theme and environment,which make it difficult for a speaker to prepare the speech text in a short time,this paper proposes a speech generation and demonstration system based on deep learning.This system is based on the Deep Learning Development Framework(PyTorch),trained through the theory of GPT-2 and the open source pretrained model,to generate multiple speeches according to the topics given by users,and the system generates thefinal speech and corresponding voice demon-stration audio through text modification,speech synthesis and other technologies to help users quickly obtain the target document and audio.Experiments show that the text generated by this model is smooth and easy to use,which helps shorten the preparation time of speakers and improves the confidence of the impromptu speaker.In addition,the paper explores the application prospects of text generation and has certain reference value.

Recent advances of neural text generation:Core tasks,datasets,models and challenges

In recent years,deep neural network has achieved great success in solving many natural language processing tasks.Particularly,substantial progress has been made on neural text generation,which takes the linguistic and non-linguistic input,and generates natural language text.This survey aims to provide an up-to-date synthesis of core tasks in neural text generation and the architectures adopted to handle these tasks,and draw attention to the challenges in neural text generation.We first outline the mainstream neural text generation frameworks,and then introduce datasets,advanced models and challenges of four core text generation tasks in detail,including AMR-to-text generation,data-to-text generation,and two text-to-text generation tasks(i.e.,text summarization and paraphrase generation).Finally,we present future research directions for neural text generation.This survey can be used as a guide and reference for researchers and practitioners in this area.

Generating Factual Text via Entailment Recognition Task

Generating diverse and factual text is challenging and is receiving increasing attention.By sampling from the latent space,variational autoencoder-based models have recently enhanced the diversity of generated text.However,existing research predominantly depends on summarizationmodels to offer paragraph-level semantic information for enhancing factual correctness.The challenge lies in effectively generating factual text using sentence-level variational autoencoder-based models.In this paper,a novel model called fact-aware conditional variational autoencoder is proposed to balance the factual correctness and diversity of generated text.Specifically,our model encodes the input sentences and uses them as facts to build a conditional variational autoencoder network.By training a conditional variational autoencoder network,the model is enabled to generate text based on input facts.Building upon this foundation,the input text is passed to the discriminator along with the generated text.By employing adversarial training,the model is encouraged to generate text that is indistinguishable to the discriminator,thereby enhancing the quality of the generated text.To further improve the factual correctness,inspired by the natural language inference system,the entailment recognition task is introduced to be trained together with the discriminator via multi-task learning.Moreover,based on the entailment recognition results,a penalty term is further proposed to reconstruct the loss of our model,forcing the generator to generate text consistent with the facts.Experimental results demonstrate that compared with competitivemodels,ourmodel has achieved substantial improvements in both the quality and factual correctness of the text,despite only sacrificing a small amount of diversity.Furthermore,when considering a comprehensive evaluation of diversity and quality metrics,our model has also demonstrated the best performance.

Recent advances and challenges in task-oriented dialog systems

Due to the significance and value in human-computer interaction and natural language processing,task-oriented dialog systems are attracting more and more attention in both academic and industrial communities.In this paper,we survey recent advances and challenges in task-oriented dialog systems.We also discuss three critical topics for task-oriented dialog systems:(1)improving data efficiency to facilitate dialog modeling in low-resource settings,(2)modeling multi-turn dynamics for dialog policy learning to achieve better task-completion performance,and(3)integrating domain ontology knowledge into the dialog model.Besides,we review the recent progresses in dialog evaluation and some widely-used corpora.We believe that this survey,though incomplete,can shed a light on future research in task-oriented dialog systems.