基于改进Sequence-to-Sequence模型的文本摘要生成方法

基于循环神经网络和注意力机制的Sequence-to-Sequence模型神经网络方法在信息抽取和自动摘要生成方面发挥了重要作用。然而,该方法不能充分利用文本的语言特征信息,且生成结果中存在未登录词问题,从而影响文本摘要的准确性和可读性。为此,利用文本语言特征改善输入的特性,同时引入拷贝机制缓解摘要生成过程未登录词问题。在此基础上,提出基于Sequence-to-Sequence模型的新方法 Copy-Generator模型,以提升文本摘要生成效果。采用中文摘要数据集LCSTS为数据源进行实验,结果表明所提方法能够有效地提高生成摘要的准确率,可应用于自动文本摘要提取任务。

Sequence-To-Sequence Learning for Online Imputation of Sensory Data

Online sensing can provide useful information in monitoring applications,for example,machine health monitoring,structural condition monitoring,environmental monitoring,and many more.Missing data is generally a significant issue in the sensory data that is collected online by sensing systems,which may affect the goals of monitoring programs.In this paper,a sequence-to-sequence learning model based on a recurrent neural network(RNN)architecture is presented.In the proposed method,multivariate time series of the monitored parameters is embedded into the neural network through layer-by-layer encoders where the hidden features of the inputs are adaptively extracted.Afterwards,predictions of the missing data are generated by network decoders,which are one-step-ahead predictive data sequences of the monitored parameters.The prediction performance of the proposed model is validated based on a real-world sensory dataset.The experimental results demonstrate the performance of the proposed RNN-encoder-decoder model with its capability in sequence-to-sequence learning for online imputation of sensory data.

电力物联网边缘计算依赖型任务卸载的低时延调度技术

现有电力物联网任务调度技术难以满足任务的低时延和实时性要求,且未考虑到电力物联网任务之间的内部依赖性。针对该问题,融合深度强化学习任务卸载模型和Sequence-to-Sequence神经网络,使用有向无环图表示任务及依赖关系,引入e-贪婪探索机制和优先经验回放来鼓励探索和提高模型训练效率,构建基于深度强化学习的电力物联网任务卸载模型。通过与其他任务卸载算法进行对比,所提模型的任务平均处理时延显著优于其他算法,验证在电力物联网依赖型任务低时延调度方面的优越性。

Soil NOx Emission Prediction via Recurrent Neural Networks

This paper presents designing sequence-to-sequence recurrent neural network(RNN)architectures for a novel study to predict soil NOx emissions,driven by the imperative of understanding and mitigating environmental impact.The study utilizes data collected by the Environmental Protection Agency(EPA)to develop two distinct RNN predictive models:one built upon the long-short term memory(LSTM)and the other utilizing the gated recurrent unit(GRU).These models are fed with a combination of historical and anticipated air temperature,air moisture,and NOx emissions as inputs to forecast future NOx emissions.Both LSTM and GRU models can capture the intricate pulse patterns inherent in soil NOx emissions.Notably,the GRU model emerges as the superior performer,surpassing the LSTM model in predictive accuracy while demonstrating efficiency by necessitating less training time.Intriguingly,the investigation into varying input features reveals that relying solely on past NOx emissions as input yields satisfactory performance,highlighting the dominant influence of this factor.The study also delves into the impact of altering input series lengths and training data sizes,yielding insights into optimal configurations for enhanced model performance.Importantly,the findings promise to advance our grasp of soil NOx emission dynamics,with implications for environmental management strategies.Looking ahead,the anticipated availability of additional measurements is poised to bolster machine-learning model efficacy.Furthermore,the future study will explore physical-based RNNs,a promising avenue for deeper insights into soil NOx emission prediction.

地图空间形状认知的自编码器深度学习方法

形状是地理空间要素的重要特征,是人们建立空间概念、形成空间认知的重要依据。本文利用深度学习的特征挖掘能力引入自编码学习方法,对二维地图空间中形状边界上多组邻域尺寸下的多个特征进行集成和整合,为空间形状认知的机理和形式化提供支撑。本文以建筑物数据为例,将建筑物形状边界转换为序列数据,并提取其描述特征;随后结合sequence-to-sequence自编码学习模型,对无标签的建筑面要素数据进行学习训练,形成形状认知编码。试验表明,本文方法能够产生符合形状认知、具有相似度计算意义的形状编码,具备对不同建筑物形状的区分能力;同时,在形状检索和匹配等应用场景中,该形状编码能有效地表示建筑物的全局和局部特征,与视觉认知结果一致。

基于语义对齐的生成式文本摘要研究

针对当前生成式文本摘要模型在解码时对摘要整体语义信息利用不充分的问题,提出一种基于语义对齐的神经网络文本摘要方法。该方法以带注意力、Pointer机制和Coverage机制的Sequence-to-Sequence模型为基础,在编码器与解码器之间加入语义对齐网络,实现文本到摘要的语义信息对齐;将获得的摘要整体语义信息与解码器的词汇预测上下文向量进行拼接,使解码器在预测当前词汇时不仅利用已预测词汇序列的部分语义,而且考虑拟预测摘要的整体语义。在中文新闻语料LCSTS上的实验表明,该模型能够有效地提高文本摘要的质量,在字粒度上的实验显示,加入语义对齐机制可以使Rouge_L值提高5.4个百分点。

一种基于BERT的自动文本摘要模型构建方法

针对传统词向量在自动文本摘要过程中因无法对多义词进行有效表征而降低文本摘要准确度和可读性的问题,提出一种基于BERT(Bidirectional Encoder Representations from Transformers)的自动文本摘要模型构建方法。该方法引入BERT预训练语言模型用于增强词向量的语义表示,将生成的词向量输入Seq2Seq模型中进行训练并形成自动文本摘要模型,实现对文本摘要的快速生成。实验结果表明,该模型在Gigaword数据集上能有效地提高生成摘要的准确率和可读性,可用于文本摘要自动生成任务。

一种基于ERNIE的军事文本实体关系抽取模型

针对军事文本实体关系抽取过程中存在的“一句对应多个三元组”,“一个主语对应多个客体”等问题提出一种基于ERNIE的军事文本三元组抽取模型,在编码层引入ERNIE模型获取每个词的编码序列,参考seq-to-seq解码器的建模方法和BIO序列标注,采用先预测主体,再传入主体标注序列预测客体和二者之间关系的方法实现三元组的抽取。在预测层使用sigmoid实现多主体、多客体甚至多关系的提取。实验结果证明,人工标注的军事新闻数据集上,该模型的抽取效果明显优于基于循环神经网络的流水线抽取模型和基于BERT的联合实体关系抽取模型,F1值达到80.04%。

基于TextCNN情感预测器的情感监督聊天机器人

聊天机器人是自然语言处理的热门研究领域之一。现有的生成式聊天机器人一般基于Sequence-to-Sequence模型实现,即使用循环神经网络将问题编码成高维语义向量,再将该向量解码成回复。但是由于解码只使用单一的语义向量,容易生成普适回复。针对上述问题,提出了基于TextCNN情感预测器的情感监督聊天机器人,利用TextCNN情感预测器,由问题直接获得回复的情感表示,在Sequence-to-Sequence模型中引入更准确的情感特征,并通过情感监督方法学习情感表达方式。实验表明该模型能有效地提高聊天机器人的回复质量。

Efficient Image Captioning Based on Vision Transformer Models

Image captioning is an emerging field in machine learning.It refers to the ability to automatically generate a syntactically and semantically meaningful sentence that describes the content of an image.Image captioning requires a complex machine learning process as it involves two sub models:a vision sub-model for extracting object features and a language sub-model that use the extracted features to generate meaningful captions.Attention-based vision transformers models have a great impact in vision field recently.In this paper,we studied the effect of using the vision transformers on the image captioning process by evaluating the use of four different vision transformer models for the vision sub-models of the image captioning The first vision transformers used is DINO(self-distillation with no labels).The second is PVT(Pyramid Vision Transformer)which is a vision transformer that is not using convolutional layers.The third is XCIT(cross-Covariance Image Transformer)which changes the operation in self-attention by focusing on feature dimension instead of token dimensions.The last one is SWIN(Shifted windows),it is a vision transformer which,unlike the other transformers,uses shifted-window in splitting the image.For a deeper evaluation,the four mentioned vision transformers have been tested with their different versions and different configuration,we evaluate the use of DINO model with five different backbones,PVT with two versions:PVT_v1and PVT_v2,one model of XCIT,SWIN transformer.The results show the high effectiveness of using SWIN-transformer within the proposed image captioning model with regard to the other models.

Emotional dialog generation via multiple classifiers based on a generative adversarial network

Background Human-machine dialog generation is an essential topic of research in the field of natural language processing.Generating high-quality,diverse,fluent,and emotional conversation is a challenging task.Based on continuing advancements in artificial intelligence and deep learning,new methods have come to the forefront in recent times.In particular,the end-to-end neural network model provides an extensible conversation generation framework that has the potential to enable machines to understand semantics and automatically generate responses.However,neural network models come with their own set of questions and challenges.The basic conversational model framework tends to produce universal,meaningless,and relatively"safe"answers.Methods Based on generative adversarial networks(GANs),a new emotional dialog generation framework called EMC-GAN is proposed in this study to address the task of emotional dialog generation.The proposed model comprises a generative and three discriminative models.The generator is based on the basic sequence-to-sequence(Seq2Seq)dialog generation model,and the aggregate discriminative model for the overall framework consists of a basic discriminative model,an emotion discriminative model,and a fluency discriminative model.The basic discriminative model distinguishes generated fake sentences from real sentences in the training corpus.The emotion discriminative model evaluates whether the emotion conveyed via the generated dialog agrees with a pre-specified emotion,and directs the generative model to generate dialogs that correspond to the category of the pre-specified emotion.Finally,the fluency discriminative model assigns a score to the fluency of the generated dialog and guides the generator to produce more fluent sentences.Results Based on the experimental results,this study confirms the superiority of the proposed model over similar existing models with respect to emotional accuracy,fluency,and consistency.Conclusions The proposed EMC-GAN model is capable of generating consistent,smooth,and fluent dialog that conveys pre-specified emotions,and exhibits better performance with respect to emotional accuracy,consistency,and fluency compared to its competitors.

A Matching Algorithm with Reinforcement Learning and Decoupling Strategy for Order Dispatching in On-Demand Food Delivery

The on-demand food delivery(OFD)service has gained rapid development in the past decades but meanwhile encounters challenges for further improving operation quality.The order dispatching problem is one of the most concerning issues for the OFD platforms,which refer to dynamically dispatching a large number of orders to riders reasonably in very limited decision time.To solve such a challenging combinatorial optimization problem,an effective matching algorithm is proposed by fusing the reinforcement learning technique and the optimization method.First,to deal with the large-scale complexity,a decoupling method is designed by reducing the matching space between new orders and riders.Second,to overcome the high dynamism and satisfy the stringent requirements on decision time,a reinforcement learning based dispatching heuristic is presented.To be specific,a sequence-to-sequence neural network is constructed based on the problem characteristic to generate an order priority sequence.Besides,a training approach is specially designed to improve learning performance.Furthermore,a greedy heuristic is employed to effectively dispatch new orders according to the order priority sequence.On real-world datasets,numerical experiments are conducted to validate the effectiveness of the proposed algorithm.Statistical results show that the proposed algorithm can effectively solve the problem by improving delivery efficiency and maintaining customer satisfaction.

Paradigm Shift in Natural Language Processing

In the era of deep learning, modeling for most natural language processing (NLP) tasks has converged into several mainstream paradigms. For example, we usually adopt the sequence labeling paradigm to solve a bundle of tasks such as POS-tagging, named entity recognition (NER), and chunking, and adopt the classification paradigm to solve tasks like sentiment analysis. With the rapid progress of pre-trained language models, recent years have witnessed a rising trend of paradigm shift, which is solving one NLP task in a new paradigm by reformulating the task. The paradigm shift has achieved great success on many tasks and is becoming a promising way to improve model performance. Moreover, some of these paradigms have shown great potential to unify a large number of NLP tasks, making it possible to build a single model to handle diverse tasks. In this paper, we review such phenomenon of paradigm shifts in recent years, highlighting several paradigms that have the potential to solve different NLP tasks.