基于语境交互感知和模式筛选的隐式篇章关系识别

Implicit Discourse Relation Recognition Based on Contextual Interaction Perception and Pattern Filtering

隐式篇章关系识别是篇章分析(Discourse Analysis)中一项具有挑战性的子任务,旨在推断出同一篇章内相邻文本片段(称为论元)之间潜在的语义连接关系,例如:时序关系、因果关系等.如何有效地表征篇章论元以及挖掘论元间的交互信息是实现该任务的核心要素.传统研究注重篇章中人工总结的表层语言特征(即情感词极性、位置特征和动词类型等),存在数据稀疏和预处理错误级联的问题,导致机器学习模型性能不高.新近的深度神经网络模型则自动提取篇章论元中的特征,利用注意力或记忆机制等捕获论元的重要信息,并组合不同神经网络提取大量关系特征,以提升模型识别性能.然而,其忽略了表示过程中论元间双向非对称的交互信息,以及识别过程中论元间交互模式的稀疏特性.受认知学相关理论的启发,本文提出基于语境交互感知和模式筛选的隐式篇章关系识别方法(MATS).首先,通过双向长短期记忆网络(Bidirectional Long Short-Term Memory,BiLSTM)分别编码两个论元,以获取带有上下文语境信息的论元表示;其次建模其动态交互注意力机制,以自动学习论元之间的非对称关联矩阵,进而得到融合语境交互感知信息的论元表示;最后,利用带有稀疏约束的张量神经网络捕捉具有篇章关系指示性的深层交互模式,从而提升模型的识别性能.Penn Discourse Treebank(PDTB)语料库上的实验结果表明,本文提出方法的精确率在其四分类上改善了2.36%.

Recognizing implicit discourse relation is a challenging sub-task in discourse analysis,which aims to understand and annotate the latent relations between two adjacent text units(called discourse arguments),such as temporal,comparison,etc.It is beneficial to many downstream natural language processing(NLP)applications,e.g.,machine translation,information retrieval,conversation system and so on.The key factors lie in appropriately representing two discourse arguments as well as modeling their interactions.Traditional methods focus on surface features of discourse(for instance,position feature,sentiment polarity,syntactic pattern,and so forth.),rule-based method and statistical algorithms.These approaches heavily rely on hand-crafted features to obtain rich linguistic information extracted from two discourse arguments,which usually suffer from data sparsity problem and are prone to error propagation due to the unsuitable preprocessing.Although previous basic neural network-based models could capture the semantic information to some extent by learning dense vector representation,and alleviate the data sparsity problem simultaneously,they model two discourse arguments separately and neglect the pair specific clues.Various complicated neural network architectures with different attention or memory mechanisms are proposed to receive the significant information of the arguments.However,they ignore the bidirectional asymmetry interactions between the arguments in the representation process.Further researches explore to mine more complicated semantic features which may include some redundant and noisy information.These studies rarely consider the sparsity of pair patterns indicating discourse relation in the identification stage.In addition,inspired by relevant cognitive linguistic theories,we have a similar feeling intuitively during judging the discourse relation:a more reasonable strategy is that people may read two arguments back and forth,and find some relevant and indicative clues to make a decision.Therefore,we propose a novel Mutual Attention-based Neural Tensor Network with Sparse Constraint framework(MATS)to excavate the specific indicative pair patterns.Specifically,we model two discourse arguments by basic bidirectional long short-term memory(BiLSTM)network to preserve the contextual information around the word embeddings.In order to catch the pair cues between the arguments,we integrate mutual attention mechanism into BiLSTM for dynamically obtaining the reciprocal information.It is aware of the asymmetry interactions between the two arguments,in a way that the bidirectional information from the arguments can directly influence the computation of their representations.Furthermore,we employ neural tensor network(NTN)model to capture the different aspects of semantic interactions between the arguments.Since there exists some redundant or irrelevant noise affecting the performance of the system,we design a sparse constraint for NTN model in training process,which could remove the noisy features and select the deeper and indicative pair patterns to identify discourse relations efficiently.The experimental results on Penn Discourse Treebank(PDTB)show that our MATS model is effective in implicit discourse relation recognition.It suggests that utilizing mutual attention mechanism to dynamically learn the bidirectional asymmetry connections between the arguments helps improve the semantic representations.Excavating the indicative pair patterns by adding a sparse constraint to NTN also plays an important role in our MATS model.