一种用于多域对话状态追踪的知识增强与自注意力引导的图神经网络

Knowledge Enhanced and Self-attention Guided Graph Neural Network for Multi-domain Dialogue State Tracking

对话状态追踪是对话系统的重要组成部分,旨在从用户与系统的对话中跟踪用户意图,其通常表示为槽位-槽值对序列.近年来,深度神经网络模型在对话状态追踪问题上取得了较大进展.然而,现有模型在槽位相关性建模方面还存在可拓展性差与易引入噪声等问题.针对上述问题,本文提出了一种知识增强与自注意力引导的图神经网络KESA-GNN(Knowledge-Enhanced&Self-Attention Guided Graph Neural Network).首先,KESA-GNN通过外部知识嵌入增强槽的语义表征提升多头自注意力机制对槽位间相关性的辨别能力.其次,为了精确建模槽位间的诸如共指、共现等相关性,提出了一种自注意力引导的图神经网络建模槽位相关性.该网络采用多头注意力机制获得槽位间的注意力矩阵以及槽位表征,通过Max-N Relation算法获得注意力矩阵中强相关关系集,将稠密的注意力矩阵稀疏化,从而引导图神经网络中强相关槽位间的信息传播,降低无关槽位的噪声影响.最后,KESA-GNN采用门控融合机制过滤槽位多头注意力和图神经网络输出的槽位表征,从而获取更准确的槽位表征向量,进一步提升了KESA-GNN的鲁棒性.在多域对话数据集上的实验结果表明,KESA-GNN模型的性能优于最新的基线模型.

Dialogue state tracking is an important part of dialogue systems,usually represented as a sequence of slot-value pairs.Dialogue state tracking aims to track user intent from the user′s dialogue with the system.In recent years,deep neural networks have made great progress in the problem of dialogue state tracking.However,existing models still have problems such as poor scalability and easy introduction of noise in slot correlation modeling.This paper proposes a knowledge enhancement and self-attention-guided graph neural network(KESA-GNN)in response to the above problems.First,KESA-GNN enhances the semantic representation of slots by embedding external knowledge to improve the ability of multi-head self-attention to discriminate correlations between slots.Secondly,to accurately model the correlation between slots,such as co-reference and co-occurrence,KESA-GNN proposes a self-attention guided graph neural network to model slot correlations.This network is used to obtain the attention matrix between slots and slot representations through a multi-head attention mechanism.Then KESA-GNN obtains the strong correlation set in the attention matrix through the Max-N Relation algorithm.In this way,KESA-GNN can sparse the dense attention matrix and guide information propagation between strongly correlated slots in the graph neural network,reducing the noise arising from irrelevant slots.Finally,KESA-GNN uses a gated fusion mechanism to filter slot representations output by multi-head attention and graph neural network.Therefore,KESA-GNN can obtain a more accurate slot representation and further improves the robustness.Experimental results on the multi-domain dialogue dataset show that the KESA-GNN model outperforms the state-of-the-art baseline models.