基于多头注意力机制融合常识知识的共情对话生成

Empathetic Dialogue Generation by Incorporating Commonsense Knowledge Based on Multi-Head Attention Mechanism

当前对话生成技术已取得显著进展,然而为了更好地满足人类交流的需求,研究人员开始将共情引入对话生成领域。共情作为人际交流的重要组成部分,有助于更好地理解他人的情感和感受。通过引入常识知识来加强对用户情感和处境的理解,然而目前方法对非情感知识采用统一编码以及对常识知识融合采用简单的向量拼接,导致某些常识知识特征的影响可能降低,并且各个知识之间的关联刻画模糊。为此,提出一种基于多头注意力机制融合常识知识的共情对话模型ATT-EDM。该模型对引入的5种关系(xReact、xWant、xNeed、xIntent、xEffect)进行单独编码,保留了每种常识各自的特征,并利用多头注意力机制融合常识知识,对每种知识在注意力层进行运算,以更准确地反映它们各自的影响,同时更有效地刻画各个常识知识之间的联系。在数据集EmpatheticDialogues上的实验结果表明,该模型在困惑度(PPL)、准确率和Distinct-2指标上优于基线模型,PPL降低到36.4350,准确率和Distinct-2分别达到37.96%、3.3455,能够生成质量更高、内容更丰富、共情能力更强的同理心响应。

Significant progress has been made in dialogue generation techniques.However,to meet the demands of human communication more effectively,researchers have incorporated empathy into dialogue generation.Empathy,a crucial component of interpersonal communication,aids in enhanced understanding of others'emotions and feelings.The latest methods realize this enhanced understanding by introducing commonsense knowledge through the use of unified encoding for non-emotional knowledge and employing a simple vector concatenation for integrating commonsense knowledge.However,this approach may potentially decrease the influence of certain commonsense knowledge features and also lacks clear representations of the interrelations among various knowledge components.An empathetic dialogue model ATT-EDM is proposed to address these limitations,which leverages the multi-head attention mechanism to effectively fuse commonsense knowledge.The model individually encodes the five introduced relationships-xReact,xWant,xNeed,xIntent,and xEffect,preserving the distinctive features of each commonsense knowledge.It leverages a multi-head attention mechanism to integrate knowledge,computes each type of knowledge separately in the attention layer to reflect their respective influences more accurately,and effectively captures the interconnections among various commonsense knowledge components.Experimental results obtained using the EmpatheticDialogues dataset demonstrate that the proposed model outperforms baseline models in terms of the Perplexity(PPL),accuracy,and Distinct-2 metrics.Specifically,it achieves a reduced PPL of 36.4350,with the accuracy and Distinct-2 metrics reaching 37.96%and 3.3455,respectively.This enables the generation of high quality responses that are content-rich and empathetic.