基于时间感知注意力机制的混合编码网络方法

The method of hybrid code networks based on time-aware attention mechanism

传统的混合编码网络在小样本数据训练情况下,捕捉用户意图与语义分析方面存在局限性,很难应用到新领域进行迁移训练。时间感知注意混合编码网络(time-aware attention hybrid code networks, TAA-HCN)通过构建时间感知的注意力机制和用户意图集成(user intent integration, UII)的门控机制建模用户意图与动作措施的关系,捕捉用户意图随时间动态变化,结合元学习的思想进行模型梯度自适应,以便模型快速收敛。TAA-HCN模型在WOZ数据集与BABI数据集上进行试验与分析,当目标域数据为总数据的5%时,F1与BLEU指标几乎全收敛,且准确率为69.3%,这表明了本研究的模型具有仅需很少的目标数据即可实现良好性能的能力。

The traditional hybrid code networkshas limitations in capturing user intent and semantic analysis in the case of small sample data training, so it was challenging to apply to new fields for transfer training.The time-awareattention hybrid code networks(TAA-HCN)with enhanced time-aware attention mechanism modelled the relationship between user intent and the action measures by constructing the time-aware attention mechanism and the gating mechanism of user intent integration(UII), to capture the dynamic changes of user intentions over time, the model gradient adaptation was combined with the idea of meta-learning, so that the model could converge quickly. The TAA-HCN model was tested and analyzed on the WOZ data set and the BABI data set. When the target domain data was 5% of the total data, themetrics of F1 and the BLEU almost fully converged, and the accuracy rate was 69.3%. It was shown that the model in this study had the ability to achieve good performance with very little target data.