面向任务型对话机器人的多任务联合模型研究

Research on Multi-task Jointing Model for Task Chat Robot

在任务型对话机器人的搭建过程中,一般需要执行多个自然语言处理的子任务。目前传统的训练方式是将每个子任务独立训练后再进行整合,这样忽视了不同子任务之间的关联性,限制了模型的预测能力。现提出一种Joint-RoBERTa-WWM-of-Theseus压缩联合模型,一方面通过多任务联合学习训练的方式对意图识别、行业识别和语义槽填充3个子任务进行联合训练,并在多分类的子任务中引入Focal loss机制来解决数据分布不平衡的问题;另一方面,模型通过Theseus方法进行压缩,在略微损失精度的前提下,大幅提高模型预测速度,提高模型在生产环境中的实时性与实用性。

In the process of building a task-oriented chatbot,it is generally necessary to execute several subtasks of Natural Language Processing.And the traditional training method is to integrate each subtask after training independently,which will ignore the relevance between different subtasks and limit the predictive power of the model.This paper proposes a compressed jointed model,i.e.,Joint-RoBERTa-WWM-of-Theseus.On the one hand,intention classification,domain classification and semantic slot filling are jointly trained through multi-task joint learning and training.And the focal loss mechanism is introduced to the multi-class classification subtask to solve the problem of data distribution imbalance.On the other hand,the model is compressed by means of Theseus compression method,which greatly improves the prediction speed of the model and improves the applicability and the real-time in the production environment with a slight loss of accuracy.