基于多门混合专家网络的情感分析与文本摘要多任务模型

Sentiment analysis and text summarization multi⁃task model based on multi⁃gate mixture⁃of⁃experts network

在目前机器学习应用场景中,大多数方法仍然专注于孤立地学习单个任务,即为每个任务建立一个单独的模型。然而许多现实问题需要多模态的方法来解决,因此需要采用多任务模型。目前多门混合专家网络MMoE在多任务领域取得了不错的效果,然而在针对特定领域的学习仍然存在没有专注于独立任务的信息学习、学习任务之间联系能力不足的问题。为此,文中在多门混合网络专家模型上针对情感分析和文本摘要这一特定领域进行了优化,采用基于解码器的架构针对MMoE的架构进行重构;为解决重新设计架构带来的数据格式和流向变化的问题,同时增加针对任务独有信息的学习,设计了新的门控制网络架构;基于情感分析与文本摘要互助理论,提出两种门控制网络权值修改机制,并通过实验选择性能最佳的机制和参数。最后通过改进前后的性能对比和消融实验,证明了在情感分析和文本摘要领域,所提模型有着更优于MMoE的性能,并且每个优化都对模型性能提升有所贡献。

In current machine learning scenarios,most methods still focus on learning individual tasks in isolation,that is,building a separate model for each task.However,many practical problems need to be solved by multi⁃modal methods,so multi⁃task model is required.At present,the MMoE(multi⁃gate mixture⁃of⁃experts)network has achieved good results in the multi⁃task field,but there are still problems in the learning of specific fields,such as not focusing on the information learning of independent task and insufficient connection ability between learning tasks.To this end,this paper optimizes the specific field of sentiment analysis and text summarization on the MMoE model.The decoder⁃based architecture is used to reconstruct the architecture of the MMoE network.A new gate control network architecture is designed to solve the problem of data format and flow direction change caused by the redesign architecture and the problem of increasing the learning of task⁃specific information.On the basis of the mutual aid theory of sentiment analysis and text summarization,two weight modification mechanisms of gate control networks are proposed,and the mechanisms and parameters with the best performance are selected by experiments.By the performance comparison and ablation experiments before and after improvement,it is proved that the proposed model has better performance than MMoE in the field of sentiment analysis and text summarization,and each optimization contributes to the performance improvement of the model.