基于双向自举蒸馏的异质云-端医疗对话联邦

Heterogeneous cloud-end medical dialogue federation based on bi-directional bootstrapping distillation

医疗对话场景下的数据/模型异质、数据类型不同,为此提出新的联邦学习方法.云模型和端模型以相互自举蒸馏的方式进行知识递进传递.端到云的自举蒸馏过程为多教师-单学生模式,知识被从多个局部模型蒸馏统一到全局模型;云到端的自举蒸馏过程为单教师-多学生模式,知识被从全局模型蒸馏回多个局部模型.在医疗对话ReMeDi和MedDG数据集上,所提方法与经典基线相比通过文本生成指标评价获得了显著提高,训练速度有所提升.

A new federated learning method was proposed in the medical dialogue scene for the heterogeneous data/models and different types of data.The cloud model and the end model transferred knowledge by mutual bootstrapping distillation.The end-to-cloud bootstrapping distillation process was a multi-teacher-single-student model,and knowledge was distilled from multiple local models to a global model.The cloud-to-end bootstrapping distillation process was a single-teacher-multi-student model,and knowledge was distilled from the global model back to multiple local models.On the medical dialogue ReMeDi and MedDG data sets,the proposed method is significantly improved compared with the classical baseline by the text generation evaluation criterion,and the training speed has also been improved.