基于小样本知识蒸馏的乳腺癌组织病理图像分类

Histopathological image classification of breast cancer based on few-shot knowledge distillation

目的:研究一种用于乳腺癌病理图像分类的小样本知识蒸馏方法。方法:采用一种双级蒸馏策略来实现小样本环境下的知识蒸馏。在多教师知识蒸馏中提出一种简单有效的多教师软目标整合方法,以提高优秀教师网络在知识蒸馏中的指导作用。第一步,将学生区块逐一嫁接到教师网络身上,并与其他教师区块交织在一起训练,训练过程只更新嫁接区块的参数。第二步,将训练好的学生区块依次嫁接到教师网络上,让习得的学生区块相互适应,最终取代教师网络,得到一个更轻量化的网络结构。结果:得到一个优秀的轻量化网络用于分类乳腺癌组织病理图像。在BreaKHis数据集上的实验结果表明,通过基于小样本的双级蒸馏策略成功地将教师网络的知识传递到学生网络,并获得了与教师网络几乎相当的决策性能,且其网络结构更加轻量化。结论:这种嫁接策略,可以更好地利用多个教师模型中训练有素的参数,并且还可以显著缩小学生模型的参数空间。

Aims:To study a small sample knowledge distillation method for classification of breast cancer pathological images.Methods:A two-stage distillation strategy was used to achieve knowledge distillation in a small-sample environment.A simple and effective multi-teacher soft target integration method was proposed in multi-teacher knowledge distillation to improve the guiding role of excellent teacher network in knowledge distillation.In the first step,the student blocks were grafted one by one onto the teacher network and intertwined with other teacher blocks for training,and the training process only updates the parameters of the grafted blocks.In the second step,the trained student blocks were grafted onto the teacher network in turn,so that the learned student blocks adapt to each other and finally replace the teacher network to obtain a lighter network structure.Results:An excellent lightweight network was obtained for classifying breast cancer histopathological images.The experimental results on the BreaKHis data set show that the knowledge of the teacher network is successfully transmitted to the student network through a two-stage distillation strategy based on small sample,and almost equivalent decision performance is obtained by the teacher network,and its network structure is more lightweight.Conclusions:This grafting strategy can make better use of the well-trained parameters in multiple teacher models,and can also significantly narrow the parameter space of the student model.