一种基于损失预测的双主动域适应算法研究

A Dual Active Domain Adaptation Algorithm Based on Loss Prediction Strategy

近年来深度学习在图像分类任务上取得了显著效果,但通常要求大量人工标记数据,模型训练成本很高.因此,领域自适应等小样本学习方法成为当前研究热点.通常,域适应方法利用源域的经验知识也仅能一定程度降低对目标域标记数据的依赖,因此可以引入主动学习方法对样本价值进行评估并做筛选,从而进一步降低标记成本.本文将典型样本价值估计模型引入域适应学习,结合特征迁移思路,提出了双主动域适应学习算法D_AcT(Dual active domain adaptation).该算法同时对源域与目标域数据进行价值度量,并挑选最具训练价值的样本,在保证模型精度的前提下,大幅度减少了模型对标签数据的需求.具体而言,首先利用极大极小熵和核心集采样方法,用主动学习价值评估模型挑选目标域样本,得到单主动域适应算法S_AcT(Single active domain adaptation).随后利用损失预测策略,将价值评估策略适配至源域,进一步提升迁移学习知识复用有效性,降低模型训练成本.本文在常用的四个图像迁移数据集进行了测试,将所提两个算法和传统主动迁移学习及半监督迁移学习算法进行了实验对比.结果表明双主动域适应方法所需标记源域数据可减少50%以上,且准确率较传统方法最大提升了4%.系列实验验证了本文所提方法的可行性和有效性.

Deep learning has made remarkable achievements in image classification tasks and various applications in recent years.However,most of the deep learning models require a large amount of labeled data in the training process because of deep structures and numerous parameters.This results in a high labeling cost in deep learning model training.To address this issue,various few-shot learning strategies have been proposed and attracted much attention recently.In which,the domain adaptation and active learning are two of the most widely studied methods.The concept of domain adaptation is to use the empirical knowledge in source domains to reduce the label requirement in target domains,while the active learning reduces labeling cost by evaluating the valuable unlabeled samples for the current model to avoid redundant labeling.Although there are a lot of achievements in both of domain adaptation and active learning fields that demonstrate their effect in reducing deep learning training cost,but most of the existing methods are only focus on one field.To further reduce the labeling cost and leverage the advantage of both knowledge reusing and sample evaluating,we propose a Dual Active Domain Adaptation(D_AcT)algorithm in this paper.It is motivated by the phenomenon that not all source domain sam ples are useful in the knowledge transfer learning.In the D_AcT algorithm,the domain adapta tion learning is combined with a typical sample value estimation model to filter the redundant or even opposite-effect samples.The algorithm simultaneously measures the value of the source and target data to select the most valuable samples for training,which further reduce the labeling cost.Specifically,we first propose a Single Active Domain Adaptation(S_AcT)algorithm to se lect the target domain samples.It uses active learning strategy that combines the Minimax Entro py(MME)and the core set model.The Minimax Entropy is used to train feature extractors by minimizing a cross entropy loss on source and target domain samples.The core set model is con structed based on the feature selection diversity.Then,the D_AcT algorithm is proposed by u sing a loss prediction module.It minimizes the difference between the predicted and actual loss to further enhance the effectiveness of source knowledge reusing and reduce the model training cost.To evaluate the performance of the proposed methods,we conduct comprehensive experiments that compare our method with the existing active transfer learning and semi-supervised transfer learning algorithms.The proposed methods are tested on four commonly used transfer learning image datasets including the Office31,the Mixed National Institute of Standards and Technology database(MNIST),the Street View House Number(SVHN)and the SubDomainNet.The ex perimental results show that the S_AcT method improves the accuracy up to 3.8%compared with the conventional active transfer learning methods and up to 1.6%compared with semi-su pervised transfer learning method.The proposed D_AcT method reduces the source domain la bels by more than 50%and improve the accuracy by up to 4%compared with the existing active transfer learning methods,which demonstrates the superiority and effectiveness of the proposed methods.