基于联合分布核适配的迁移学习及其隐私保护

Transfer learning based on joint distribution kernel adaptation and its privacy protection

迁移学习利用不同但相关的源域标记数据来解决目标领域的学习问题,大多数减小域间分布差异的方法依赖于最大均值差异距离,但其仅仅能匹配域间数据分布的各阶矩.此外,隐私保护意识的增强限制了对数据源的访问,对迁移学习的发展提出了新的挑战.本文提出一种基于联合分布核适配的迁移学习及其隐私保护方法,直接在再生核希尔伯特空间中同时减小域间边缘分布和条件分布的差异,从而学习一个域不变核矩阵.此外,我们设置数据源双方首先访问一个相同的随机投影函数,然后聚合器发布基于目标扰动的差分隐私核分类器,在实现基于核的联合分布适配的同时,避免了数据源与聚合器直接共享原始特征数据.在多个文本和图像迁移学习基准数据集上进行了对比实验和参数分析,结果显示本文方法具有良好的有效性.

Transfer learning solves a learning problem in a target domain by utilizing the labeled data in a different but related source domain. Most prior methods to reduce the distribution difference between domains depend on the maximum mean discrepancy(MMD) distance, but MMD can only match the moments of data distributions between domains. In addition, the increasing privacy protection awareness restricts access to data sources and poses new challenges for the development of transfer learning. This paper proposes a transfer learning method based on joint distribution kernel adaptation and puts forward its privacy protection approach. We learn a domain-invariant kernel by directly matching both the marginal distribution and conditional distribution in the reproducing kernel Hilbert space. Besides, both data sources are designed to access the same random projection function firstly, then the aggregator is set to release a differential privacy kernel classifier based on objective perturbation. While implementing kernel-based joint distribution adaptation, it avoids the data source and the aggregator from directly sharing the original feature data. Comparative experiments and parameter analysis on multiple text and image transfer benchmark data sets verify the effectiveness of the proposed method.