基于模型分解的小样本学习

Few-shot learning via model composition

机器学习依赖大量样本的统计信息进行模型的训练,从而能对未知样本进行精准的预测.搜集样本及标记需要耗费大量的资源,因而如何基于少量样本(few-shot learning)进行模型的训练至关重要.有效的模型先验(prior)能够降低模型训练对样本的需求.本文基于元学习(meta learning)框架,从相关的、类别不同的数据中学习模型先验,并将这种先验应用于新类别的少样本任务.与此同时,本文提出"模型组合先验"(MCP,model composition prior)方法,通过目标函数的最优条件对模型结构进行分解,并分别估计模型的各个组成部分,得到有效的分类器.这种分解方式具有较高的可解释性,能够指导在不同小样本任务中"共享"与"独立"的成分,从而指导元学习的具体实现.在人造数据中,本文方法能够恢复出小样本任务之间的关联性;在图像数据上,MCP方法能取得比当前主流方法更优异的效果.

Although achieve inspiring performance in many real-world applications,machine learning methods require a huge amount of training examples to obtain an effective model.Considering the effort collecting labeled training data,the few-shot learning,i.e.,learning with budgeted training set,is necessary and useful.Model prior,e.g.,the feature embedding,initialization,and configuration,is the key to the few-shot learning.This study metalearns such prior from seen classes and apply the learned prior over few-shot task on unseen classes.Meanwhile,based on the first order optimal condition of the objective,the model composition prior(MCP)is stressed to decompose the model prior and estimate each component.The composition strategy improves the explainability,while guiding the shared and specific parts among those few-shot tasks.We verify the ability of our approach to recover task relationship over the synthetic dataset,and our MCP method achieves better results on two benchmark datasets(MiniImageNet and CUB).