多标记众包学习

Multi-label Crowdsourcing Learning

传统的多标记学习任务要求训练数据拥有完整的或者至少部分的真实标记,而真实标记耗费昂贵并且难以获取.不同于由昂贵受限的专家标注真实标记,众包环境下,多标记任务被分配给多个容易获取的非专家标注,学习目标是从有错误的非专家标注中估计样本的真实标记.这一问题的关键在于如何融合非专家标注.以往的众包学习主要集中在单标记任务上,忽视了多标记任务的标记相关性;而多标记任务上的众包工作集中在局部标记相关性的利用如标记共同出现的概率,标记间条件相关性,其估计很敏感地受到标记数量和质量的影响.考虑到多标记任务上多个标注者的标注结果整体上存在低秩结构关系,提出一种基于低秩张量矫正的方法.首先,将标注结果组织成三维的张量(样本,标记,标注者),用低秩张量补全的方法对收集到的标注做预处理,以同时达到两个目的:1)优化已有标注;2)补全标注者在其未标注的标记上的标注结果.然后,对所有标注融合,测试了3种融合方法,分别从不同的方面考虑标注的置信度.真实数据上的实验结果验证了所提方法的有效性.

Previous multi-label learning requires that all or at least a subset of ground truth labels is given for the training example. This study investigates how to utilize the wisdom of crowds for multi-label tasks, where rather than high cost ground truth labels, imperfect annotations from crowds are collected for learning. The target is to infer the instances’ ground truth labels. The key challenge lies in how to aggregate the noisy annotations. Different from previous crowdsourcing works on single-label problems which ignore the correlation between labels, and multi-label works which consider local label correlations whose estimation heavily depends on the annotations’ quality and quantity, this study proposes an approach considering the global low rank structure of the crowds’ annotations. Regarding the crowds’ annotations for multi-label tasks as a three-way tensor(instance, label, worker), the crowds’ annotations are firstly preconditioned using low rank tensor completion, such that it is able to simultaneously correct the observed noisy annotations and at the same time predict the unobserved annotations. After that, the preconditioned annotations are aggregated through some heuristic methods. Three aggregation methods taking into account the crowds’ annotation confidence from different perspectives are tested. Experimental results on real world multi-label crowdsourcing data sets demonstrate the superiority of the proposed approach.