集成加权软投票的众包真值推理方法

Ensemble weighted soft voting truth inference method for crowdsourcing

为了提高众包的质量并获取高质量的标注数据,已有许多基于真值推理的众包标注方法被提出。传统真值推理利用多噪声标签作为输入,通过聚合策略推断出真实标签,该过程只考虑了实例的标签而忽略了实例的特征,且忽略了不同工人对不同实例的标注质量。该文引入实例的特征以最大程度地挖掘实例中蕴含的有用信息。通过计算众包实例属于每一个类别的概率,得到新划分的众包数据集;提出一种基于元学习的集成分类器,在新数据集上训练并利用相似度计算得到工人权重,即工人对不同的实例有不同的标注能力;在投票模型的基础上引入工人权重,提出加权软投票的方法用于推测标签。在公开数据集和创建的数据集上的实验结果表明,所提出的方法优于现有的真值推理算法。

Many truth inference methods have been proposed to improve crowdsourcing quality and to obtain high-quality annotated data. Traditional truth inference uses multiple noisy labels as inputs to deduce the real labels through an aggregation strategy. This paper introduces the features of the instances that most effectively mine the useful information contained in the instances. The probability that a crowdsourcing instance belongs to each category is used to divide the crowd-sourcing dataset. An integrated meta-learning classifier is trained on the new dataset to calculate a similarity degree to get worker weights that show each worker’s annotation ability for different instances. Finally, a weighted soft voting method is used to predict the labels. Tests show that this method is superior to existing truth inference algorithms for public and constructed datasets.