大数据群体计算中用户主题感知的任务分配

Theme-Aware Task Assignment in Crowd Computing on Big Data

大数据问题所固有的规模繁杂性、高速增长性、形式多样性、价值密度低等特点为传统计算处理方法带来了严峻的挑战.一方面,大数据的规模繁杂性和高速增长性带来了海量计算分析的需求;另一方面,形式多样性和价值密度低等特点使得大数据计算任务高度依赖复杂认知推理技术.针对大数据计算中海量计算分析和复杂认知推理需求并存的技术挑战,传统的基于计算机的算法已经无法满足日益苛刻的数据处理要求,而基于人机协作的群体计算是有效的解决途径.在大数据群体计算中,最基础的就是任务的分配方式.考虑到大量网络用户不同的专业背景、诚信程度,因此不能简单随机地将要处理的任务交给大众来完成.针对此问题,提出了一种基于用户主题感知的迭代式任务分配算法.利用已知答案的测试问题迭代地检测不同人群的专业背景和完成任务的准确率.在充分了解用户真实主题和准确率的情况下为他们分配合适的问题.通过和随机任务分配算法在模拟数据和真实数据上的对比,有效显示了基于主题感知任务分配算法的准确性.

Big data has brought tremendous challenges for the traditional computing model, because of its inherent characteristics such as large volume, high velocity, high variety, low-density value. On the one hand, the large volume and high velocity require the techniques of massive data computation and analysis; on the other hand, the high variety and low-density value make big data computing tasks highly depend on the complex cognitive reasoning technology. To overcome the coexistence challenges of massive data analysis and complex cognitive reasoning, human-machine collaboration based crowd computing is an effective way to solve the big data problem. In crowd computing, task assignment is one of the ＇basic pro^blems. H：owever the current cr0wdSourcing platf0rrns danndt support the active task assignment, which iteratively assigns tasks to appropriate workers based on the knowledge background or users. To address this problem, we propose an iterative theme-aware task assignment framework, and deploy it into existing crowdsourcing platforms. The framework includes two components. The first component is task modeling, which models the tasks as a graph where vertices are tasks and edges are task relationships. The second component is the iterative task assignment algorithm, which identifies the themes of the workers by their historical records, computes the workers accuracy on different themes, and assigns the tasks to the appropriate workers. Various experiments validate the effectiveness of our method.