4CHRISTIAN TERWIESCH, YI XU. Innovation contests, open innovation, and multiagent problem solving[J]. Man- agement Science,2008,54(9) :1529-1543.
5Howe J.The Rise of Crowdsourcing[J].Wired Magazine,2006,14(6):1-4.
6Difallah D E,Demartini G,Cudré-Mauroux P.Mechanical Cheat:Spamming Schemes and Adversarial Techniques on Crowdsourcing Platforms[C]//Proceedings of the1st International Workshop on Crowdsourcing Web Search.Lyon,France:[s.n.],2012:26-30.
7Raykar V C,Yu Shipeng.Eliminating Spammers and Ranking Annotators for Crowdsourced Labeling Tasks[J].Journal of Machine Learning Research,2012,13(1):491-518.
8Ross J,Irani L,Silberman M,et al.Who Are the Crowdlabelers?:Shifting Demographics in Mechanical Turk[C]//Proceedings of the 28th International Conference on Human Factors in Computing Systems.New York,USA:ACM Press,2010:2863-2872.
9Eickhoff C,Vries A P.Increasing Cheat Robustness of Crowdsourcing Tasks[J].Information Retrieval,2013,16(2):121-137.
10Kazai G,Kamps J,Koolen M,et al.Crowdsourcing for Book Search Evaluation:Impact of Hit Design on Comparative System Ranking[C]//Proceedings of the34th International ACM SIGIR Conference on Research and Development in Information Retrieval.New York,USA:ACM Press,2011:205-214.
