移动应用众包测试报告自动化评估算法设计

Design of Automated Evaluation Algorithm for Mobile Application Crowdsourcing Test Report

合理的评估机制能帮助众包测试商业平台客观衡量众包测试工作者测试任务完成质量,对众包测试平台有重要商业意义。众包测试的在线自由任务模式,使得如何识别欺骗类型工作者,有效评价任务完成质量成为目前众包研究中急需解决的问题。鉴于此,提出一种对移动应用众包测试报告自动化评分的CTRAEA算法,利用过滤规则对无效测试报告进行剔除,通过MMDBK聚类算法,结合报告提交者历史可信度,按权重计算确定缺陷等级,再对报告描述规范性构建度量指标及离散型度量函数,并计算出测试报告规范性评分,最终加权求和两方面评分以衡量工作者任务完成质量。实验结果平均相对误差为9.24%,证明评估算法准确性较高。

A reasonable evaluation mechanism can help the crowdsourcing test business platform objectively measure the quality of the completion of test tasks by the crowdsourcing test workers,which is of great commercial significance to the platform.Due to the online free task mode of crowdsourcing test,how to identify cheat workers and evaluate task completion quality effectively has become an urgent problem to be solved in crowdsourcing research.Aiming at the problem in the mobile applications crowdsourcing test,this paper proposes a CTRAEA algorithm for scoring the crowdsourcing test report.Firstly,the filtering rule is used to eliminate the invalid test report.The defect level was determined by weight calculation through MMDBK clustering algorithm combined with the credibility of the report submitter.And then the normative metrics of report description and the discrete metric function are put forward to calculate the normative score of test report.Finally,the quality of the task completion of the worker is measured by making weighted summation of the two score.The average absolute error of the experimental results is 0.83690,which proves the accuracy of the evaluation algorithm.