对软件工程中经验研究的调查

Investigation of Empirical Researches in Software Engineering

为了描述、理解、评估、预测、控制、管理或者改善与软件相关的内容,研究者常常使用经验研究的方法.经验研究在软件工程领域已经得到广泛的应用并备受关注.为了了解近年来软件工程中经验研究的特点,并希望经验研究方法为更多研究者所了解,通过系统映射的方法,对软件工程中经验研究的典型期刊《Empirical Software Engineering》(ESE)近5年的论文做了调研,搜集了2013年1月~2017年6月发表在该期刊的250篇论文.通过定性和定量的分析,给出了软件工程领域采用经验研究的主要目的、常用的经验研究方法以及这些方法在软件工程各个领域中的使用情况和呈现的一些新特征.之后,分析了经验研究的主要数据来源、采集手段、常用的数理统计方法以及开源项目在经验研究中的使用情况等,给出了研究者对有效性和可重现性问题的关心程度.最后进行了有效性分析,并进一步探讨了经验研究的发展方向和大数据时代下经验研究面临的机遇和一些开放性问题.

To depict, understand, evaluate, predict, control, manage or enhance software-related artifacts, researchers and practitioners often rely on empirical methods. Empirical methods have been widely used in software engineering, and they are attracting increasing attention over the years. By conducting a systematic mapping, this paper aims to provide a literature survey of 250 papers published in a typical journal--Empirical Software Engineering, from January 2013 to June 2017. With qualitative and quantitative analysis, this survey reveals the commonly used empirical research methods, research purposes, and the application of the methods in subfields of software engineering, including the solved problems and some new features. The findings also cover the use of open source projects, data source, data collection methods and commonly used mathematical statistics methods. Finally, this paper illustrates validity threats and discusses the future work, opportunity and some open issues of empirical research in the era of big data.