基于风险分析的回归测试用例优先级排序

Risk Analysis-Based Test Case Prioritization for Regression Testing

该文利用软件组件间信息流的传递过程,提出了基于风险分析的回归测试用例优先级排序算法(Risk Analysis-based Test Case Prioritization,RA-TCP).该算法针对现有的优先级排序技术未能有效利用测试用例所覆盖信息的问题,在类粒度下将软件抽象为基于信息流的类级有向网络模型,然后将每个测试用例所覆盖的类间信息传递关系用一组杠铃模型表示,结合概率风险评估方法和故障树理论计算杠铃模型的风险值,最后以测试用例所覆盖的杠铃风险总和作为其排序依据.实验结果表明,风险越高的测试用例覆盖错误的可能性越大,RA-TCP算法提高了具有严重风险的错误发现速率,与7种排序算法对比,RA-TCP算法具有较高的错误检出率及较好的稳定性.

The target of regression testing is to ensure that the modified program still satisfies its requirements.Due to the evolution of a software system,software engineers should re-run the existing test cases to detect whether new faults have been introduced into previously tested code.To improve the effectiveness of regression testing,researchers have studied various techniques including regression test selection,test suite minimization and test case prioritization.Test case prioritization techniques allow testers to schedule their test cases so that those test cases with the highest priority,according to some criterion,are executed earlier in the regression testing process than lower priority test cases.Increasing a test suite’s rate of fault detection is considered as one potential goal of test case prioritization,which is a measurement of how quickly a test suite detects faults during the testing process.An improved rate of fault detection can provide earlier feedback on the system under regression test,and let developers begin debugging earlier than might otherwise be possible.In this study,we present a Risk Analysis-based Test Case Prioritization(RA-TCP)algorithm based on the transmission of information flows among software components.The existing works didn’t effectively use the information covered by test cases to assign their priorities.To address this problem,the proposed algorithm firstly abstracts software as an information flow-based directed network model under classes.Then,class relationships covered by each test case are represented by a set of barbell motifs.On this basis,the execution probabilities,vulnerabilities and failure consequences of nodes and edges in the software network are calculated using risk analysis theory.Furthermore,to quantitatively analyze the risk coverage of test cases,we decompose the functional paths covered by test cases into a series of barbell motifs.With the aid of fault tree model,we provide the formal semantics of causal chain of the system failure.Finally,we assign a priority to each test case by calculating the sum of risk indexes of all the barbell motifs covered by it.Assigning higher priority to the critical test cases helps to detect the severer faults in the early testing steps,thereby the system reliability and test efficiency are improved.Four open source software are adopted as our experimental subjects to verify the validity of the proposed approach.The experimental results show that for these systems,the risk coverage distribution of test cases conforms to the Pareto principle,i.e.,80 percent of critical codes in the system are covered by 20 percent of test cases.Comparing with the other state-of-the-art prioritization algorithms,RA-TCP technique has a higher error detection rate and performs stable across different systems.The prioritization techniques based on historical code change information are more appropriate to regression testing,while RA-TCP technique which based on the complexity metric can be applied to solve the non-regression testing problem.In addition,RA-TCP technique can maximize the system risk reduction rate during the testing process.This advantage will be more obvious if the test case suite covers more high-risk barbell motifs of the system.