C-CORE:Clustering by Code Representation to Prioritize Test Cases in Compiler Testing

Edge devices,due to their limited computational and storage resources,often require the use of compilers for program optimization.Therefore,ensuring the security and reliability of these compilers is of paramount importance in the emerging field of edge AI.One widely used testing method for this purpose is fuzz testing,which detects bugs by inputting random test cases into the target program.However,this process consumes significant time and resources.To improve the efficiency of compiler fuzz testing,it is common practice to utilize test case prioritization techniques.Some researchers use machine learning to predict the code coverage of test cases,aiming to maximize the test capability for the target compiler by increasing the overall predicted coverage of the test cases.Nevertheless,these methods can only forecast the code coverage of the compiler at a specific optimization level,potentially missing many optimization-related bugs.In this paper,we introduce C-CORE(short for Clustering by Code Representation),the first framework to prioritize test cases according to their code representations,which are derived directly from the source codes.This approach avoids being limited to specific compiler states and extends to a broader range of compiler bugs.Specifically,we first train a scaled pre-trained programming language model to capture as many common features as possible from the test cases generated by a fuzzer.Using this pre-trained model,we then train two downstream models:one for predicting the likelihood of triggering a bug and another for identifying code representations associated with bugs.Subsequently,we cluster the test cases according to their code representations and select the highest-scoring test case from each cluster as the high-quality test case.This reduction in redundant testing cases leads to time savings.Comprehensive evaluation results reveal that code representations are better at distinguishing test capabilities,and C-CORE significantly enhances testing efficiency.Across four datasets,C-CORE increases the average of the percentage of faults detected(APFD)value by 0.16 to 0.31 and reduces test time by over 50% in 46% of cases.When compared to the best results from approaches using predicted code coverage,C-CORE improves the APFD value by 1.1% to 12.3% and achieves an overall time-saving of 159.1%.

Modified Harris Hawks Optimization Based Test Case Prioritization for Software Testing

Generally,software testing is considered as a proficient technique to achieve improvement in quality and reliability of the software.But,the quality of test cases has a considerable influence on fault revealing capability of software testing activity.Test Case Prioritization(TCP)remains a challenging issue since prioritizing test cases is unsatisfactory in terms of Average Percentage of Faults Detected(APFD)and time spent upon execution results.TCP ismainly intended to design a collection of test cases that can accomplish early optimization using preferred characteristics.The studies conducted earlier focused on prioritizing the available test cases in accelerating fault detection rate during software testing.In this aspect,the current study designs aModified Harris Hawks Optimization based TCP(MHHO-TCP)technique for software testing.The aim of the proposed MHHO-TCP technique is to maximize APFD and minimize the overall execution time.In addition,MHHO algorithm is designed to boost the exploration and exploitation abilities of conventional HHO algorithm.In order to validate the enhanced efficiency of MHHO-TCP technique,a wide range of simulations was conducted on different benchmark programs and the results were examined under several aspects.The experimental outcomes highlight the improved efficiency of MHHO-TCP technique over recent approaches under different measures.

A hybrid algorithm based on ILP and genetic algorithm for time-aware test case prioritization

To solve the problem of time-awarc test case prioritization,a hybrid algorithm composed of integer linear programming and the genetic algorithm(ILP-GA)is proposed.First,the test case suite which cm maximize the number of covered program entities a d satisfy time constraints is selected by integer linea progamming.Secondly,the individual is encoded according to the cover matrices of entities,and the coverage rate of program entities is used as the fitness function and the genetic algorithm is used to prioritize the selected test cases.Five typical open source projects are selected as benchmark programs.Branch and method are selected as program entities,and time constraint percentages a e 25%and 75%.The experimental results show that the ILP-GA convergence has faster speed and better stability than ILP-additional and IP-total in most cases,which contributes to the detection of software defects as early as possible and reduces the software testing costs.

Value-Based Test Case Prioritization for Regression Testing Using Genetic Algorithms

Test Case Prioritization(TCP)techniques perform better than other regression test optimization techniques including Test Suite Reduction(TSR)and Test Case Selection(TCS).Many TCP techniques are available,and their performance is usually measured through a metric Average Percentage of Fault Detection(APFD).This metric is value-neutral because it only works well when all test cases have the same cost,and all faults have the same severity.Using APFD for performance evaluation of test case orders where test cases cost or faults severity varies is prone to produce false results.Therefore,using the right metric for performance evaluation of TCP techniques is very important to get reliable and correct results.In this paper,two value-based TCP techniques have been introduced using Genetic Algorithm(GA)including Value-Cognizant Fault Detection-Based TCP(VCFDB-TCP)and Value-Cognizant Requirements Coverage-Based TCP(VCRCB-TCP).Two novel value-based performance evaluation metrics are also introduced for value-based TCP including Average Percentage of Fault Detection per value(APFDv)and Average Percentage of Requirements Coverage per value(APRCv).Two case studies are performed to validate proposed techniques and performance evaluation metrics.The proposed GA-based techniques outperformed the existing state-of-the-art TCP techniques including Original Order(OO),Reverse Order(REV-O),Random Order(RO),and Greedy algorithm.

Fault Coverage-Based Test Case Prioritization and Selection Using African Buffalo Optimization

Software needs modifications and requires revisions regularly.Owing to these revisions,retesting software becomes essential to ensure that the enhancements made,have not affected its bug-free functioning.The time and cost incurred in this process,need to be reduced by the method of test case selection and prioritization.It is observed that many nature-inspired techniques are applied in this area.African Buffalo Optimization is one such approach,applied to regression test selection and prioritization.In this paper,the proposed work explains and proves the applicability of the African Buffalo Optimization approach to test case selection and prioritization.The proposed algorithm converges in polynomial time(O(n^(2))).In this paper,the empirical evaluation of applying African Buffalo Optimization for test case prioritization is done on sample data set with multiple iterations.An astounding 62.5%drop in size and a 48.57%drop in the runtime of the original test suite were recorded.The obtained results are compared with Ant Colony Optimization.The comparative analysis indicates that African Buffalo Optimization and Ant Colony Optimization exhibit similar fault detection capabilities(80%),and a reduction in the overall execution time and size of the resultant test suite.The results and analysis,hence,advocate and encourages the use of African Buffalo Optimization in the area of test case selection and prioritization.

Test Case Prioritization in Unit and Integration Testing:A Shuffled-Frog-Leaping Approach

Both unit and integration testing are incredibly crucial for almost any software application because each of them operates a distinct process to examine the product.Due to resource constraints,when software is subjected to modifications,the drastic increase in the count of test cases forces the testers to opt for a test optimization strategy.One such strategy is test case prioritization(TCP).Existing works have propounded various methodologies that re-order the system-level test cases intending to boost either the fault detection capabilities or the coverage efficacy at the earliest.Nonetheless,singularity in objective functions and the lack of dissimilitude among the re-ordered test sequences have degraded the cogency of their approaches.Considering such gaps and scenarios when the meteoric and continuous updations in the software make the intensive unit and integration testing process more fragile,this study has introduced a memetics-inspired methodology for TCP.The proposed structure is first embedded with diverse parameters,and then traditional steps of the shuffled-frog-leaping approach(SFLA)are followed to prioritize the test cases at unit and integration levels.On 5 standard test functions,a comparative analysis is conducted between the established algorithms and the proposed approach,where the latter enhances the coverage rate and fault detection of re-ordered test sets.Investigation results related to the mean average percentage of fault detection(APFD)confirmed that the proposed approach exceeds the memetic,basic multi-walk,PSO,and optimized multi-walk by 21.7%,13.99%,12.24%,and 11.51%,respectively.

Aware Time in Bug Fix—A Novel Automatic Test Case Selection for Prioritization of Version Control

The supreme goal of the Automatic Test case selection techniques is to guarantee systematic coverage, to recognize the usual error forms and to lessen the test of redundancy. It is unfeasible to carry out all the test cases consistently. For this reason, the test cases are picked and prioritize it. The major goal of test case prioritization is to prioritize the test case sequence and finds faults as early as possible to improve the efficiency. Regression testing is used to ensure the validity and the enhancement part of the changed software. In this paper, we propose a new path compression technique (PCUA) for both old version and new version of BPEL dataset. In order to analyze the enhancement part of an application and to find an error in an enhancement part of an application, center of the tree has been calculated. Moreover in the comparative analysis, our proposed PCUA- COT technique is compared with the existing XPFG technique in terms of time consuming and error detection in the path of an enhancement part of BPEL dataset. The experimental results have been shown that our proposed work is better than the existing technique in terms of time consuming and error detection.

An Optimized Test Case Minimization Technique Using Genetic Algorithm for Regression Testing

Regression testing is a widely used approach to confirm the correct functionality of the software in incremental development.The use of test cases makes it easier to test the ripple effect of changed requirements.Rigorous testingmay help in meeting the quality criteria that is based on the conformance to the requirements as given by the intended stakeholders.However,a minimized and prioritized set of test cases may reduce the efforts and time required for testingwhile focusing on the timely delivery of the software application.In this research,a technique named Test Reduce has been presented to get a minimal set of test cases based on high priority to ensure that the web applicationmeets the required quality criteria.A new technique TestReduce is proposed with a blend of genetic algorithm to find an optimized and minimal set of test cases.The ultimate objective associated with this study is to provide a technique that may solve the minimization problem of regression test cases in the case of linked requirements.In this research,the 100-Dollar prioritization approach is used to define the priority of the new requirements.

User Session-Based Test Case Generation and Optimization Using Genetic Algorithm

An approach to generating and optimizing test cases is proposed for Web application testing based on user sessions using genetic algorithm. A large volume of meaningful user sessions are obtained after purging their irrelevant information by analyzing user logs on the Web server. Most of the redundant user sessions are also removed by the reduction process. For test reuse and test concurrency, it divides the user sessions obtained into different groups, each of which is called a test suite, and then prioritizes the test suites and the test cases of each test suite. So, the initial test suites and test cases, and their initial executing sequences are achieved. However, the test scheme generated by the elementary prioritization is not much approximate to the best one. Therefore, genetic algorithm is employed to optimize the results of grouping and prioritization. Meanwhile, an approach to generating new test cases is presented using crossover. The new test cases can detect faults caused by the use of possible conflicting data shared by different users.

Improved metrics for evaluating fault detection efficiency of test suite

By analyzing the average percent of faults detected （APFD） metric and its variant versions, which are widely utilized as metrics to evaluate the fault detection efficiency of the test suite, this paper points out some limitations of the APFD series metrics. These limitations include APFD series metrics having inaccurate physical explanations and being unable to precisely describe the process of fault detection. To avoid the limitations of existing metrics, this paper proposes two improved metrics for evaluating fault detection efficiency of a test suite, including relative-APFD and relative-APFDc. The proposed metrics refer to both the speed of fault detection and the constraint of the testing source. The case study shows that the two proposed metrics can provide much more precise descriptions of the fault detection process and the fault detection efficiency of the test suite.

基于相似性的CITCP强化学习奖励策略

在面向持续集成测试用例优先排序(continuous integration test case prioritization,CITCP)的强化学习方法中,智能体通过对测试用例实施奖励从而调整测试用例优先排序策略以适应后续集成测试,可以满足持续集成测试频繁迭代和快速反馈的需求.智能体通常只奖励执行失效测试用例,但实际工业程序持续集成测试具有集成高频繁但测试低失效的特点,对CITCP的实际应用提出新的挑战.测试低失效,即稀少的执行失效测试用例数量,会导致强化学习中奖励对象稀少,引发强化学习的稀疏奖励问题.本文研究一种强化学习奖励对象选择策略,在奖励执行失效测试用例的基础上,通过选择与执行失效测试用例相似的执行通过测试用例实施奖励,从而增加奖励对象,以解决奖励稀疏问题.研究具体包括,设计了一种测试用例历史执行信息序列和执行时间特征向量表示的相似性度量方法,并基于相似性度量选择与执行失效测试用例集相似的执行通过测试用例集实施奖励.在6个工业数据集上开展了实验研究,结果表明基于相似性的奖励对象选择策略通过增加有效奖励对象解决了稀疏奖励问题,并进一步提高了基于强化学习的持续集成测试用例优先排序质量.

Regression Testing in Developer Environment for Absence of Code Coverage

The techniques of test case prioritization schedule the execution order of test cases to attain respective target, such as enhanced level of forecasting the fault. The requirement of the prioritization can be viewed as the en-route for deriving an order of relation on a given set of test cases which results from regression testing. Alteration of programs between the versions can cause more test cases which may respond differently to following versions of software. In this, a fixed approach to prioritizing test cases avoids the preceding drawbacks. The JUnit test case prioritization techniques operating in the absence of coverage information, differs from existing dynamic coverage-based test case prioritization techniques. Further, the prioritization test cases relying on coverage information were projected from fixed structures relatively other than gathered instrumentation and execution.

A Comparative Evaluation of Test Coverage Techniques Effectiveness

Software systems have become complex and challenging to develop and maintain because of the large size of test cases with increased scalability issues. Test case prioritization methods have been successfully utilized in test case management. However, the prohibitively exorbitant cost of large test cases is now the mainstream in the software industry. The growth of agile test-driven development has increased the expectations for software quality. Yet, our knowledge of when to use various path testing criteria for cost-effectiveness is inadequate due to the inherent complexity in software testing. Existing researches attempted to address the issue without effectively tackling the scalability of large test suites to reduce time in regression testing. In order to provide a more accurate way of fault detection in software projects, we introduced novel coverage criteria, called Incremental Cluster-based test case Prioritization (ICP), and investigated its potentials by making a comparative evaluation with three un-clustered traditional coverage-based criteria: Prime-Path Coverage (PPC), Edge-Pair Coverage (EPC) and Edge Coverage (EC) based on mutation analysis. By clustering test suites, based on their dynamic run-time behavior, the number of pair-wise comparisons is reduced significantly. To compare, we analyzed 20 functions from 25 C programs, instrumented faults into the programs, and used the Mull mutation tool to generate mutants and perform a statistical analysis of the results. The experimental results show that ICP can lead to cost-effective improvements in fault detection.

基于测试树模型的软件测试能力值优先级排序

为了有效提升软件测试用例排序的执行效率,保障软件运行安全,提出基于UML顺序图的软件测试用例优先级排序方法。根据UML顺序图场景概念,通过遍历顺序图中的时间序列获取全部场景,得到与之对应的场景测试树模型。设置约束条件,根据场景环境条件形成软件测试用例。计算各个软件测试用例的迁移重要度,通过反馈机制动态调整软件测试用例的总测试能力值,根据测试能力值展开优先级排序。实验结果表明,采用所提方法可以全面提升软件测试用例缺陷检测率,确保在最短的时间内获取最优的排序结果。

基于深度学习的回归测试用例优先级排序方法

在回归测试中对测试用例排序可以更快地发现代码缺陷,节约测试时间和资源,提高测试效率。现有的测试用例排序方法没有同时考虑代码的变更信息以及测试用例的历史执行信息,也没有考虑不同测试用例执行历史长短的区别,因此排序效果不佳。针对这些问题,提出基于深度学习的回归测试用例优先级排序方法。首先分别构建基于代码变更信息和历史执行信息的分类模型;然后基于类间关系图识别受代码变更影响的类,对这些类的测试用例以及近期执行发现缺陷的测试用例进行分类,使用分类模型和启发式排序方法对测试用例分类进行排序;最后通过交替排序融合排序结果。在RTPTorrent数据集上选取6个项目进行实验,结果表明:1)在无时间约束时,所提方法在所有项目上都取得了不错的排序效果,在cloudify项目上的APFD指标达到0.972;2)在有时间约束时,所提方法的NAPFD指标超过了目前主流的排序方案。

Multi-objective test case prioritization based on multi-population cooperative particle swarm optimization

Test case prioritization(TCP) technique is an efficient approach to improve regression testing activities. With the continuous improvement of industrial testing requirements, traditional single-objective TCP is limited greatly, and multi-objective test case prioritization(MOTCP) technique becomes one of the hot topics in the field of software testing in recent years. Considering the problems of traditional genetic algorithm(GA) and swarm intelligence algorithm in solving MOTCP problems, such as falling into local optimum quickly and weak stability of the algorithm, a MOTCP algorithm based on multi-population cooperative particle swarm optimization(MPPSO) was proposed in this paper. Empirical studies were conducted to study the influence of iteration times on the proposed MOTCP algorithm, and compare the performances of MOTCP based on single-population particle swarm optimization(PSO) and MOTCP based on non-dominated sorting genetic algorithm Ⅱ(NSGA-Ⅱ) with the MOTCP algorithm proposed in this paper. The results of experiments show that the TCP algorithm based on MPPSO has stronger global optimization ability, is not easy to fall into local optimum, and can solve the MOTCP problem better than TCP algorithm based on the single-population PSO and NSGA-Ⅱ.

Test-case prioritization： achievements and challenges

Test-case prioritization, proposed at the end of last century, aims to schedule the execution order of test cases so as to improve test effectiveness. In the past years, test-case prioritization has gained much attention, and has significant achievements in five aspects： prioritization algorithms, coverage criteria, measurement, practical concems involved, and application scenarios. In this article, we will first review the achievements of test-case prioritization from these five as- pects and then give our perspectives on its challenges.

基于I-MOEA/D的多目标测试用例优先级排序

多目标测试用例优先级排序(MOTCP)是回归测试领域中的热门问题,其目的是获得测试用例的执行顺序,最大限度地提高发现缺陷的能力和效率。文章提出一种基于改进MOEA/D算法的多目标测试用例优先级排序方法(I-MOEA/D):首先将多目标测试用例优先级排序问题建模为一个多目标优化问题,然后通过改进MOEA/D算法来解决该优化问题。具体而言:通过引入权重向量自适应策略,以保持子问题之间的多样性;通过位置交叉法对交叉算子进行改进,以加快算法的收敛速度,抵消权重向量计算时间开销;对邻域动态更新,以促进测试用例之间的信息交流和共享。实验结果表明:所提算法在MOTCP方面取得了较好的效果,与其他方法相比,该方法能提高测试用例的发现缺陷能力和效率。

多目标优化的测试用例优先级在线调整策略

从需求的角度对测试用例的优先级进行排序,定义了一个多目标的测试用例优化排序问题,引入关注需求覆盖率、测试用例重要度和测试用例失效率这3个测试用例优先级影响因子,分别定义权重因子?,?,?用于权衡3个因子.设计了关注需求覆盖率和测试用例失效率的在线估计方法及算法,在此基础上,设计了一种基于多目标优化的测试用例优先级在线调整策略,该策略可利用测试过程中收集到的反馈信息,对测试用例优先级进行在线调整,实现在尽早达到测试覆盖率标准的同时,尽早覆盖重要的和具有较高失效率的测试需求,从而解决尽早检测到更多的、严重等级较高的软件缺陷这一多目标测试用例优化问题.实验结果表明:与随机测试、传统的单目标优先级排序方法和确定性排序方法相比,所提出的策略能够在更短的时间内完成同等质量的软件测试,从而提高了测试效率.

回归测试中的测试用例优先排序技术述评

测试用例优先排序(test case prioritization,简称TCP)问题是回归测试研究中的一个热点.通过设定特定排序准则,对测试用例进行排序以优化其执行次序,旨在最大化排序目标,例如最大化测试用例集的早期缺陷检测速率.TCP问题尤其适用于因测试预算不足以致不能执行完所有测试用例的测试场景.首先对TCP问题进行描述,并依次从源代码、需求和模型这3个角度出发对已有的TCP技术进行分类;然后对一类特殊的TCP问题(即测试资源感知的TCP问题)的已有研究成果进行总结;随后依次总结实证研究中常用的评测指标、评测数据集和缺陷类型对实证研究结论的影响;接着依次介绍TCP技术在一些特定测试领域中的应用,包括组合测试、事件驱动型应用测试、Web服务测试和缺陷定位等;最后对下一步工作进行展望.