Test Case Generation Evaluator for the Implementation of Test Case Generation Algorithms Based on Learning to Rank

In software testing,the quality of test cases is crucial,but manual generation is time-consuming.Various automatic test case generation methods exist,requiring careful selection based on program features.Current evaluation methods compare a limited set of metrics,which does not support a larger number of metrics or consider the relative importance of each metric to the final assessment.To address this,we propose an evaluation tool,the Test Case Generation Evaluator(TCGE),based on the learning to rank(L2R)algorithm.Unlike previous approaches,our method comprehensively evaluates algorithms by considering multiple metrics,resulting in a more reasoned assessment.The main principle of the TCGE is the formation of feature vectors that are of concern by the tester.Through training,the feature vectors are sorted to generate a list,with the order of the methods on the list determined according to their effectiveness on the tested assembly.We implement TCGE using three L2R algorithms:Listnet,LambdaMART,and RFLambdaMART.Evaluation employs a dataset with features of classical test case generation algorithms and three metrics—Normalized Discounted Cumulative Gain(NDCG),Mean Average Precision(MAP),and Mean Reciprocal Rank(MRR).Results demonstrate the TCGE’s superior effectiveness in evaluating test case generation algorithms compared to other methods.Among the three L2R algorithms,RFLambdaMART proves the most effective,achieving an accuracy above 96.5%,surpassing LambdaMART by 2%and Listnet by 1.5%.Consequently,the TCGE framework exhibits significant application value in the evaluation of test case generation algorithms.

The first power generation test of hot dry rock resources exploration and production demonstration project in the Gonghe Basin,Qinghai Province,China

Hot dry rock(HDR)is a kind of clean energy with significant potential.Since the 1970s,the United States,Japan,France,Australia,and other countries have attempted to conduct several HDR development research projects to extract thermal energy by breaking through key technologies.However,up to now,the development of HDR is still in the research,development,and demonstration stage.An HDR exploration borehole(with 236℃ at a depth of 3705 m)was drilled into Triassic granite in the Gonghe Basin in northwest China in 2017.Subsequently,China Geological Survey(CGS)launched the HDR resources exploration and production demonstration project in 2019.After three years of efforts,a sequence of significant technological breakthroughs have been made,including the genetic model of deep heat sources,directional drilling and well completion in high-temperature hard rock,large-scale reservoir stimulation,reservoir characterization,and productivity evaluation,reservoir connectivity and flow circulation,efficient thermoelectric conversion,monitoring,and geological risk assessment,etc.Then the whole-process technological system for HDR exploration and production has been preliminarily established accordingly.The first power generation test was completed in November 2021.The results of this project will provide scientific support for HDR development and utilization in the future.

Automated Test Case Generation from Requirements: A Systematic Literature Review

Software testing is an important and cost intensive activity in software development.The major contribution in cost is due to test case generations.Requirement-based testing is an approach in which test cases are derivative from requirements without considering the implementation’s internal structure.Requirement-based testing includes functional and nonfunctional requirements.The objective of this study is to explore the approaches that generate test cases from requirements.A systematic literature review based on two research questions and extensive quality assessment criteria includes studies.The study identies 30 primary studies from 410 studies spanned from 2000 to 2018.The review’s nding shows that 53%of journal papers,42%of conference papers,and 5%of book chapters’address requirementsbased testing.Most of the studies use UML,activity,and use case diagrams for test case generation from requirements.One of the signicant lessons learned is that most software testing errors are traced back to errors in natural language requirements.A substantial amount of work focuses on UML diagrams for test case generations,which cannot capture all the system’s developed attributes.Furthermore,there is a lack of UML-based models that can generate test cases from natural language requirements by rening them in context.Coverage criteria indicate how efciently the testing has been performed 12.37%of studies use requirements coverage,20%of studies cover path coverage,and 17%study basic coverage.

Study and performance test of 10 kW molten carbonate fuel cell power generation system

The use of high-temperature fuel cells as a power technology can improve the efficiency of electricity generation and achieve near-zero emissions of carbon dioxide.This work explores the performance of a 10 kW high-temperature molten carbonate fuel cell.The key materials of a single cell were characterized and analyzed using X-ray diffraction and scanning electron microscopy.The results show that the pore size of the key electrode material is 6.5 lm and the matrix material is a-LiAlO_(2).Experimentally,the open circuit voltage of the single cell was found to be 1.23 V.The current density was greater than 100 mA/cm^(2)at an operating voltage of 0.7 V.The 10 kW fuel cell stack comprised 80 single fuel cells with a total area of 2000 cm^(2)and achieved an open circuit voltage of greater than 85 V.The fuel cell stack power and current density could reach 11.7 kW and 104.5 mA/cm2 at an operating voltage of 56 V.The influence and long-term stable operation of the stack were also analyzed and discussed.The successful operation of a 10 kW high-temperature fuel cell promotes the large-scale use of fuel cells and provides a research basis for future investigations of fuel cell capacity enhancement and distributed generation in China.

Test Case Generation from UML-Diagrams Using Genetic Algorithm

Software testing has been attracting a lot of attention for effective software development.In model driven approach,Unified Modelling Language(UML)is a conceptual modelling approach for obligations and other features of the system in a model-driven methodology.Specialized tools interpret these models into other software artifacts such as code,test data and documentation.The generation of test cases permits the appropriate test data to be determined that have the aptitude to ascertain the requirements.This paper focuses on optimizing the test data obtained from UML activity and state chart diagrams by using Basic Genetic Algorithm(BGA).For generating the test cases,both diagrams were converted into their corresponding intermediate graphical forms namely,Activity Diagram Graph(ADG)and State Chart Diagram Graph(SCDG).Then both graphs will be combined to form a single graph called,Activity State Chart Diagram Graph(ASCDG).Both graphs were then joined to create a single graph known as the Activity State Chart Diagram Graph(ASCDG).Next,the ASCDG will be optimized using BGA to generate the test data.A case study involving a withdrawal from the automated teller machine(ATM)of a bank was employed to demonstrate the approach.The approach successfully identified defects in various ATM functions such as messaging and operation.

Structure-Oriented RTL Automatic Test Generation

The paper presents structure-oriented Register Transfer Level (RTL) test generation algorithm, which hierarchically tests large-scale circuits. It generates tests for low-level circuit with gate-level test generation technology, and generates tests for high-level circuit with combining module test sets. It also presents a new fault-simulation algorithm at RT level circuit to adapt test generation hierarchically.

Test Generation and Design-for-Testability Based on Acyclic Structure with Hold Registers

We present a method of test generation for acyclic sequential circuits with hold registers. A complete (100% fault efficiency) test sequence for an acyclic sequential circuit can be obtained by applying a combinational test generator to all the maximal time-expansion models (TEMs) of the circuit. We propose a class of acyclic sequential circuits for which the number of maximal TEMs is one, i.e, the maximum TEM exists. For a circuit in the class, test generation can be performed by using only the maximum TEM. The proposed class of sequential circuits with the maximum TEM properly includes several known classes of acyclic sequential circuits such as balanced structures and acyclic sequential circuits without hold registers for which test generation can be also performed by using a combinational test generator. Therefore, in general, the hardware overhead for partial scan based on the proposed structure is smaller than that based on balanced or acyclic sequential structure without hold registers.

MC/DC Test Data Generation Algorithm Based on Whale Genetic Algorithm

The automatic generation of test data is a key step in realizing automated testing.Most automated testing tools for unit testing only provide test case execution drivers and cannot generate test data that meets coverage requirements.This paper presents an improved Whale Genetic Algorithm for generating test data re-quired for unit testing MC/DC coverage.The proposed algorithm introduces an elite retention strategy to avoid the genetic algorithm from falling into iterative degradation.At the same time,the mutation threshold of the whale algorithm is introduced to balance the global exploration and local search capabilities of the genetic al-gorithm.The threshold is dynamically adjusted according to the diversity and evolution stage of current popu-lation,which positively guides the evolution of the population.Finally,an improved crossover strategy is pro-posed to accelerate the convergence of the algorithm.The improved whale genetic algorithm is compared with genetic algorithm,whale algorithm and particle swarm algorithm on two benchmark programs.The results show that the proposed algorithm is faster for test data generation than comparison methods and can provide better coverage with fewer evaluations,and has great advantages in generating test data.

An EFSM-Based Test Data Generation Approach in Model-Based Testing

Testing is an integral part of software development.Current fastpaced system developments have rendered traditional testing techniques obsolete.Therefore,automated testing techniques are needed to adapt to such system developments speed.Model-based testing(MBT)is a technique that uses system models to generate and execute test cases automatically.It was identified that the test data generation(TDG)in many existing model-based test case generation(MB-TCG)approaches were still manual.An automatic and effective TDG can further reduce testing cost while detecting more faults.This study proposes an automated TDG approach in MB-TCG using the extended finite state machine model(EFSM).The proposed approach integrates MBT with combinatorial testing.The information available in an EFSM model and the boundary value analysis strategy are used to automate the domain input classifications which were done manually by the existing approach.The results showed that the proposed approach was able to detect 6.62 percent more faults than the conventionalMB-TCG but at the same time generated 43 more tests.The proposed approach effectively detects faults,but a further treatment to the generated tests such as test case prioritization should be done to increase the effectiveness and efficiency of testing.

Minimal-Length Interoperability Test Sequences Generation via Genetic Algorithm

A novel interoperability test sequences optimization scheme is proposed in which the genetic algorithm （GA） is used to obtain the minimal-length interoperability test sequences. During our work, the basic interoperability test sequences are generated based on the minimal-complete-coverage criterion, which removes the redundancy from conformance test sequences. Then interoperability sequences minimization problem can be considered as an instance of the set covering problem, and the GA is applied to remove redundancy in interoperability transitions. The results show that compared to conventional algorithm, the proposed algorithm is more practical to avoid the state space explosion problem, for it can reduce the length of the test sequences and maintain the same transition coverage.

SMINER:Detecting Unrestricted and Misimplemented Behaviors of Software Systems Based on Unit Test Cases

Despite the advances in automated vulnerability detection approaches,security vulnerabilities caused by design flaws in software systems are continuously appearing in real-world systems.Such security design flaws can bring unrestricted and misimplemented behaviors of a system and can lead to fatal vulnerabilities such as remote code execution or sensitive data leakage.Therefore,it is an essential task to discover unrestricted and misimplemented behaviors of a system.However,it is a daunting task for security experts to discover such vulnerabilities in advance because it is timeconsuming and error-prone to analyze the whole code in detail.Also,most of the existing vulnerability detection approaches still focus on detecting memory corruption bugs because these bugs are the dominant root cause of software vulnerabilities.This paper proposes SMINER,a novel approach that discovers vulnerabilities caused by unrestricted and misimplemented behaviors.SMINER first collects unit test cases for the target system from the official repository.Next,preprocess the collected code fragments.SMINER uses pre-processed data to show the security policies that can occur on the target system and creates a test case for security policy testing.To demonstrate the effectiveness of SMINER,this paper evaluates SMINER against Robot Operating System(ROS),a real-world system used for intelligent robots in Amazon and controlling satellites in National Aeronautics and Space Administration(NASA).From the evaluation,we discovered two real-world vulnerabilities in ROS.

Enhancing SAT-Based Test Pattern Generation

This paper presents modeling tools based on Boolean satisfiability (SAT) to solve problems of test generation for combinational circuits. It exploits an added layer to maintain circuit-related information and value justification relations to a generic SAT algorithm. It dovetails binary decision graphs (BDD) and SAT techniques to improve the efficiency of automatic test pattern generation (ATPG). More specifically, it first exploits inexpensive reconvergent fanout analysis of circuit to gather information on the local signal correlation by using BDD learning, then uses the above learned information to restrict and focus the overall search space of SAT-based ATPG. Its learning technique is effective and lightweight. The experimental results demonstrate the effectiveness of the approach.

Generating of Test Data by Harmony Search Against Genetic Algorithms

Many search-based algorithms have been successfully applied in sev-eral software engineering activities.Genetic algorithms(GAs)are the most used in the scientific domains by scholars to solve software testing problems.They imi-tate the theory of natural selection and evolution.The harmony search algorithm(HSA)is one of the most recent search algorithms in the last years.It imitates the behavior of a musician tofind the best harmony.Scholars have estimated the simi-larities and the differences between genetic algorithms and the harmony search algorithm in diverse research domains.The test data generation process represents a critical task in software validation.Unfortunately,there is no work comparing the performance of genetic algorithms and the harmony search algorithm in the test data generation process.This paper studies the similarities and the differences between genetic algorithms and the harmony search algorithm based on the ability and speed offinding the required test data.The current research performs an empirical comparison of the HSA and the GAs,and then the significance of the results is estimated using the t-Test.The study investigates the efficiency of the harmony search algorithm and the genetic algorithms according to(1)the time performance,(2)the significance of the generated test data,and(3)the adequacy of the generated test data to satisfy a given testing criterion.The results showed that the harmony search algorithm is significantly faster than the genetic algo-rithms because the t-Test showed that the p-value of the time values is 0.026<α(αis the significance level=0.05 at 95%confidence level).In contrast,there is no significant difference between the two algorithms in generating the adequate test data because the t-Test showed that the p-value of thefitness values is 0.25>α.

Model Checking-Based Testing of Web Applications

A formal model representing the navigation behavior of a Web application as the Kripke structure is proposed and an approach that applies model checking to test case generation is presented. The Object Relation Diagram as the object model is employed to describe the object structure of a Web application design and can be translated into the behavior model. A key problem of model checking-based test generation for a Web application is how to construct a set of trap properties that intend to cause the violations of model checking against the behavior model and output of counterexamples used to construct the test sequences. We give an algorithm that derives trap properties from the object model with respect to node and edge coverage criteria.

IC Testing by Phase Classification Based on Behavioral Description of RTL

The growing complexity of integrated circuits (ICs) is driving the trend of IC testing towards testing based on behavioral descriptions of register-transfer level (RTL). A behavioral description contains an algorithmic specification of functionality of design. It may contain little or even no information about the design’s cycle-by-cycle behavior or structural implementation. However, it usually has an interior variable to lead the process of its functional phases. This interior variable is named phase variable. The functional behavior of a digital circuit changes according to different values of a phase variable. By analyzing some ITC99 benchmark circuits, this paper presents a way to generate tests for a circuit by tracing the value change of a phase variable in the circuit.

Multi-Objective Modified Particle Swarm Optimization for Test Suite Reduction(MOMPSO)

Software testing plays a pivotal role in entire software development lifecycle.It provides researchers with extensive opportunities to develop novel methods for the optimized and cost-effective test suite Although implementation of such a cost-effective test suite with regression testing is being under exploration still it contains lot of challenges and flaws while incorporating with any of the new regression testing algorithm due to irrelevant test cases in the test suite which are not required.These kinds of irrelevant test cases might create certain challenges such as code-coverage in the test suite,fault-tolerance,defects due to uncovered-statements and overall-performance at the time of execution.With this objective,the proposed a new Modified Particle Swarm optimization used for multi-objective test suite optimization.The experiment results involving six subject programs show that MOMPSO method can outer perform with respect to both reduction rate(90.78%to 100%)and failure detection rate(44.56%to 55.01%).Results proved MOMPSO outperformed the other stated algorithms.

Software Testing Method Based on Model Comparison

A model comparison based software testing method （MCST） is proposed. In this method, the requirements aria programs or software under test are transformed into the ones in the same form, and described by the same model describe language （MDL）. Then, the requirements are transformed into a specification model and the programs into an implementation model. Thus, the elements and structures of the two models are compared, and the differences between them are obtained. Based on the diffrences, a test suite is generated. Different MDLs can be chosen for the software under test. The usages of two classical MDLs in MCST, the equivalence classes model and the extended finite state machine （EFSM） model, are described with example applications. The results show that the test suites generated by MCST are more efficient and smaller than some other testing methods, such as the pathcoverage testing method, the object state diagram testing method, etc.

Research of testing method based on UML statecharts

Unified modeling language （UML） is a powerful graphical modeling language with intuitional meaning. It provides various diagrams to depict system characteristics and complex environment from different viewpoints and different application layers. UML-based software development and modeling environments have been widely accepted in industry, including areas in which safety is an important issue such as spaceflight, defense, automobile, etc. To ensure and improve software quality becomes a main concern in the field. As one of the key techniques for software quality, software testing can effectively detect system faults. UML based software testing based is an important research direction in software engineering. The key to software testing is the generation of test cases. This dissertation studies an approach to generating test cases from UML statecharts.

Search-Based Software Test Data Generation for Path Coverage Based on a Feedback-Directed Mechanism

Automatically generating test cases by evolutionary algorithms to satisfy the path coverage criterion has attracted much research attention in software testing.In the context of generating test cases to cover many target paths,the efficiency of existing methods needs to be further improved when infeasible or difficult paths exist in the program under test.This is because a significant amount of the search budget(i.e.,time allocated for the search to run)is consumed when computing fitness evaluations of individuals on infeasible or difficult paths.In this work,we present a feedback-directed mechanism that temporarily removes groups of paths from the target paths when no improvement is observed for these paths in subsequent generations.To fulfill this task,our strategy first organizes paths into groups.Then,in each generation,the objective scores of each individual for all paths in each group are summed up.For each group,the lowest value of the summed up objective scores among all individuals is assigned as the best aggregated score for a group.A group is removed when no improvement is observed in its best aggregated score over the last two generations.The experimental results show that the proposed approach can significantly improve path coverage rates for programs under test with infeasible or difficult paths in case of a limited search budget.In particular,the feedback-directed mechanism reduces wasting the search budget on infeasible paths or on difficult target paths that require many fitness evaluations before getting an improvement.

Review of Software Model-Checking Techniques for Dealing with Error Detection in Program Codes

Debugging software code has been a challenge for software developers since the early days of computer programming. A simple need, because the world is run by software. So perhaps the biggest engineering challenge is finding ways to make software more reliable. This review provides an overview of techniques developed over time in the field of software model checking to solve the problem of detecting errors in program code. In addition, the challenges posed by this technology are discussed and ways to mitigate them in future research and applications are proposed. A comprehensive examination of the various model verification methods used to detect program code errors is intended to lay the foundation for future research in this area.