Test Case Generation Evaluator for the Implementation of Test Case Generation Algorithms Based on Learning to Rank

In software testing,the quality of test cases is crucial,but manual generation is time-consuming.Various automatic test case generation methods exist,requiring careful selection based on program features.Current evaluation methods compare a limited set of metrics,which does not support a larger number of metrics or consider the relative importance of each metric to the final assessment.To address this,we propose an evaluation tool,the Test Case Generation Evaluator(TCGE),based on the learning to rank(L2R)algorithm.Unlike previous approaches,our method comprehensively evaluates algorithms by considering multiple metrics,resulting in a more reasoned assessment.The main principle of the TCGE is the formation of feature vectors that are of concern by the tester.Through training,the feature vectors are sorted to generate a list,with the order of the methods on the list determined according to their effectiveness on the tested assembly.We implement TCGE using three L2R algorithms:Listnet,LambdaMART,and RFLambdaMART.Evaluation employs a dataset with features of classical test case generation algorithms and three metrics—Normalized Discounted Cumulative Gain(NDCG),Mean Average Precision(MAP),and Mean Reciprocal Rank(MRR).Results demonstrate the TCGE’s superior effectiveness in evaluating test case generation algorithms compared to other methods.Among the three L2R algorithms,RFLambdaMART proves the most effective,achieving an accuracy above 96.5%,surpassing LambdaMART by 2%and Listnet by 1.5%.Consequently,the TCGE framework exhibits significant application value in the evaluation of test case generation algorithms.

The first power generation test of hot dry rock resources exploration and production demonstration project in the Gonghe Basin,Qinghai Province,China

Hot dry rock(HDR)is a kind of clean energy with significant potential.Since the 1970s,the United States,Japan,France,Australia,and other countries have attempted to conduct several HDR development research projects to extract thermal energy by breaking through key technologies.However,up to now,the development of HDR is still in the research,development,and demonstration stage.An HDR exploration borehole(with 236℃ at a depth of 3705 m)was drilled into Triassic granite in the Gonghe Basin in northwest China in 2017.Subsequently,China Geological Survey(CGS)launched the HDR resources exploration and production demonstration project in 2019.After three years of efforts,a sequence of significant technological breakthroughs have been made,including the genetic model of deep heat sources,directional drilling and well completion in high-temperature hard rock,large-scale reservoir stimulation,reservoir characterization,and productivity evaluation,reservoir connectivity and flow circulation,efficient thermoelectric conversion,monitoring,and geological risk assessment,etc.Then the whole-process technological system for HDR exploration and production has been preliminarily established accordingly.The first power generation test was completed in November 2021.The results of this project will provide scientific support for HDR development and utilization in the future.

Automated Test Case Generation from Requirements: A Systematic Literature Review

Software testing is an important and cost intensive activity in software development.The major contribution in cost is due to test case generations.Requirement-based testing is an approach in which test cases are derivative from requirements without considering the implementation’s internal structure.Requirement-based testing includes functional and nonfunctional requirements.The objective of this study is to explore the approaches that generate test cases from requirements.A systematic literature review based on two research questions and extensive quality assessment criteria includes studies.The study identies 30 primary studies from 410 studies spanned from 2000 to 2018.The review’s nding shows that 53%of journal papers,42%of conference papers,and 5%of book chapters’address requirementsbased testing.Most of the studies use UML,activity,and use case diagrams for test case generation from requirements.One of the signicant lessons learned is that most software testing errors are traced back to errors in natural language requirements.A substantial amount of work focuses on UML diagrams for test case generations,which cannot capture all the system’s developed attributes.Furthermore,there is a lack of UML-based models that can generate test cases from natural language requirements by rening them in context.Coverage criteria indicate how efciently the testing has been performed 12.37%of studies use requirements coverage,20%of studies cover path coverage,and 17%study basic coverage.

Test Case Generation from UML-Diagrams Using Genetic Algorithm

Software testing has been attracting a lot of attention for effective software development.In model driven approach,Unified Modelling Language(UML)is a conceptual modelling approach for obligations and other features of the system in a model-driven methodology.Specialized tools interpret these models into other software artifacts such as code,test data and documentation.The generation of test cases permits the appropriate test data to be determined that have the aptitude to ascertain the requirements.This paper focuses on optimizing the test data obtained from UML activity and state chart diagrams by using Basic Genetic Algorithm(BGA).For generating the test cases,both diagrams were converted into their corresponding intermediate graphical forms namely,Activity Diagram Graph(ADG)and State Chart Diagram Graph(SCDG).Then both graphs will be combined to form a single graph called,Activity State Chart Diagram Graph(ASCDG).Both graphs were then joined to create a single graph known as the Activity State Chart Diagram Graph(ASCDG).Next,the ASCDG will be optimized using BGA to generate the test data.A case study involving a withdrawal from the automated teller machine(ATM)of a bank was employed to demonstrate the approach.The approach successfully identified defects in various ATM functions such as messaging and operation.

Class hierarchical test case generation algorithm based on expanded EMDPN model

A new model of event and message driven Petri network(EMDPN) based on the characteristic of class interaction for messages passing between two objects was extended. Using EMDPN interaction graph, a class hierarchical test-case generation algorithm with cooperated paths (copaths) was proposed, which can be used to solve the problems resulting from the class inheritance mechanism encountered in object-oriented software testing such as oracle, message transfer errors, and unreachable statement. Finally, the testing sufficiency was analyzed with the ordered sequence testing criterion(OSC). The results indicate that the test cases stemmed from newly proposed automatic algorithm of copaths generation satisfies synchronization message sequences testing criteria, therefore the proposed new algorithm of copaths generation has a good coverage rate.

Study and performance test of 10 kW molten carbonate fuel cell power generation system

The use of high-temperature fuel cells as a power technology can improve the efficiency of electricity generation and achieve near-zero emissions of carbon dioxide.This work explores the performance of a 10 kW high-temperature molten carbonate fuel cell.The key materials of a single cell were characterized and analyzed using X-ray diffraction and scanning electron microscopy.The results show that the pore size of the key electrode material is 6.5 lm and the matrix material is a-LiAlO_(2).Experimentally,the open circuit voltage of the single cell was found to be 1.23 V.The current density was greater than 100 mA/cm^(2)at an operating voltage of 0.7 V.The 10 kW fuel cell stack comprised 80 single fuel cells with a total area of 2000 cm^(2)and achieved an open circuit voltage of greater than 85 V.The fuel cell stack power and current density could reach 11.7 kW and 104.5 mA/cm2 at an operating voltage of 56 V.The influence and long-term stable operation of the stack were also analyzed and discussed.The successful operation of a 10 kW high-temperature fuel cell promotes the large-scale use of fuel cells and provides a research basis for future investigations of fuel cell capacity enhancement and distributed generation in China.

A scheme on automated test data generation and its evaluation

By analyzing some existing test data generation methods, a new automated test data generation approach was presented. The linear predicate functions on a given path was directly used to construct a linear constrain system for input variables. Only when the predicate function is nonlinear, does the linear arithmetic representation need to be computed. If the entire predicate functions on the given path are linear, either the desired test data or the guarantee that the path is infeasible can be gotten from the solution of the constrain system. Otherwise, the iterative refining for the input is required to obtain the desired test data. Theoretical analysis and test results show that the approach is simple and effective, and takes less computation. The scheme can also be used to generate path-based test data for the programs with arrays and loops.

Structure-Oriented RTL Automatic Test Generation

The paper presents structure-oriented Register Transfer Level (RTL) test generation algorithm, which hierarchically tests large-scale circuits. It generates tests for low-level circuit with gate-level test generation technology, and generates tests for high-level circuit with combining module test sets. It also presents a new fault-simulation algorithm at RT level circuit to adapt test generation hierarchically.

Test Generation and Design-for-Testability Based on Acyclic Structure with Hold Registers

We present a method of test generation for acyclic sequential circuits with hold registers. A complete (100% fault efficiency) test sequence for an acyclic sequential circuit can be obtained by applying a combinational test generator to all the maximal time-expansion models (TEMs) of the circuit. We propose a class of acyclic sequential circuits for which the number of maximal TEMs is one, i.e, the maximum TEM exists. For a circuit in the class, test generation can be performed by using only the maximum TEM. The proposed class of sequential circuits with the maximum TEM properly includes several known classes of acyclic sequential circuits such as balanced structures and acyclic sequential circuits without hold registers for which test generation can be also performed by using a combinational test generator. Therefore, in general, the hardware overhead for partial scan based on the proposed structure is smaller than that based on balanced or acyclic sequential structure without hold registers.

MC/DC Test Data Generation Algorithm Based on Whale Genetic Algorithm

The automatic generation of test data is a key step in realizing automated testing.Most automated testing tools for unit testing only provide test case execution drivers and cannot generate test data that meets coverage requirements.This paper presents an improved Whale Genetic Algorithm for generating test data re-quired for unit testing MC/DC coverage.The proposed algorithm introduces an elite retention strategy to avoid the genetic algorithm from falling into iterative degradation.At the same time,the mutation threshold of the whale algorithm is introduced to balance the global exploration and local search capabilities of the genetic al-gorithm.The threshold is dynamically adjusted according to the diversity and evolution stage of current popu-lation,which positively guides the evolution of the population.Finally,an improved crossover strategy is pro-posed to accelerate the convergence of the algorithm.The improved whale genetic algorithm is compared with genetic algorithm,whale algorithm and particle swarm algorithm on two benchmark programs.The results show that the proposed algorithm is faster for test data generation than comparison methods and can provide better coverage with fewer evaluations,and has great advantages in generating test data.

An EFSM-Based Test Data Generation Approach in Model-Based Testing

Testing is an integral part of software development.Current fastpaced system developments have rendered traditional testing techniques obsolete.Therefore,automated testing techniques are needed to adapt to such system developments speed.Model-based testing(MBT)is a technique that uses system models to generate and execute test cases automatically.It was identified that the test data generation(TDG)in many existing model-based test case generation(MB-TCG)approaches were still manual.An automatic and effective TDG can further reduce testing cost while detecting more faults.This study proposes an automated TDG approach in MB-TCG using the extended finite state machine model(EFSM).The proposed approach integrates MBT with combinatorial testing.The information available in an EFSM model and the boundary value analysis strategy are used to automate the domain input classifications which were done manually by the existing approach.The results showed that the proposed approach was able to detect 6.62 percent more faults than the conventionalMB-TCG but at the same time generated 43 more tests.The proposed approach effectively detects faults,but a further treatment to the generated tests such as test case prioritization should be done to increase the effectiveness and efficiency of testing.

Research of the test generation algorithm based on search state dominance for combinational circuit

On the basis of EST (Equivalent STate hashing) algorithm, this paper researches a kind of test generation algorithm based on search state dominance for combinational circuit. According to the dominance relation of the E-frontier (evaluation frontier), we can prove that this algorithm can terminate unnecessary searching step of test pattern earlier than the EST algorithm through some examples, so this algorithm can reduce the time of test generation. The test patterns calculated can detect faults given through simulation.

Minimal-Length Interoperability Test Sequences Generation via Genetic Algorithm

A novel interoperability test sequences optimization scheme is proposed in which the genetic algorithm （GA） is used to obtain the minimal-length interoperability test sequences. During our work, the basic interoperability test sequences are generated based on the minimal-complete-coverage criterion, which removes the redundancy from conformance test sequences. Then interoperability sequences minimization problem can be considered as an instance of the set covering problem, and the GA is applied to remove redundancy in interoperability transitions. The results show that compared to conventional algorithm, the proposed algorithm is more practical to avoid the state space explosion problem, for it can reduce the length of the test sequences and maintain the same transition coverage.

The structure-based multi-fault test generation algorithm for combinational circuit

In this paper the structure-based test generation algorithm has been studied for the problem that test patterns are obtained by determined finite faults set in the past. This Algorithm can find out all test patterns one tithe, so faults detection is very convenient. By simulation, the smallest test patterns set can be obtained and faults coverage rate is 100%.

Principle and Design of a Test Suite Generation System——TSGS

Formal methods for test sequence generation from FSM have been studied widely andthoroughly,but most real communication systems can only be modeled as EFSM exactly.Data portion in EFSM brings difficulties for test suite generation.In this paper,the strategyof generating test suite from protocols modelled as EFSM is presented.This strategy consid-ers testing of both the control portion and data portion of protocols.A software,the testsuite generation system(TSGS)based on above strategy,is introduced.

A new approach to test generation for combinational circuits

Aimed at the generation of high-quality test set in the shortest possible time, the test generation for combinational circuits （CC） based on the chaotic particle swarm optimization （CPSO） algorithm is presented according to the analysis of existent problems of CC test generation, and an appropriate CPSO algorithm model has been constructed. With the help of fault simulator, the test set of ISCAS＇ 85 benchmark CC is generated using the CPSO, and some techniques are introduced such as half-random generation, and simulation of undetected fauhs.with original test vector, and inverse test vector. Experimental results show that this algorithm can generate the same fault coverage and small-size test set in short time compared with other known similar methods, which proves that the proposed method is applicable and effective.

Question Bank Construction and Test Paper Auto-generation System Implementation Based on Delphi

Nowadays, the computer is increasingly popular, and college examination is developing in the direction of traditional examination means to automation and intelligence ones gradually, all these make it inevitable to construct question bank for courses, and to generate test paper using computers. This paper uses the Delphi technique, to make improvements to existing components, combining with VBA programming, and use of SQL Server to implement the question bank management and test paper auto-generation system, which could generate test paper in Word Document. A large number of tests show that the software is running stably and system features are functioning correctly on Windows 2000/XP/2003 platform with Office XP/2003 environment.

SMINER:Detecting Unrestricted and Misimplemented Behaviors of Software Systems Based on Unit Test Cases

Despite the advances in automated vulnerability detection approaches,security vulnerabilities caused by design flaws in software systems are continuously appearing in real-world systems.Such security design flaws can bring unrestricted and misimplemented behaviors of a system and can lead to fatal vulnerabilities such as remote code execution or sensitive data leakage.Therefore,it is an essential task to discover unrestricted and misimplemented behaviors of a system.However,it is a daunting task for security experts to discover such vulnerabilities in advance because it is timeconsuming and error-prone to analyze the whole code in detail.Also,most of the existing vulnerability detection approaches still focus on detecting memory corruption bugs because these bugs are the dominant root cause of software vulnerabilities.This paper proposes SMINER,a novel approach that discovers vulnerabilities caused by unrestricted and misimplemented behaviors.SMINER first collects unit test cases for the target system from the official repository.Next,preprocess the collected code fragments.SMINER uses pre-processed data to show the security policies that can occur on the target system and creates a test case for security policy testing.To demonstrate the effectiveness of SMINER,this paper evaluates SMINER against Robot Operating System(ROS),a real-world system used for intelligent robots in Amazon and controlling satellites in National Aeronautics and Space Administration(NASA).From the evaluation,we discovered two real-world vulnerabilities in ROS.

Enhancing SAT-Based Test Pattern Generation

This paper presents modeling tools based on Boolean satisfiability (SAT) to solve problems of test generation for combinational circuits. It exploits an added layer to maintain circuit-related information and value justification relations to a generic SAT algorithm. It dovetails binary decision graphs (BDD) and SAT techniques to improve the efficiency of automatic test pattern generation (ATPG). More specifically, it first exploits inexpensive reconvergent fanout analysis of circuit to gather information on the local signal correlation by using BDD learning, then uses the above learned information to restrict and focus the overall search space of SAT-based ATPG. Its learning technique is effective and lightweight. The experimental results demonstrate the effectiveness of the approach.

Generating of Test Data by Harmony Search Against Genetic Algorithms

Many search-based algorithms have been successfully applied in sev-eral software engineering activities.Genetic algorithms(GAs)are the most used in the scientific domains by scholars to solve software testing problems.They imi-tate the theory of natural selection and evolution.The harmony search algorithm(HSA)is one of the most recent search algorithms in the last years.It imitates the behavior of a musician tofind the best harmony.Scholars have estimated the simi-larities and the differences between genetic algorithms and the harmony search algorithm in diverse research domains.The test data generation process represents a critical task in software validation.Unfortunately,there is no work comparing the performance of genetic algorithms and the harmony search algorithm in the test data generation process.This paper studies the similarities and the differences between genetic algorithms and the harmony search algorithm based on the ability and speed offinding the required test data.The current research performs an empirical comparison of the HSA and the GAs,and then the significance of the results is estimated using the t-Test.The study investigates the efficiency of the harmony search algorithm and the genetic algorithms according to(1)the time performance,(2)the significance of the generated test data,and(3)the adequacy of the generated test data to satisfy a given testing criterion.The results showed that the harmony search algorithm is significantly faster than the genetic algo-rithms because the t-Test showed that the p-value of the time values is 0.026<α(αis the significance level=0.05 at 95%confidence level).In contrast,there is no significant difference between the two algorithms in generating the adequate test data because the t-Test showed that the p-value of thefitness values is 0.25>α.