Test case generation based on orthogonal table for software black-box testing

Software testing is an important means to assure the software quality. This paper presents a practicable method to generate test cases of software testing, which is operational and high efficient. We discuss the identification of software specification categories and choices and make a classification tree. Based on the orthogonal array, it is easy to generate test cases. The number of this method is less than that of all combination of the choices.

Towards Improving the Quality of Requirement and Testing Process in Agile Software Development:An Empirical Study

Software testing is a critical phase due to misconceptions about ambiguities in the requirements during specification,which affect the testing process.Therefore,it is difficult to identify all faults in software.As requirement changes continuously,it increases the irrelevancy and redundancy during testing.Due to these challenges;fault detection capability decreases and there arises a need to improve the testing process,which is based on changes in requirements specification.In this research,we have developed a model to resolve testing challenges through requirement prioritization and prediction in an agile-based environment.The research objective is to identify the most relevant and meaningful requirements through semantic analysis for correct change analysis.Then compute the similarity of requirements through case-based reasoning,which predicted the requirements for reuse and restricted to error-based requirements.Afterward,the apriori algorithm mapped out requirement frequency to select relevant test cases based on frequently reused or not reused test cases to increase the fault detection rate.Furthermore,the proposed model was evaluated by conducting experiments.The results showed that requirement redundancy and irrelevancy improved due to semantic analysis,which correctly predicted the requirements,increasing the fault detection rate and resulting in high user satisfaction.The predicted requirements are mapped into test cases,increasing the fault detection rate after changes to achieve higher user satisfaction.Therefore,the model improves the redundancy and irrelevancy of requirements by more than 90%compared to other clustering methods and the analytical hierarchical process,achieving an 80%fault detection rate at an earlier stage.Hence,it provides guidelines for practitioners and researchers in the modern era.In the future,we will provide the working prototype of this model for proof of concept.

A Parallel Hybrid Testing Technique for Tri-Programming Model-Based Software Systems

Recently,researchers have shown increasing interest in combining more than one programming model into systems running on high performance computing systems(HPCs)to achieve exascale by applying parallelism at multiple levels.Combining different programming paradigms,such as Message Passing Interface(MPI),Open Multiple Processing(OpenMP),and Open Accelerators(OpenACC),can increase computation speed and improve performance.During the integration of multiple models,the probability of runtime errors increases,making their detection difficult,especially in the absence of testing techniques that can detect these errors.Numerous studies have been conducted to identify these errors,but no technique exists for detecting errors in three-level programming models.Despite the increasing research that integrates the three programming models,MPI,OpenMP,and OpenACC,a testing technology to detect runtime errors,such as deadlocks and race conditions,which can arise from this integration has not been developed.Therefore,this paper begins with a definition and explanation of runtime errors that result fromintegrating the three programming models that compilers cannot detect.For the first time,this paper presents a classification of operational errors that can result from the integration of the three models.This paper also proposes a parallel hybrid testing technique for detecting runtime errors in systems built in the C++programming language that uses the triple programming models MPI,OpenMP,and OpenACC.This hybrid technology combines static technology and dynamic technology,given that some errors can be detected using static techniques,whereas others can be detected using dynamic technology.The hybrid technique can detect more errors because it combines two distinct technologies.The proposed static technology detects a wide range of error types in less time,whereas a portion of the potential errors that may or may not occur depending on the 4502 CMC,2023,vol.74,no.2 operating environment are left to the dynamic technology,which completes the validation.

Personalized Learning Path Recommendations for Software Testing Courses Based on Knowledge Graphs

Software testing courses are characterized by strong practicality,comprehensiveness,and diversity.Due to the differences among students and the needs to design personalized solutions for their specific requirements,the design of the existing software testing courses fails to meet the demands for personalized learning.Knowledge graphs,with their rich semantics and good visualization effects,have a wide range of applications in the field of education.In response to the current problem of software testing courses which fails to meet the needs for personalized learning,this paper offers a learning path recommendation based on knowledge graphs to provide personalized learning paths for students.

Strategy and Methodology of Integration Testing for GUI Software

In this paper, by means of effective testing practices, main strategies of integration testing for GUI software, including differentiating strategy for distinguished system, strategy of personnel organization, incremental testing strategy based on baseline version, testing strategy of circulating loop through the whole life, and the strategy of test suite construction, were briefly investigated. Moreover, for the code analysis, the FTA (Fault Tree analysis) is proposed to deal with the software change in regression testing. For test suite constructing, the constructing methods for baseline version and the incremental change are deeply discussed, in which main points focus on the testing strategy based on “Sheet/Form”, the “Grey-box approach” for integration testing process, and the application of the improved STD (State Transform Diagram) in state testing. At the same time, the suite construction of integration testing for two types, including small scale program and large scale software, is analyzed and discussed in detail. For testing execution, the specific method based on “Cross-testing” is investigated. Concurrently, by a lot of examples, all results of testing activity indicate that these strategies and methods are useful and fitted to integration testing for GUI software.

文献“Software Testing in Space Programs”点评

作者以他个人的观点,对该文献中重要部分进行了点评,指出:空间软件地面可靠性测试尽管困难重重且有诸多限制,但不能放松甚至放弃执行现有的软件地面测试标准;应从"精神号"火星探测器飞行失败中吸取教训,加强软件地面测试,包括传统软件黑盒和白盒测试方法中的典型测试项目(如:健壮测试、应力测试、极值测试、随机测试等);已经公布的新软件研发标准——Aerospace Report No.TOR-2004(3909)-3537,Software Development Standard for Space Systems值得我们参考。

Guideline of Test Suite Construction for GUI Software Centered on Grey-Box Approach

In this paper, the test suite construction for GUI (Graphical User Interface) software may be executed centered on grey-box approach with the prior test design of window access controls for unit testing, including front-end method of white box and follow-up black box method for integration testing. Moreover, two key opinions are proposed for the test suite construction for GUI software, the first one is that the “Triple-step method” should be used for unit testing with the prior disposing of data boundary value testing of input controls, and another one is that the “Grey-box approach” should be applied in integration testing for GUI software with necessary testing preparation in the precondition. At the same time, the testing of baseline version and the incremental testing should be considered for the test case construction to coordinate with the whole evolution of software product today. Additionally, all our opinion and thought are verified and tested with a typical case of GUI software—PQMS (Product Quality Monitoring Software/System), and results indicate that these methods and specific disposing are practical and effective.

On modeling approach for embedded real-time software simulation testing

Modeling technology has been introduced into software testing field. However, how to carry through the testing modeling effectively is still a difficulty. Based on combination of simulation modeling technology and embedded real-time software testing method, the process of simulation testing modeling is studied first. And then, the supporting environment of simulation testing modeling is put forward. Furthermore, an approach of embedded real-time software simulation testing modeling including modeling of cross-linked equipments of system under testing （SUT）, test case, testing scheduling, and testing system service is brought forward. Finally, the formalized description and execution system of testing models are given, with which we can realize real-time, closed loop, mad automated system testing for embedded real-time software.

Real-time embedded software testing method based on extended finite state machine

The reliability of real-time embedded software directly determines the reliability of the whole real-time embedded sys- tem, and the effective software testing is an important way to ensure software quality and reliability. Based on the analysis of the characteristics of real-time embedded software, the formal method is introduced into the real-time embedded software testing field and the real-time extended finite state machine （RT-EFSM） model is studied firstly. Then, the time zone division method of real-time embedded system is presented and the definition and description methods of time-constrained transition equivalence class （timeCTEC） are presented. Furthermore, the approaches of the testing sequence and test case generation are put forward. Finally, the proposed method is applied to a typical avionics real- time embedded software testing practice and the examples of the timeCTEC, testing sequences and test cases are given. With the analysis of the testing result, the application verification shows that the proposed method can effectively describe the real-time embedded software state transition characteristics and real-time requirements and play the advantages of the formal methods in accuracy, effectiveness and the automation supporting. Combined with the testing platform, the real-time, closed loop and automated simulation testing for real-time embedded software can be realized effectively.

Software Test Case Generation with Adequacy Analysis on Scenario-Based Testing

This paper studies the software scenario testing, which is commonly used in black-box testing at present. In the paper, the workflow model based on task-driven, which is very common in scenario testing, is analyzed. According to test adequacy criteria in scenario testing, the model is designed to correspond test cases in the light of logic block(LB). The final test cases that conform to the test adequacy criteria can be obtained through test case combination and test case reduction. In the last part of the paper, example of actual workflow is to design the efficient test case. Therefore the method is proved to be effective.

Empirical Measurement of the Software Testing and Reliability

The meanings of parameters of software reliabi- lity models are investigated in terms of the process of the software testing and in terms of other measurements of software. Based on the investigation, the empirical estimation of the parameters is addressed. On one hand, these empirical estimates are also measurements of the software, which can be used to control and to optimize the process of the software development. On the other hand, by treating these empirical estimates as Bayes priors, software reliability models are extended such that the engineers’ experience can be integrated into and hence to improve the models.

NHPP-based software reliability model considering testing effort and multivariate fault detection rate

In recent decades,many software reliability growth models（SRGMs） have been proposed for the engineers and testers in measuring the software reliability precisely.Most of them is established based on the non-homogeneous Poisson process（NHPP）,and it is proved that the prediction accuracy of such models could be improved by adding the describing of characterization of testing effort.However,some research work indicates that the fault detection rate（FDR） is another key factor affects final software quality.Most early NHPPbased models deal with the FDR as constant or piecewise function,which does not fit the different testing stages well.Thus,this paper first incorporates a multivariate function of FDR,which is bathtub-shaped,into the NHPP-based SRGMs considering testing effort in order to further improve performance.A new model framework is proposed,and a stepwise method is used to apply the framework with real data sets to find the optimal model.Experimental studies show that the obtained new model can provide better performance of fitting and prediction compared with other traditional SRGMs.

Modified Harris Hawks Optimization Based Test Case Prioritization for Software Testing

Generally,software testing is considered as a proficient technique to achieve improvement in quality and reliability of the software.But,the quality of test cases has a considerable influence on fault revealing capability of software testing activity.Test Case Prioritization(TCP)remains a challenging issue since prioritizing test cases is unsatisfactory in terms of Average Percentage of Faults Detected(APFD)and time spent upon execution results.TCP ismainly intended to design a collection of test cases that can accomplish early optimization using preferred characteristics.The studies conducted earlier focused on prioritizing the available test cases in accelerating fault detection rate during software testing.In this aspect,the current study designs aModified Harris Hawks Optimization based TCP(MHHO-TCP)technique for software testing.The aim of the proposed MHHO-TCP technique is to maximize APFD and minimize the overall execution time.In addition,MHHO algorithm is designed to boost the exploration and exploitation abilities of conventional HHO algorithm.In order to validate the enhanced efficiency of MHHO-TCP technique,a wide range of simulations was conducted on different benchmark programs and the results were examined under several aspects.The experimental outcomes highlight the improved efficiency of MHHO-TCP technique over recent approaches under different measures.

Analysis on testing and operational reliability of software

Software reliability was estimated based on NHPP software reliability growth models. Testing reliability and operational reliability may be essentially different. On the basis of analyzing similarities and differences of the testing phase and the operational phase, using the concept of operational reliability and the testing reliability, different forms of the comparison between the operational failure ratio and the predicted testing failure ratio were conducted, and the mathematical discussion and analysis were performed in detail. Finally, software optimal release was studied using software failure data. The results show that two kinds of conclusions can be derived by applying this method, one conclusion is to continue testing to meet the required reliability level of users, and the other is that testing stops when the required operational reliability is met, thus the testing cost can be reduced.

An Approach to Event-Driven Software Testing

In this paper, the testing technology of event-driven software is focused. It is first analyzed the difference between event-driven software and the traditional procedure-oriented software, and based on the above analysis, the mechanism of event-driven and the effect of introduction of event-driven mechanism on software testing are unveiled. Then based on the characteristic of the event-driven software, the traditional software testing method is improved, and testing policy of event based test is presented in this paper.Moreover the event coverage criteria are defined and given here. At the same time the event executing rule are further uncovered, such as ordinal event, non-ordinal event, predecessor event and concurrent event etc., and also the methods of testing according to event executing rule are studied.

Using Test Employee Capability Maturity Model for Supporting Gaps Bridging in Software Testing

This study addresses the challenge of assessing gaps among the differences of test people in eight groups by matching them based on four scenarios. The proposed model called Test Employee Capability Maturity Model (TEC-MM) helps find the gaps and measure the capability and maturity levels of each sub-maturity models in four dimensions as technicality, management, business, and personality. We applied TEC-MM to a software company which has instituted test people improvement strategy and plans to be a front runner in software testing industry. The findings reveal that gaps existed and consequently affect software testing process quality.

Managing Software Testing Technical Debt Using Evolutionary Algorithms

Technical debt(TD)happens when project teams carry out technical decisions in favor of a short-term goal(s)in their projects,whether deliberately or unknowingly.TD must be properly managed to guarantee that its negative implications do not outweigh its advantages.A lot of research has been conducted to show that TD has evolved into a common problem with considerable financial burden.Test technical debt is the technical debt aspect of testing(or test debt).Test debt is a relatively new concept that has piqued the curiosity of the software industry in recent years.In this article,we assume that the organization selects the testing artifacts at the start of every sprint.Implementing the latest features in consideration of expected business value and repaying technical debt are among candidate tasks in terms of the testing process(test cases increments).To gain the maximum benefit for the organization in terms of software testing optimization,there is a need to select the artifacts(i.e.,test cases)with maximum feature coverage within the available resources.The management of testing optimization for large projects is complicated and can also be treated as a multi-objective problem that entails a trade-off between the agile software’s short-term and long-term value.In this article,we implement a multi-objective indicatorbased evolutionary algorithm(IBEA)for fixing such optimization issues.The capability of the algorithm is evidenced by adding it to a real case study of a university registration process.

Research and Practice of Hybrid Teaching for Software Testing based on Interactive Learning

To address the problems of insufficient number of personalized exercises and cases and teachers’lack of grasp of students’weak knowledge points in the current software testing online courses,we study the strategy of establishing and updating intelligent exercise sets and case libraries and analyze the answers and dig out the weak points of knowledge through group intelligence reasoning and interactive machine learning methods.This will help teachers to make uniform and targeted explanations,reduce manual judgment,and achieve intelligent teaching quality reform,and implement the educational concepts of“keeping up with the times”and“teaching according to students’abilities”.

A Study of Performance Testing in Configurable Software Systems

Customizing applications through program configuration options has been proved by many open-source and commercial projects as one of the best practices in software engineering. However, traditional performance testing is not in synch with this industrial practice. Traditional performance testing techniques consider program inputs as the only external factor. It ignores the performance influence of configuration options. This study aims to stimulate research interest in performance testing in the context of configurable software systems by answering three research questions. That is, why it is necessary to conduct research in performance testing, what are the state-of-the-art techniques, and how do we conduct performance testing research in configurable software systems. In this study, we examine the unique characteristics and challenges of performance testing research in configurable software systems. We review and discuss research topics on the performance bug study, performance anti-patterns, program analysis, and performance testing. We share the research findings from the empirical study and outline the opening opportunities for new and advanced researchers to contribute to the research community.

Optimal Budget Spending for Software Testing under the Condition of Nonlinear Constraint

Software testing is a very important phase of the software development process. It is a very difficult job for a software manager to allocate optimally the financial budget to a software project during testing. In this paper the problem of optimal allocation of the software testing cost is studied. There exist several models focused on the development of software costs measuring the number of software errors remaining in the software during testing. The purpose of this paper is to use these models to formulate the optimization problems of resource allocation: Minimization of the total number of software errors remaining in the system. On the assumption that a software project consists of some independent modules, the presented approach extends previous work by defining new goal functions and extending the primary assumption and precondition.