A Comparative Evaluation of Test Coverage Techniques Effectiveness

Software systems have become complex and challenging to develop and maintain because of the large size of test cases with increased scalability issues. Test case prioritization methods have been successfully utilized in test case management. However, the prohibitively exorbitant cost of large test cases is now the mainstream in the software industry. The growth of agile test-driven development has increased the expectations for software quality. Yet, our knowledge of when to use various path testing criteria for cost-effectiveness is inadequate due to the inherent complexity in software testing. Existing researches attempted to address the issue without effectively tackling the scalability of large test suites to reduce time in regression testing. In order to provide a more accurate way of fault detection in software projects, we introduced novel coverage criteria, called Incremental Cluster-based test case Prioritization (ICP), and investigated its potentials by making a comparative evaluation with three un-clustered traditional coverage-based criteria: Prime-Path Coverage (PPC), Edge-Pair Coverage (EPC) and Edge Coverage (EC) based on mutation analysis. By clustering test suites, based on their dynamic run-time behavior, the number of pair-wise comparisons is reduced significantly. To compare, we analyzed 20 functions from 25 C programs, instrumented faults into the programs, and used the Mull mutation tool to generate mutants and perform a statistical analysis of the results. The experimental results show that ICP can lead to cost-effective improvements in fault detection.

A Fault-Based Testing Approach in Safety Critical Medical Systems

The advent of technology has opened unprecedented opportunities in health care delivery system as the demand for intelligent and knowledge-based systems has increased as modern medical practices become more knowledge-intensive. As a result of this, there is greater need to investigate the pervasiveness of software faults in Safety critical medical systems for proper diagnosis. The sheer volume of code in these systems creates significant concerns about the quality of the software. The rate of untimely deaths nowadays is alarming partly due to the medical device used to carry out the diagnosis process. A safety-critical medical (SCM) system is a complex system in which the malfunctioning of software could result in death, injury of the patient or damage to the environment. The malfunctioning of the software could be as a result of the inadequacy in software testing due to test suit problem or oracle problem. Testing a SCM system poses great challenges to software testers. One of these challenges is the need to generate a limited number of test cases of a given regression test suite in a manner that does not compromise its defect detection ability. This paper presents a novel five-stage fault-based testing procedure for SCM, a model-based approach to generate test cases for differential diagnosis of Tuberculosis. We used Prime Path Coverage and Edge-Pair Coverage as coverage criteria to ensure maximum coverage to identify feasible paths. We analyzed the proposed testing procedure with the help of three metrics consisting of Fault Detection Density, Fault Detection Effectiveness and Mutation Adequacy Score. We evaluated the effectiveness of our testing procedure by running the suggested test cases on a sample historical data of tuberculosis patients. The experimental results show that our developed testing procedure has some advantages such as creating mutant graphs and Fuzzy Cognitive Map Engine while resolving the problem of eliminating infeasible test cases for effective decision making.