Effectiveness Assessment of the Search-Based Statistical Structural Testing

Search-based statistical structural testing(SBSST)is a promising technique that uses automated search to construct input distributions for statistical structural testing.It has been proved that a simple search algorithm,for example,the hill-climber is able to optimize an input distribution.However,due to the noisy fitness estimation of the minimum triggering probability among all cover elements(Tri-Low-Bound),the existing approach does not show a satisfactory efficiency.Constructing input distributions to satisfy the Tri-Low-Bound criterion requires an extensive computation time.Tri-Low-Bound is considered a strong criterion,and it is demonstrated to sustain a high fault-detecting ability.This article tries to answer the following question:if we use a relaxed constraint that significantly reduces the time consumption on search,can the optimized input distribution still be effective in faultdetecting ability?In this article,we propose a type of criterion called fairnessenhanced-sum-of-triggering-probability(p-L1-Max).The criterion utilizes the sum of triggering probabilities as the fitness value and leverages a parameter p to adjust the uniformness of test data generation.We conducted extensive experiments to compare the computation time and the fault-detecting ability between the two criteria.The result shows that the 1.0-L1-Max criterion has the highest efficiency,and it is more practical to use than the Tri-Low-Bound criterion.To measure a criterion’s fault-detecting ability,we introduce a definition of expected faults found in the effective test set size region.To measure the effective test set size region,we present a theoretical analysis of the expected faults found with respect to various test set sizes and use the uniform distribution as a baseline to derive the effective test set size region’s definition.