Finds in Testing Experiments for Model Evaluation

To evaluate the fault location and the failure prediction models, simulation-based and code- based experiments were conducted to collect the required failure data. The PIE model was applied to simu- late failures in the simulation-based experiment. Based on syntax and semantic level fault injections, a hy- brid fault injection model is presented. To analyze the injected faults, the difficulty to inject (DTI) and diffi- culty to detect (DTD) are introduced and are measured from the programs used in the code-based experi- ment. Three interesting results were obtained from the experiments: 1) Failures simulated by the PIE model without consideration of the program and testing features are unreliably predicted; 2) There is no obvious correlation between the DTI and DTD parameters; 3) The DTD for syntax level faults changes in a different pattern to that for semantic level faults when the DTI increases. The results show that the parameters have a strong effect on the failures simulated, and the measurement of DTD is not strict.