Discrete Software Reliability Growth Modeling for Errors of Different Severity Incorporating Change-point Concept

Several software reliability growth models （SRGM） have been developed to monitor the reliability growth during the testing phase of software development. In most of the existing research available in the literatures, it is considered that a similar testing effort is required on each debugging effort. However, in practice, different types of faults may require different amounts of testing efforts for their detection and removal. Consequently, faults are classified into three categories on the basis of severity： simple, hard and complex. This categorization may be extended to r type of faults on the basis of severity. Although some existing research in the literatures has incorporated this concept that fault removal rate （FRR） is different for different types of faults, they assume that the FRR remains constant during the overall testing period. On the contrary, it has been observed that as testing progresses, FRR changes due to changing testing strategy, skill, environment and personnel resources. In this paper, a general discrete SRGM is proposed for errors of different severity in software systems using the change-point concept. Then, the models are formulated for two particular environments. The models were validated on two real-life data sets. The results show better fit and wider applicability of the proposed models as to different types of failure datasets.

A New Analysis Concept in Applying Software Reliability Growth Models and Tool Implementation: The SafeMan

In recent years, many software development organizations have been assessing and analyzing their software product’s reliability/quality and judging whether the software product is releasable by using Software Reliability Growth Models (SRGMs) at the final stage of software development. The usage of SRGMs originates in the advantage that various reliability analysis results based on the SRGMs can be acquired easily. However, it is very difficult for general software project managers to grasp the achievement level of reliability/quality based on its analysis results because some sort of professional knowledge is required in order to understand the information on the attainment progress of software product’s reliability/quality. Moreover, it is also difficult for software project managers and inspectors who do not deeply comprehend the details of their project to evaluate the degree of software reliability and quality, if they assess it without grasping the live development situation and only see the documents submitted from their staff. In this paper, we propose a new analysis concept for assessing the software product’s reliability/quality, and illustrate the output results obtained by a tool, the SafeMan.

Using Artificial Neural-Networks in Stochastic Differential Equations Based Software Reliability Growth Modeling

Due to high cost of fixing failures, safety concerns, and legal liabilities, organizations need to produce software that is highly reliable. Software reliability growth models have been developed by software developers in tracking and measuring the growth of reliability. Most of the Software Reliability Growth Models, which have been proposed, treat the event of software fault detection in the testing and operational phase as a counting process. Moreover, if the size of software system is large, the number of software faults detected during the testing phase becomes large, and the change of the number of faults which are detected and removed through debugging activities becomes sufficiently small compared with the initial fault content at the beginning of the testing phase. Therefore in such a situation, we can model the software fault detection process as a stochastic process with a continuous state space. Recently, Artificial Neural Networks (ANN) have been applied in software reliability growth prediction. In this paper, we propose an ANN based software reliability growth model based on Ito type of stochastic differential equation. The model has been validated, evaluated and compared with other existing NHPP model by applying it on actual failure/fault removal data sets cited from real software development projects. The proposed model integrated with the concept of stochastic differential equation performs comparatively better than the existing NHPP based model.

Incorporating S-shaped testing-effort functions into NHPP software reliability model with imperfect debugging

Testing-effort（TE） and imperfect debugging（ID） in the reliability modeling process may further improve the fitting and prediction results of software reliability growth models（SRGMs）. For describing the S-shaped varying trend of TE increasing rate more accurately, first, two S-shaped testing-effort functions（TEFs）, i.e.,delayed S-shaped TEF（DS-TEF） and inflected S-shaped TEF（IS-TEF）, are proposed. Then these two TEFs are incorporated into various types（exponential-type, delayed S-shaped and inflected S-shaped） of non-homogeneous Poisson process（NHPP）SRGMs with two forms of ID respectively for obtaining a series of new NHPP SRGMs which consider S-shaped TEFs as well as ID. Finally these new SRGMs and several comparison NHPP SRGMs are applied into four real failure data-sets respectively for investigating the fitting and prediction power of these new SRGMs.The experimental results show that：（i） the proposed IS-TEF is more suitable and flexible for describing the consumption of TE than the previous TEFs;（ii） incorporating TEFs into the inflected S-shaped NHPP SRGM may be more effective and appropriate compared with the exponential-type and the delayed S-shaped NHPP SRGMs;（iii） the inflected S-shaped NHPP SRGM considering both IS-TEF and ID yields the most accurate fitting and prediction results than the other comparison NHPP SRGMs.

Web software reliability modeling with random impulsive shocks

As the web-server based business is rapidly developed and popularized, how to evaluate and improve the reliability of web-servers has been extremely important. Although a large num- ber of software reliability growth models （SRGMs）, including those combined with multiple change-points （CPs）, have been available, these conventional SRGMs cannot be directly applied to web soft- ware reliability analysis because of the complex web operational profile. To characterize the web operational profile precisely, it should be realized that the workload of a web server is normally non-homogeneous and often observed with the pattern of random impulsive shocks. A web software reliability model with random im- pulsive shocks and its statistical analysis method are developed. In the proposed model, the web server workload is characterized by a geometric Brownian motion process. Based on a real data set from IIS server logs of ICRMS website （www.icrms.cn）, the proposed model is demonstrated to be powerful for estimating impulsive shocks and web software reliability.

NHPP-based software reliability model considering testing effort and multivariate fault detection rate

In recent decades,many software reliability growth models（SRGMs） have been proposed for the engineers and testers in measuring the software reliability precisely.Most of them is established based on the non-homogeneous Poisson process（NHPP）,and it is proved that the prediction accuracy of such models could be improved by adding the describing of characterization of testing effort.However,some research work indicates that the fault detection rate（FDR） is another key factor affects final software quality.Most early NHPPbased models deal with the FDR as constant or piecewise function,which does not fit the different testing stages well.Thus,this paper first incorporates a multivariate function of FDR,which is bathtub-shaped,into the NHPP-based SRGMs considering testing effort in order to further improve performance.A new model framework is proposed,and a stepwise method is used to apply the framework with real data sets to find the optimal model.Experimental studies show that the obtained new model can provide better performance of fitting and prediction compared with other traditional SRGMs.

A Bivariate Software Reliability Model with Change-Point and Its Applications

Testing-time when a change of a stochastic characteristic of the software failure-occurrence time or software failure-occurrence time-interval is observed is called change-point. It is said that effect of the change-point on the software reliability growth process influences on accuracy for software reliability assessment based on a software reliability growth model (SRGM). We propose an SRGM with the effect of the change-point based on a bivariate SRGM, in which the software reliability growth process is assumed to depend on the testing-time and testing-effort factors simultaneously, for accurate software reliability assessment. And we discuss an optimal software release problem for deriving optimal testing-effort expenditures based on our model. Further, we show numerical examples of software reliability assessment based on our bivariate SRGM and estimation of optimal testing-effort expenditures by using actual data.

Lognormal Process Software Reliability Modeling with Testing-Effort

We propose a software reliability growth model with testing-effort based on a continuous-state space stochastic process, such as a lognormal process, and conduct its goodness-of-fit evaluation. We also discuss a parameter estimation method of our model. Then, we derive several software reliability assessment measures by the probability distribution of its solution process, and compare our model with existing continuous-state space software reliability growth models in terms of the mean square error and the Akaike’s information criterion by using actual fault count data.

Component-based software reliability process simulation considering imperfect debugging

In view of the flaws of component-based software （CBS） reliability modeling and analysis, the low recognition degree of debugging process, too many assumptions and difficulties in obtaining the solution, a CBS reliability simulation process is presented incorporating the imperfect debugging and the limitation of debugging resources. Considering the effect of imperfect debugging on fault detec- tion and correction process, a CBS integration testing model is sketched by multi-queue muhichannel and finite server queuing model （MMFSQM）. Compared with the analytical method based on pa- rameters and other nonparametric approaches, the simulation approach can relax more of the usual reliability modeling assumptions and effectively expound integration testing process of CBS. Then, CBS reliability process simulation procedure is developed accordingly. The proposed simulation ap- proach is validated to be sound and effective by simulation experiment studies and analysis.

基于明考斯基最优距离的不完美排错SRGM决策机制

针对软件可靠性增长模型(software reliability growth model,SRGM)决策机制欠缺的问题,提出了基于明考斯基最优距离的不完美排错SRGM决策方法。首先,对真实软件测试过程与可靠性模型进行分析,建立分类视角下的SRGM归类集合;其次,从矩阵分析的角度形式化描述SRGM的评价与选择问题,建立基于明考斯基距离的双最优化排序方法;最后,对模型的性能进行综合评价,给出SRGM间的性能偏序关系。在4个实际计算机应用系统测试过程中记录并公开发表的失效数据集上进行实验,验证和阐释模型性能之间的排序机制效用,并进行参数敏感性分析,为模型决策提供定量化的参考。

Reliability Growth Modeling and Optimal Release Policy Under Fuzzy Environment of an N-version Programming System Incorporating the Effect of Fault Removal Efficiency

Failure of a safety critical system can lead to big losses. Very high software reliability is required for automating the working of systems such as aircraft controller and nuclear reactor controller software systems. Fault-tolerant softwares are used to increase the overall reliability of software systems. Fault tolerance is achieved using the fault-tolerant schemes such as fault recovery （recovery block scheme）, fault masking （N-version programming （NVP）） or a combination of both （Hybrid scheme）. These softwares incorporate the ability of system survival even on a failure. Many researchers in the field of software engineering have done excellent work to study the reliability of fault-tolerant systems. Most of them consider the stable system reliability. Few attempts have been made in reliability modeling to study the reliability growth for an NVP system. Recently, a model was proposed to analyze the reliability growth of an NVP system incorporating the effect of fault removal efficiency. In this model, a proportion of the number of failures is assumed to be a measure of fault generation while an appropriate measure of fault generation should be the proportion of faults removed. In this paper, we first propose a testing efficiency model incorporating the effect of imperfect fault debugging and error generation. Using this model, a software reliability growth model （SRGM） is developed to model the reliability growth of an NVP system. The proposed model is useful for practical applications and can provide the measures of debugging effectiveness and additional workload or skilled professional required. It is very important for a developer to determine the optimal release time of the software to improve its performance in terms of competition and cost. In this paper, we also formulate the optimal software release time problem for a 3VP system under fuzzy environment and discuss a the fuzzy optimization technique for solving the problem with a numerical illustration.

Early Stage Software Reliability Estimation with Stochastic Reward Nets

This paper presents software reliability modeling issues at the early stage of a software development for fault tolerant software management system. Based on Stochastic Reward Nets, an effective model of hierarchical view for a fault tolerant software management system is put forward, and an approach that consists of system transient performance analysis is adopted. A quantitative approach for software reliability analysis is given. The results show its usefulness for the design and evaluation of the early-stage software reliability modeling when failure data is not available.

A software cost model with maintenance and risk costs for safety-critical systems

According to the consequences of software failures, software faults remaining in safety-critical systems can be classified into two sets： common faults and fatal faults. Common faults cause slight loss when they are activated. A fatal fault can lead to significant loss, and even damage the safety-crltical system entirely when it is activated. A software reliability growth model for safety-critical systems is developed based on G - 0 model. And a software cost model is proposed too. The cost model considers maintenance and risk costs due to software failures. The optimal release policies are discussed to minimize the total software cost. A numerical exampie is provided to illustrate how to use the results we obtained.

Software Reliability Growth Model Considering First-Step and Second-Step Fault Dependency

As one of the most important indexes to evaluate the quality of software, software reliability experiences an increasing development in recent years. We investigate a software reliability growth model(SRGM). The application of this model is to predict the occurrence of the software faults based on the non-homogeneous Poisson process(NHPP). Unlike the independent assumptions in other models, we consider fault dependency. The testing faults are divided into three classes in this model: leading faults, first-step dependent faults and second-step dependent faults. The leading faults occurring independently follow an NHPP, while the first-step dependent faults only become detectable after the related leading faults are detected. The second-step dependent faults can only be detected after the related first-step dependent faults are detected. Then, the combined model is built on the basis of the three sub-processes. Finally, an illustration based on real dataset is presented to verify the proposed model.

SRGM建模类别与性能分析

针对软件可靠性增长模型SRGM(software reliability growth model)在可靠性评估与保障中的重要作用,为全面掌握SRGM的建模与工作机理,对SRGM的典型建模过程以及不同模型间的性能差异进行深入研究.首先剖析了SRGM建模的基础假设和含义,梳理了SRGM的发展演化历程;然后分析了两类基本SRGM建模流程与关联,针对考虑更多真实测试情况的建模趋势,对不完美排错相关与考虑测试工作量TE(Testing-Effort)相关的SRGM建模过程进行了剖析;最后选取8个典型的模型在4个失效数据集上进行实验,依据度量与拟合结果进行了模型差异化的深入分析.研究分析表明,客观上不同失效数据集间的差异以及主观上研究人员对测试过程认知的差异是造成SRGM性能差异的主要根源.进一步建立更为准确与全面的SRGM,在有限的数据集上选取出优秀的SRGM已成为当前研究中亟待解决的难题.

基于系统状态特征分析的软件可靠性评估方法

针对以指挥信息系统为典型的复杂任务型软件试验鉴定领域中可靠性评估问题,在总结现有评价方法的基础上,提出了基于系统状态特征分析的可靠性评估方法。首先,从软件使命任务的角度解析系统的状态特征,设计充分覆盖系统典型应用样式及状态特征的测试用例,通过定义“使用强度”,替代原有可靠性评价中“持续时间”指标,在此基础上收集可靠性失效数据,并给出软件可靠性增长模型(software reliability and growth model,SRGM)参数估计及可靠性评估过程。最后通过某任务型软件的可靠性评估试验,验证提出方法在工程实践中的可行性,同时给出了工程示例中收集的真实失效数据和SRGM参数估计结果,确保研究成果真实性和可复现性。

SRGM下失效数据集效用与验证分析

针对软件可靠性增长模型SRGM研究中的参数拟合与性能评测对失效数据集FDS的依赖,对FDS在SRGM中的效用以及其对SRGM的影响进行深入研究,并给出FDS的不足与发布建议。首先给出了基于FDS的SRGM性能评测流程,提出一般化的不完美排错框架模型,对收集到的FDS进行结构化描述与归类分析。对7个典型的不完美排错相关的SRGM在公开发表的9个真实计算机工程系统FDSs上进行实验,从拟合与预测角度分析FDS与SRGM的关系及影响。从发布方与科研人员视角对当前FDS的不足进行分析,并据此给出了FDS的发布建议。研究结果表明,科研人员尚需要充分挖掘、分析FDS中待发布的更多测试信息,用以建立更为准确的SRGM。最后指出,描述新型软件结构以及含有更多数据量的FDS的缺乏已成为制约SRGM发展的主要客观事实。

一种非参数化SRGM预测能力的研究

提出了一种非参数化软件可靠性增长模型的改进,该模型考虑软件工程的多样性,利用支持向量回归技术并结合机器学习相关知识,完成故障检测和预测过程。通过三组真实的失效数据集的实验,将所改进方法与传统软件可靠性增长模型比较,结果显示,改进的非参数化软件可靠性增长模型具有更好的模型通用性和预测性能。

An Improved NHPP Model with Time-Varying Fault Removal Delay

In this paper, an improved NHPP model is proposed by replacing constant fault removal time with time-varying fault removal delay in NHPP model, proposed by Daniel R Jeske. In our model, a time-dependent delay function is established to fit the fault removal process. By using two sets of practical data, the descriptive and predictive abilities of the improved NHPP model are compared with those of the NHPP model, G-O model, and delayed S-shape model. The results show that the improved model can fit and predict the data better.

A Study of Quantitative Progress Evaluation Models for Open Source Projects

Open source software (OSS) has become an indispensable part of society, not only for personal use but also for corporate use. Projects developed and operated by OSS are called open source projects, and the number of such projects is increasing. On the other hand, because anyone can participate in an open source project, the progress of the project is uncertain due to differences in project members’ skills, development environments, and time zones of activity. Therefore, many users and companies need to understand the development and operation status of open source project. Then, the developers carefully make decisions on upgrading or installing new OSS. In this paper, we focus on the maintenance effort estimation for open source projects considering uncertainty. Also, we evaluate the project quantitatively using Earned Value Management (EVM). Moreover, we examine the appropriateness of the model for predicting the maintenance effort expeditures. Furthermore, we discuss the appropriateness of this EVM method.