1 Introduction

Software design is one of the prime areas of research for developers in this emerging technology era in both hardware and software. Software design comprises a few well-defined phases, including fault detection, bug fixing, and on-time reliable delivery. The inherent limitation of software development is a maintenance cost, which may be substantially high compared to the total development cost. In case of failure, the software returns nothing but all the cost is directly converted to penalty. Another significant issue is that software development has a few quantifiable measures for accounting. Most of the well-defined software metrics result in categorical data that may be utilized to assess software quality.

Programming complexity and completeness are critical estimations to describe the quality and decide a few questions concerning testing. It includes questions like when to quit testing, what is more, to test, which module has high coupling, the bug occurrence pattern. The objective is to release software as early as possible without identifying bugs in the system, and all stated targets are achieved [1]. Several programming driven models have been given in the literature to decide the release time, which may depend upon a non-homogeneous Poisson process (NHPP) [2], or consider proper investigation for targets defined some uses defect amplification model. The fault reduction factor(FRF) given by Musa [3] gives the fraction of the number of faults reduced to the number of faults that occurred, and the execution time is related to the amount of FRF. To assess the quality of software and measure its efficiency before release is one of the prime objectives of software reliability modeling. Reliability metrics are majorly dependent upon how to use allocated system resources for testing. Several other deterministic factors for estimating software reliability include programmer efficiency, interdependency between software modules, learning process, fault occurrence probability, and debugging time lag. Another major constraint for software release estimation is cost. Most of the software estimates move around cost. Delay in any of the predefined set of software development processes leads to an increase in cost, which is not required and may lead to software failure.

Concerning open source development, software development has its issues because it adopts a simple mechanism "release early and often release" [4]. This mechanism requires fast development and release with minimum desirable functionalities that may be extended as and when required. This software comes in small bundles whose functionalities may be expanded or replaced easily with each update. The development of open-source software follows either feature-driven release or duration specific release. Release management in open source is a complex process dependent upon users' and developers' procedures. The empirical study of open-source software may extract several qualitative features. This study's few attributes are the organization's background developing the software, the process adopted for software development, inter-organizational arrangements, and software tools. Still, release management is considered as an under-researched area in the case of open-source software.

Software reliability estimation is essential because it helps in resource utilization, cost management, effective time management, and fault prediction. The primary research objective is to develop a reliability estimation model that utilizes bug detection and prediction of bug occurrence, which follows a few sub-processes, including initial screening of bugs, pre-processing, sampling, and the regression model, and the correlation between actual and predicted bugs. The "Least square estimation" and "Maximum likelihood estimation" techniques are widely used for non-linear regression. Software failures are considered stochastic, resulting in errors being customarily distributed, and the collected sample set is large.

The motivations of this paper are described as follows. Firstly, some works [9, 10, 13, 15] have integrated the complexity of environments or a single factor in the creation of the model, but with random application environments they do not represent a generalised SRM. Secondly, environmental variables were not clearly specified, such as the amount of programming effort, the number of team members, the organisation of programmers, human nature, the testing environment, meeting hours, software complexity, design methodology. We also proposed a generalised multiple-environmental-factors SRM with multiple environmental factors in view of certain critical impacts.

The remainder of this paper is structured into parts. In Sect. 2, we discuss work carried out in the past on the estimation of software reliability. A brief analysis is carried out in this section, highlighting the primary techniques proposed in different articles presented before. Section 3 discusses assumptions and proposed software reliability. The description of the proposed algorithm and the data set description are carried out in Sect. 4. In Sect. 5, a comparison with similar algorithms is carried out to analyze the proposed model's impact based upon several performance metrics. Finally, Sect. 6 discussion on the conclusions is given.

2 Related work

The authors of [5] present an S-shaped model that ensures reliability and intonation in a software project. The work carried out in this paper gives flexibility in estimating the release time of software and the results clearly show that the above model is entirely applicable for real-time projects. A complex fuzzy network is used in [6] for determining software release metrics. The major estimators are projected to release time and success rate. It uses a fuzzy network along with its series and parallel node and branch architecture. To accurately determine completion time, self back loops, probability nodes, and reversible loops were considered. Software reliability is studied in [7], and uncertainty is assessed using varying resource allocation. It was suggested that most of the existing models consider time, cost, or quality as primary factors for determining reliability. They devised a solution to implement a trade-off among these factors. Defect removal and bug identification are included to provide robustness in the proposed algorithm.

The authors of [2] researched "blast furnace smelting of Magnitogorsk Iron and Steel Works." Testing is identified as a critical approach to software success. With the integration of existing and new modules, unit testing is key to achieving success. They worked upon the tools and techniques used during regression testing in case of changes done after installation. The proposed solution results in a low cost of maintenance, thereby increasing the delivered product's reliability and scalability. Software reliability study is carried out in [8] in a different way, focusing on the "power law" software testing function. They highlighted faults as the main concern for the testing process. Open-source software is selected to validate the proposed approach with the introduction of multigenerational faults. The software release time problem is identified, and a solution about the same is presented using the cost of software testing. Experiments are carried out using software failure data, and results are compared with Goel-Okumoto and S-shaped model [9, 10]. The work given in [2] focuses on modularity. They used different mathematical equations such as Fokker-Plank and Petri net for reliability study in open-source software. The presented approach is a comparative analysis between testing and design strategies adopted to make a reliable software product. A network technology-based routing software technique is implemented in this paper.

The prediction model is built and analyzed for imposing security in software in [11]. The paper presents a detailed analysis of vulnerabilities and potential risks in software development. Machine learning is applied for estimating time to fix an error. Another prediction technique is applied in [12] to provide a reliable software model that reduces faults in designing software. The paper identifies fault reduction factor (FRF) as a primary source of reliability growth. The authors suggest key factors behind FRF and develop a function that estimates the time-dependent value of FRF. By using the approach mentioned above, the authors prove that it enhances accuracy and provides reliability. The authors of [13] identify various factors that affect software reliability, such as software complexity, organizational policies, capabilities, and software design proficiency. The paper suggests a genetic algorithm-based approach for determining faults in software design. Various performance metrics are compared using the proposed model, and it outperforms several existing algorithms in terms of risk and prediction accuracy.

The authors of [14] suggest that removing risk at any stage of software development may result in new risks. The solution may be software release, including some low probable risks that might not negatively affect software performance. These leftover risks may be further taken into consideration during the maintenance phase. The authors carried out a comparative analysis for these leftover risks that may be dealt with before or after release. The book published by the authors of [15] provides a case study of various release modes of software in open source systems. They suggested that most of the developers prefer time-based software release in comparison to feature driven software release.

3 Proposed approach

3.1 Assumptions

A general assumption for most of the existing software reliability models is that fault is independent and, on detection is can be flawlessly removed. Nevertheless, it is not always correct because of several factors, including programmer skills and expertise, software complexity, organizational hierarchy. A software time-variant reliability model (SRM) based non-homogeneous Poisson process (NHPP) is presented in this paper. The Time variant fault detection SRM deliberates the detection of fault/failure reliant and inadequate fault/failure elimination process and considers the number of faults/failures based upon their extreme count. The working of a SRM in flowchart is illustrated in Fig. 1.

Fig. 1
figure 1

Workflow of Software reliability model

Full size image

The following are the assumptions for this proposed Software reliability time-variant model based on NHPP:

  • The software failure practice is an NHPP.

  • Fault/failure detection is a progressive curve based upon learning-spectacle.

  • Fault cannot be flawlessly upon detection.

  • The proposed approach is based on fault/failure detection.

  • The debugging process can also announce new faults in the software. There can be a process that leads to imperfect debugging, but the number of faults limiting the software reliability is restricted to L.

  • Each fault is independent and correspondingly likely to cause a failure during the testing of the program.

  • The software failure intensity \(\lambda (t)\) is enlightened about the proportion of errors that were successfully removed in the software product.

  • The software error rate that is not rectified is assumed to be constant.

  • The software contains \(N\) faults at the initial level, which are assumed to be unknown but fixed.

  • The time-intervals among incidences of faults are also independent of each other.

3.2 Software reliability model representation

Let \({t}_{i}\) be the time between \(\left( {i - 1} \right){\text{th}} and i\;{\text{th}}\) failures in the program. The total number of software failures at any time instance \({t}_{i}\) is be notated as \(N({t}_{i})\) and depends on the non-homogeneous Poisson process (NHPP).

The estimated number of software failures until time \({t}_{i}\) is called the mean value function (MVF) and denoted by \(m\left( {t_{i} } \right)\). φ is notated as a proportional constant, which defines anyone's fault makes to the whole program. At the i failure interval, the program failure rate is given by \(\lambda \left( {t_{i} } \right)\) \(\lambda ({t}_{i})\), and L defines the maximum number of faults a software can contain.

Let \(\left( {t_{i} } \right)\), \(\beta \left( {t_{i} } \right)\), and \(\gamma \left( {t_{i} } \right)\) defines the fault content function at time instance \(t_{i}\), the rate at which software fault can be detected per time unit, and the rate of un-removed error per unit time.

The Software reliability function by time \({t}_{i}\) given a mission time \(t_{i - 1}\) is given by \(R(t_{i - 1} |t_{i}\)).

Numerous reliability models prove that \(\lambda \left( t \right)\) is directly proportional to the number of faults left out in the software in most NHPP models. Furthermore, the proposed models ensure that software results in independent faults that may be rectified entirely upon detection. Consequently, we may model generalized NHPP software MVF, which considers time-variant fault/failure detection rate is obtained by solving the following proposed differential equation (with the initial condition \(m\left( {t_{0} } \right) ! = 0\))

$$\frac{{\partial m\left( {t_{i} } \right)}}{{\partial t_{i} }} = m\left( {t_{i} } \right) \times \beta \left( {t_{i} } \right)*\left( {1 - \frac{{m\left( {t_{i} } \right)}}{L}} \right) - m\left( {t_{i} } \right) \times \gamma \left( {t_{i} } \right)$$
(1)

In this work, we have considered time-variant fault detection rate \(\beta ({t}_{i})\) as follow:

$$\beta \left( {t_{i} } \right) = \frac{\beta }{{\left( {1 + \alpha \left( {t_{i} } \right) \times e^{ - \beta t} } \right)}}$$
(2)

The resolution of the expected number of software faults detected by time \({t}_{i}\), i.e., mean value function (MVF) of the differential equation in Eq. (1) considering the time-variant fault detection rate \(\beta \left( {t_{i} } \right)\) in Eq. (2), can be attained as follows:

$$m\left( {t_{i} } \right) = \frac{L}{{\begin{array}{*{20}c} {1 + \varphi \times \frac{{\beta \left( {t_{i} } \right) + 1}}{{\beta \left( {t_{i} } \right) + e^{\beta t} }}} \\ \end{array} }}$$
(3)

To make it simple and more understandable, we named the time-variant fault-detection reliability model. From Eq. (3),\(m\left( {t_{i} } \right) = \frac{L}{{\begin{array}{*{20}c} {1 + \varphi } \\ \end{array} }}\) when the value of \(t_{i} = 0\), and \(m\left( {t_{i} } \right) = L\) when \(t_{i} = \infty\).

4 Performance evaluation

4.1 System platform:

The proposed time-variant fault detection software reliability model is implemented in Jira [2] as it supports web services likes Representational State Transfer (REST), Simple Object Access Protocol (SOAP), and Extensible Markup Language- Remote Procedure Call (XML-RPC) for the integration of various sources control programs such as Clear case, Mercurial, Concurrent Versions System (CVS), Perforce, Subversion, and Team Foundation Server. Jira designed and developed in Java and procedures the Pico inversion of control container, Apache OFBiz entity engine, and WebWork.

Four packages are commonly used with JIRA, i.e., Core, Software, Service Desk, and Ops. The product set JIRA is a software project tool specially designed for project management and bug tracking. It allows Atlassian to develops the management of products and services using agile methodology and the complete package. The given tool provides flexibility in the execution where the software or a product code can be executed from anywhere. A variety of languages can be supported by this tool, including Japanese, English, Spanish, German, and French. In this context, where multiple languages can be supported by this tool, it becomes compelling and can be considered a multi-lingual tool. This tool's scope is not limited to bug tracking; it also includes various testing methodologies, incident management, and a complete set of software design and workflow management tasks. The software provides an additional set of customized tools that add to its flexibility and usability. JIRA supports MySQL, PostgreSQL Oracle, and SQL Server in the backend. The handling of issues is made easy by JIRA's dashboard, as it has many features and functionalities. JIRA allows tracking the progress of the project from time to time.

The dataset given in Table 1 contains software attributes data, which is obtained for 74 software projects. Data is collected over a period, which is initiated with the start of the project and is continuously monitored until its completion. Several parameters are analyzed, and a collection of 115 attributes are given with 11 different time frames in terms of product and process characteristics. A total of 383 persons were involved in software design, where the issue count total is 255. This dataset can be used to predict software release. The attribute collection is focused on qualitative and quantitative features during software development. This dataset gives deliverables about individual and team activity. The overall quality of the software design, requirements, and specifications are monitored continuously. It contains the analysis of bugs encountered, countermeasures taken, meeting statistics, time efficiency, team performance, leadership impact, gender-sensitive information, programmer proficiency. Figure 2 shows the dataset Description over Range of different attributes for the given time frame and Fig. 3 illustrates the Pair-plot of the dataset concerning team member response count, non-coding deliverables hours, coding deliverables hours, and total issue count.

Table 1 List of Attributes selected from the dataset
Full size table
Fig. 2
figure 2

Dataset Description–Range of different attributes for the given time frame a Distribution of Total issue count with each time frame b Distribution of nonCodingDeliverableHours with each time frame c Distribution of codingDeliverableHours with each time frame d Distribution of teamMemberResponseCount with each time frame

Full size image
Fig. 3
figure 3

Pair-plot of the dataset concerning team member response count, non-coding deliverables hours, coding deliverables hours, and total issue count

Full size image

4.2 Performance metrics

Various performance metrics are measured to assess the proposed software reliability time-variant fault detection model's effectiveness and accuracy. The proposed technique has been evaluated by comparing with Infection-S-shaped, Delayed S-shaped, and Goel–Okumoto (G–O) model. The performance metrics which needs to be evaluated to check the validity and performance of the proposed software reliability time-variant fault detection technique are:

4.2.1 Mean Square Error (MSE)

It gives an estimate of deviation, measured by finding out the square of the error term. The outcome of MSE is utilized as an indicator of accuracy evaluated for predicting any unseen faults/failures. The above metric is evaluated by the formula given in Eq. 4, which uses the errors' squares. An error term is measured by finding out the symmetric difference between the actual and the estimated value. MSE is used as a default metric for performance evaluation in most regression algorithms implemented in MATLAB, R, or python.

$$MSE = \frac{1}{N - n}\mathop \sum \limits_{i = 1}^{N} \left( {m\left( {t_{i} } \right) - y_{i} } \right)^{2}$$
(4)

The total number of observations and the number of parameters for evaluation is given by N and n, respectively. The term \(m\left({t}_{i}\right)\) is the MVF by the software reliability model, which estimates the number of faults expected, detected by time \({t}_{i}\), and \({y}_{i}\) is the observation value.

4.2.2 Root Mean Square Error (RMSE)

It gives an estimate of deviation, measured by finding out the square root of the MSE term. The normalized values of RMSE are obtained using prediction model values and actual values. RMSE is utilized for finding out residuals and its standard deviation. The graph depicting residuals gives the deviation of the regression line from actual data points. It also gives the variation and shows the scattered nature of residuals concerning each data point's best fit.

$$MSE = \sqrt {\mathop \sum \limits_{i = 1}^{N} \frac{{m\left( {t_{i} } \right) - y_{i} )^{2} }}{N - n}}$$
(5)

The total number of observations and the number of parameters for evaluation is given by N and n, respectively. The term \(m\left( {t_{i} } \right)\) is the MVF by the software reliability model, which estimates the expected numbers of fault detected by time ti, and \({y}_{i}\) is the observation value.

4.2.3 R-squared (R2)

The goodness-of-fit measure is frequently used for linear regression models. This parameter gives the statistic that indicates the variance ratio calculated with the dependent and independent variables' help. R-squared measures' strength and effectiveness lie with the value that models the relationship between the proposed indicator and the dependent variable on a convenient scale of 0 to 100.R2is the normalized version of MSE for reporting as it is a simple metric and the loss function we are minimizing when we solve the standard equation.

$$R^{2} = 1 - \frac{{\mathop \sum \nolimits_{i = 1}^{N} \left( {m\left( {t_{i} } \right) - y_{i} } \right)^{2} }}{{\mathop \sum \nolimits_{i = 1}^{N} \left( {m\left( {t_{i} } \right) - \overline{{y_{i} }} } \right)^{2} }}$$
(6)

\(m\left( {t_{i} } \right)\)The total number of observations and the number of parameters for evaluation is given by N and n, respectively. The term \(m\left( {t_{i} } \right)\) is the MVF by the software reliability model, which estimates the expected numbers of fault detected by time \({t}_{i}\) and \({y}_{i}\) is the observation value and \(\stackrel{-}{{y}_{i}}\) is the mean of the observed value.

5 Results and discussion

We apply the time-variant software reliability model to obtain the proposed time-variant fault detection model's parameter estimates with Infection-S-shaped, Delayed S-shaped, and Goel–Okumoto (G–O) models. These models are implemented in JIRA as already discussed in Sect. 4.1. The validity, effectiveness, and goodness of fit can be expressed by evaluating MSE, RMSE, and R-squared criteria as described above in Sect. 4.2.

The proposed time-variant fault detection model fulfills the condition \(0 < m\left( {t_{0} } \right) \le y_{1}\),at \(t = 0\), by the number of faults present initially, and \(y_{1}\) is the total instances of observed failures \(t = 1\). The model initializes \(m\left( {t_{0} } \right)\) with an integer value. The integer value is selected based on a desirable constraint that a software tester uses for pre-analysis. The above procedure is performed to eliminate errors present in the software, whose impact is negligible in the complete software's performance. All those errors are trivial and must be ignored before the software moves to the formal testing phase. These insignificant errors are considered to appear due to unintentional humane mistakes or some other issues related to software configuration and management. The objective is to consider the total number of faults having an extreme impact on the software. These errors are present in the modeling phase itself and may be removed as early as possible. All the trivial errors may be assumed to be a part of these total numbers of faults dealt during a complete set of modeling and may be ignored at initial phase fault detection using the proposed scheme.

The results of the parameters estimated by model execution and performance parameters (MSE, RMSE, and R-squared) corresponding to each of the compared algorithms are listed in Table 2. The values given below are the outcome of execution for the proposed model and other existing models. The system test data represents Infection-S-shaped. Therefore, it cannot seamlessly capture the characteristics of Delayed S-shaped and Goel–Okumoto (G–O) models.

Table 2 Comparison and parameter estimation of proposed software reliability and other existing technique
Full size table

For the system test data, the estimated parameters are \(\alpha = 13.45\), \(\beta = 0.18\),and \(\gamma = 0.34\). As observed in Table 2, the MSE, RMSE, and R-squared values for the proposed software reliability model are 0.51, 0.714, and 0.86, respectively, which are the minimum among all four models presented here. The estimated values for the Inflection S-shaped model are \(\alpha = 28.34\), \(\beta = 0.24\) and \(\gamma = 0.63\). However, the value of MSE, RMSE, and R-squared are 1.33.1.532 and 0.53, respectively, relatively low than the Delayed S-shaped and Goel–Okumoto (G–O) models. Though both these models cannot seamlessly capture the characteristics of the proposed software reliability model. The estimated parameter for Delayed S-shaped is \(\alpha = 47.64\) and \(\beta = 0.26\), which is slightly higher than that of the Inflection S-shaped model. The MSE, RMSE, and R-squared are 2.07, 1.438, and 0.32, respectively. The RMSE and R-square value of the Delayed S-shaped model are relatively less than the Inflection S-shaped model. The estimated parameter value for Goel–Okumoto (G–O) is \(\alpha = 70.67\) and \(\beta = 0.35\). The performance parameter values of MSE, RMSE, and R-squared are 6.65, 2.578, and 0.09, respectively. Based on the above analysis, the conclusion may be drawn that the proposed software reliability time-variant fault detection model performs better (compared to other existing models) and can be stated as the best fit for the given system.

Figure 4 illustrates the evaluation of cumulative failures (actual)with cumulative failures (predicted). Figure 5 depicts the comparison of MSE, RMSE, and R-squared parameters by four software reliability models.

Fig. 4
figure 4

Comparison of predicted failures (cumulative)

Full size image
Fig. 5
figure 5

Comparison of MSE, RMSE, and R-squared parameters

Full size image

6 Conclusion and future direction

In this paper, we introduced the Time-variant fault detection Software Reliability Model that includes different software attributes for model building and compares the same with several well-known algorithms upon different performance metrics. The proposed model is compared with Infection-S-Shaped, Delayed-S-Shaped, and Goel-Okumoto model in terms of cumulative failures, mean square error, and root mean square error parameters. The proposed model works upon a non-homogeneous Poisson process and incorporates fault dependent detection and software failure intensity and un-removed error in the software. We considered the programmer's proficiency, software complexity, organization hierarchy, and perfect debugging as the determining factors for presenting the software reliability model. This novel work is carried out, ensuring time-varying fault detection, measured by considering response count, coding and non-coding deliverables, and bugs in the software. The findings show that the new model proposed has substantially better goodness-of-fit and predicts better value than the other compared models. The proposed model shows improvement over existing algorithms as the residual errors are reduced, and prediction accuracy is high in cumulative fault detection.

The proposed approach's future work may include defining exact time boundaries for software release and assessing factors that need to be addressed for the software's multi-release. Future work will involve broader validation of this conclusion based on recent data sets.