Skip to main content

Advertisement

SpringerLink
Log in

Stochastic debugging based reliability growth models for Open Source Software project

  • Original Research
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Open Source Software (OSS) is one of the most trusted technologies for implementing industry 4.0 solutions. The study aims to assist a community of OSS developers in quantifying the product’s reliability. This research proposes reliability growth models for OSS by incorporating dynamicity in the debugging process. For this, stochastic differential equation-based analytical models are developed to represent the instantaneous rate of error generation. The fault introduction rate is modeled using exponential and Erlang distribution functions. The empirical applications of the proposed methodology are verified using the real-life failure data of the Open Source Software projects, GNU Network Object Model Environment, and Eclipse. A soft computing technique, Genetic Algorithm, is applied to estimate model parameters. Cross-validation is also performed to examine the forecasting efficacy of the model. The predictive power of the developed models is compared with various benchmark studies. The data analysis is conducted using the R statistical computing software. The results demonstrate the proposed models’ efficacy in parameter estimation and predictive performance. In addition, the optimal release time policy based on the proposed mathematical models is presented by formulating the optimization model that intends to minimize the total cost of software development under reliability constraints. The numerical illustration and sensitivity analysis exhibit the proposed problem's practical significance. The findings of the numerical analysis exemplify the proposed study's capability of decision-making under uncertainty.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. White Paper: Open Source Software for Industry 4.0—https://iot.eclipse.org/community/resources/white-papers/industry40/ (accessed on 29 December, 2021).

References

  • Aktekin, T., & Caglar, T. (2013). Imperfect debugging in software reliability: A Bayesian approach. European Journal of Operational Research, 227(1), 112–121.

    Article  Google Scholar 

  • Arora, A., Caulkins, J. P., & Telang, R. (2006). Research note—Sell first, fix later: Impact of patching on software quality. Management Sci., 52(3), 465–471.

    Article  Google Scholar 

  • Bhaskar, T., & Kumar, U. D. (2006). A cost model for N-version programming with imperfect debugging. Journal of the Operational Research Society, 57(8), 986–994.

    Article  Google Scholar 

  • Boehret, K. (2010). For the iPad, Apps with their own wow factor. The Wall Street Journal (April 7)

  • Chatterjee, S., & Shukla, A. (2016). Modeling and analysis of software fault detection and correction process through Weibull-type fault reduction factor, change point and imperfect debugging. Arabian Journal for Science and Engineering, 41(12), 5009–5025.

    Article  Google Scholar 

  • Chatterjee, S., & Shukla, A. (2019). A unified approach of testing coverage-based software reliability growth modelling with fault detection probability, imperfect debugging, and change point. Journal of Software: Evolution and Process, 31(3), e2150.

    Google Scholar 

  • Chatterjee, S., Chaudhuri, B., & Bhar, C. (2020). Optimal release time determination in intuitionistic fuzzy environment involving randomized cost budget for SDE-based software reliability growth model. Arabian Journal for Science and Engineering, 45(4), 2721–2741.

    Article  Google Scholar 

  • Chatterjee, S., Chaudhuri, B., & Bhar, C. (2021a). Optimal release time determination via fuzzy goal programming approach for SDE-based software reliability growth model. Soft Computing, 25(5), 3545–3564.

    Article  Google Scholar 

  • Chatterjee, S., Chaudhuri, R., Shah, M., & Maheshwari, P. (2022). Big data driven innovation for sustaining SME supply chain operation in post COVID-19 scenario: Moderating role of SME technology leadership. Computers & Industrial Engineering, 168, 108058.

    Article  Google Scholar 

  • Chatterjee, S., Rana, N. P., Dwivedi, Y. K., & Baabdullah, A. M. (2021b). Understanding AI adoption in manufacturing and production firms using an integrated TAM-TOE model. Technological Forecasting and Social Change, 170, 120880.

    Article  Google Scholar 

  • DiBona, C., & Ockman, S. (1999). Open sources: Voices from the open source revolution. O’Reilly Media, Inc.

    Google Scholar 

  • Dinh-Trong, T. T., & Bieman, J. M. (2005). The FreeBSD project: A replication case study of open source development. IEEE Transactions on Software Engineering, 31(6), 481–494.

    Article  Google Scholar 

  • Fitzgerald, B., & Kenny, T. (2004). Developing an information systems infrastructure with open source software. Ieee Software, 21(1), 50–55.

    Article  Google Scholar 

  • German, D. M. (2003). The GNOME project: A case study of open source, global software development. Software Process: Improvement and Practice, 8(4), 201–215.

    Article  Google Scholar 

  • Goel, A. L. (1985). Software reliability models: Assumptions, limitations, and applicability. IEEE Transactions on Software Engineering, 12, 1411–1423.

    Article  Google Scholar 

  • Goel, A. L., & Okumoto, K. (1979). Time-dependent error-detection rate model for software reliability and other performance measures. IEEE Transactions on Reliability, 28(3), 206–211.

    Article  Google Scholar 

  • Hsu, C. J., & Huang, C. Y. (2010, July). A study on the applicability of modified genetic algorithms for the parameter estimation of software reliability modeling. In 2010 IEEE 34th annual computer software and applications conference (pp. 531–540). IEEE.

  • Huang, C. Y. (2005). Cost-reliability-optimal release policy for software reliability models incorporating improvements in testing efficiency. Journal of Systems and Software, 77(2), 139–155.

    Article  Google Scholar 

  • Huang, C. Y., & Lyu, M. R. (2011). Estimation and analysis of some generalized multiple change-point software reliability models. IEEE Transactions on Reliability, 60(2), 498–514.

    Article  Google Scholar 

  • Huang, Y. S., Chiu, K. C., & Chen, W. M. (2022). A software reliability growth model for imperfect debugging. Journal of Systems and Software, 188, 111267.

    Article  Google Scholar 

  • Inoue, S., & Yamada, S. (2004). Testing-coverage dependent software reliability growth modeling. International Journal of Reliability, Quality and Safety Engineering, 11(04), 303–312.

    Article  Google Scholar 

  • Jiang, Z., Sarkar, S., & Jacob, V. S. (2012). Postrelease testing and software release policy for enterprise-level systems. Information Systems Research, 23(3-part-1), 635–657.

    Article  Google Scholar 

  • Jin, C., & Jin, S. W. (2016). Parameter optimization of software reliability growth model with S-shaped testing-effort function using improved swarm intelligent optimization. Applied Soft Computing, 40, 283–291.

    Article  Google Scholar 

  • Kapur, P. K., & Garg, R. B. (1990). Optimal software release policies for software reliability growth models under imperfect debugging. RAIRO-Operations Research-Recherche Opérationnelle, 24(3), 295–305.

    Article  Google Scholar 

  • Kapur, P. K., & Garg, R. B. (1991). Optimal release policies for software systems with testing effort. International Journal of Systems Science, 22(9), 1563–1571.

    Article  Google Scholar 

  • Kapur, P. K., Anand, S., Yamada, S., & Yadavalli, V. S. (2009). Stochastic differential equation-based flexible software reliability growth model. Mathematical Problems in Engineering

  • Kapur, P. K., Goswami, D. N., Bardhan, A., & Singh, O. (2008). Flexible software reliability growth model with testing effort dependent learning process. Applied Mathematical Modelling, 32(7), 1298–1307.

    Article  Google Scholar 

  • Kapur, P. K., Kumar, D., Gupta, A., & Jha, P. C. (2006). On how to model software reliability growth in the presence of imperfect debugging and error generation. In Proceedings of 2nd international conference on reliability and safety engineering (pp. 515–523).

  • Kapur, P. K., Kumar, S., & Garg, R. B. (1999). Contributions to hardware and software reliability (Vol. 3). World Scientific.

    Google Scholar 

  • Kapur, P. K., Panwar, S., & Kumar, V. (2021). Should software testing continue after release of a software: A new perspective. Handbook of advanced performability engineering (pp. 709–737). Springer.

    Chapter  Google Scholar 

  • Kapur, P. K., Panwar, S., Kumar, V., & Singh, O. (2020). Entropy-based two-dimensional software reliability growth modeling for open-source software incorporating change-point. International Journal of Reliability, Quality and Safety Engineering, 27(05), 2040009.

    Article  Google Scholar 

  • Kapur, P. K., Panwar, S., Singh, O., & Kumar, V. (2019a). Joint optimization of software time-to-market and testing duration using multi-attribute utility theory. Annals of Operations Research, 1–28.

  • Kapur, P. K., Panwar, S., Singh, O., & Kumar, V. (2019b). Joint release and testing stop time policy with testing-effort and change point. Risk based technologies (pp. 209–222). Springer.

    Chapter  Google Scholar 

  • Kapur, P. K., Pham, H., Anand, S., & Yadav, K. (2011a). A unified approach for developing software reliability growth models in the presence of imperfect debugging and error generation. IEEE Transactions on Reliability, 60(1), 331–340.

    Article  Google Scholar 

  • Kapur, P. K., Pham, H., Gupta, A., & Jha, P. C. (2011b). Software reliability assessment with OR applications. Springer.

    Book  Google Scholar 

  • Kapur, P. K., Singh, O., Garmabaki, A. S., & Singh, J. (2010). Multi up-gradation software reliability growth model with imperfect debugging. International Journal of System Assurance Engineering and Management, 1(4), 299–306.

    Google Scholar 

  • Kumar, V., Panwar, S., Kapur, P. K., & Singh, O. (2021). Optimal decisions on software release and post-release testing: A unified approach. Yugoslav Journal of Operations Research, 31(2), 165–180.

  • Kuo, S. Y., Huang, C. Y., & Lyu, M. R. (2001). Framework for modeling software reliability, using various testing-efforts and fault-detection rates. IEEE Transactions on Reliability, 50(3), 310–320.

    Article  Google Scholar 

  • Li, H. F., Li, Q. Y., & Lu, M. Y. (2011a). A software reliability growth model considering an S-shaped testing effort function under imperfect debugging. Journal of Harbin Engineering University, 32(11), 1460–1467.

    Google Scholar 

  • Li, Q., & Pham, H. (2017). NHPP software reliability model considering the uncertainty of operating environments with imperfect debugging and testing coverage. Applied Mathematical Modelling, 51, 68–85.

    Article  Google Scholar 

  • Li, Q., & Pham, H. (2019). A generalized software reliability growth model with consideration of the uncertainty of operating environments. IEEE Access, 7, 84253–84267.

    Article  Google Scholar 

  • Li, X., Li, Y. F., Xie, M., & Ng, S. H. (2011b). Reliability analysis and optimal version-updating for open source software. Information and Software Technology, 53(9), 929–936.

    Article  Google Scholar 

  • Li, X., Xie, M., & Ng, S. H. (2010). Sensitivity analysis of release time of software reliability models incorporating testing effort with multiple change-points. Applied Mathematical Modelling, 34(11), 3560–3570.

    Article  Google Scholar 

  • Malaiya, Y. K., Li, M. N., Bieman, J. M., & Karcich, R. (2002). Software reliability growth with test coverage. IEEE Transactions on Reliability, 51(4), 420–426.

    Article  Google Scholar 

  • Miclo, L., & Del Moral, P. (2001). Genealogies and increasing propagation of chaos for Feynman-Kac and genetic models. The Annals of Applied Probability, 11(4), 1166–1198.

    Article  Google Scholar 

  • Minamino, Y., Sakaguchi, S., Inoue, S., & Yamada, S. (2019). Two-dimensional NHPP models based on several testing-time functions and their applications. International Journal of Reliability, Quality and Safety Engineering, 26(04), 1950018.

    Article  Google Scholar 

  • Mockus, A., Fielding, R. T., & Herbsleb, J. D. (2002). Two case studies of open source software development: Apache and Mozilla. ACM Transactions on Software Engineering and Methodology (TOSEM), 11(3), 309–346.

    Article  Google Scholar 

  • Nagaraju, V., & Fiondella, L. (2020, January). Online Optimal Release Time for Non-homogeneous Poisson Process Software Reliability Growth Model. In 2020 annual reliability and maintainability symposium (RAMS) (pp. 1–6). IEEE.

  • Ohba, M., & Chou, X. M. (1989, May). Does imperfect debugging affect software reliability growth?. In Proceedings of the 11th international conference on Software engineering (pp. 237–244).

  • Ohba, M., & Yamada, S. (1984). S-shaped software reliability growth models. In International colloquium on reliability and maintainability (pp. 430–436), 4th, Tregastel, France.

  • Øksendal, B. (2003). Stochastic differential equations. Stochastic differential equations (pp. 65–84). Springer.

    Chapter  Google Scholar 

  • Pachauri, B., Dhar, J., & Kumar, A. (2015). Incorporating inflection S-shaped fault reduction factor to enhance software reliability growth. Applied Mathematical Modelling, 39(5–6), 1463–1469.

    Article  Google Scholar 

  • Panwar, S., Kumar, V., Kapur, P. K., & Singh, O. (2021). Software reliability prediction and release time management with coverage. International Journal of Quality & Reliability Management. https://doi.org/10.1108/IJQRM-05-2021-0139

    Article  Google Scholar 

  • Peng, R., Li, Y. F., Zhang, W. J., & Hu, Q. P. (2014). Testing effort dependent software reliability model for imperfect debugging process considering both detection and correction. Reliability Engineering & System Safety, 126, 37–43.

    Article  Google Scholar 

  • Pham, H., & Zhang, X. (1997). An NHPP software reliability model and its comparison. International Journal of Reliability, Quality and Safety Engineering, 4(03), 269–282.

    Article  Google Scholar 

  • Pham, H., & Zhang, X. (2003). NHPP software reliability and cost models with testing coverage. European Journal of Operational Research, 145(2), 443–454.

    Article  Google Scholar 

  • Pham, H., Nordmann, L., & Zhang, Z. (1999). A general imperfect-software-debugging model with S-shaped fault-detection rate. IEEE Transactions on Reliability, 48(2), 169–175.

    Article  Google Scholar 

  • Rani, P., & Mahapatra, G. S. (2018). Neural network for software reliability analysis of dynamically weighted NHPP growth models with imperfect debugging. Software Testing, Verification and Reliability, 28(5), e1663.

    Article  Google Scholar 

  • Raymond, E. (1999). The cathedral and the bazaar. Knowledge, Technology & Policy, 12(3), 23–49.

    Article  Google Scholar 

  • Razavi, S., & Gupta, H. V. (2015). What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in Earth and Environmental systems models. Water Resources Research, 51(5), 3070–3092.

    Article  Google Scholar 

  • Roy, P., Mahapatra, G. S., & Dey, K. N. (2014). An NHPP software reliability growth model with imperfect debugging and error generation. International Journal of Reliability, Quality and Safety Engineering, 21(02), 1450008.

    Article  Google Scholar 

  • Schmidt, D. C., & Porter, A. (2001, May). Leveraging open-source communities to improve the quality & performance of open-source software. In Proceedings of the 1st workshop on open source software engineering (Vol. 1).

  • Sharma, S., Sugumaran, V., & Rajagopalan, B. (2002). A framework for creating hybrid-open source software communities. Information Systems Journal, 12(1), 7–25.

    Article  Google Scholar 

  • Singh, O., Panwar, S., & Kapur, P. K. (2020). Determining software time-to-market and testing stop time when release time is a change-point. International Journal of Mathematical, Engineering and Management Sciences, 5(2), 208–224. https://doi.org/10.33889/IJMEMS.2020.5.2.017

    Article  Google Scholar 

  • Singhal, S., Anand, A., & Singh, O. (2020). Studying dynamic market size-based adoption modeling & product diffusion under stochastic environment. Technological Forecasting and Social Change, 161, 120285.

    Article  Google Scholar 

  • Song, K. Y., Chang, I. H., & Pham, H. (2019). A testing coverage model based on NHPP software reliability considering the software operating environment and the sensitivity analysis. Mathematics, 7(5), 450.

    Article  Google Scholar 

  • Tamura, Y., & Yamada, S. (2006). A flexible stochastic differential equation model in distributed development environment. European Journal of Operational Research, 168(1), 143–152.

    Article  Google Scholar 

  • Tamura, Y., & Yamada, S. (2019). Maintenance effort management based on double jump diffusion model for OSS project. Annals of Operations Research, 1–16.

  • Tamura, Y., & Yamada, S. (2021). Performance assessment based on stochastic differential equation and effort data for edge computing. Software Testing, Verification and Reliability, 31, e1766.

    Article  Google Scholar 

  • Tan, L., Liu, C., Li, Z., Wang, X., Zhou, Y., & Zhai, C. (2014). Bug characteristics in open source software. Empirical Software Engineering, 19(6), 1665–1705.

    Article  Google Scholar 

  • Valdivia-Garcia, H., Shihab, E., & Nagappan, M. (2018). Characterizing and predicting blocking bugs in open source projects. Journal of Systems and Software, 143, 44–58.

    Article  Google Scholar 

  • Venkatesan, R., Krishnan, T. V., & Kumar, V. (2004). Evolutionary estimation of macro-level diffusion models using genetic algorithms: An alternative to nonlinear least squares. Marketing Science, 23(3), 451–464.

    Article  Google Scholar 

  • Waal, S. (2010). Release management in open source software projects. Retrieved from July 28, 2022 from http://oss-watch.ac.uk/resources/releasemanagement.

  • Wang, J. (2017). An imperfect software debugging model considering irregular fluctuation of fault introduction rate. Quality Engineering, 29(3), 377–394.

    Article  Google Scholar 

  • Wang, J., & Wu, Z. (2016). Study of the nonlinear imperfect software debugging model. Reliability Engineering & System Safety, 153, 180–192.

    Article  Google Scholar 

  • Wang, J., Wu, Z., Shu, Y., & Zhang, Z. (2015). An imperfect software debugging model considering log-logistic distribution fault content function. Journal of Systems and Software, 100, 167–181.

    Article  Google Scholar 

  • Wu, M. W., & Lin, Y. D. (2001). Open Source software development: An overview. Computer, 34(6), 33–38.

    Article  Google Scholar 

  • Xie, M., & Hong, G. Y. (1998). A study of the sensitivity of software release time. Journal of Systems and Software, 44(2), 163–168.

    Article  Google Scholar 

  • Xie, M., & Yang, B. (2003). A study of the effect of imperfect debugging on software development cost. IEEE Transactions on Software Engineering, 29(5), 471–473.

    Article  Google Scholar 

  • Yaghoobi, T. (2020). Parameter optimization of software reliability models using improved differential evolution algorithm. Mathematics and Computers in Simulation, 177, 46–62.

    Article  Google Scholar 

  • Yamada, S. (2014). Software reliability modeling: Fundamentals and applications (Vol. 5). Springer.

    Book  Google Scholar 

  • Yamada, S., & Osaki, S. (1987). Optimal software release policies with simultaneous cost and reliability requirements. European Journal of Operational Research, 31(1), 46–51.

    Article  Google Scholar 

  • Yamada, S., & Tamura, Y. (2016). OSS reliability measurement and assessment. Springer.

    Book  Google Scholar 

  • Yamada, S., Ohba, M., & Osaki, S. (1983). S-shaped reliability growth modeling for software error detection. IEEE Transactions on Reliability, 32(5), 475–484.

    Article  Google Scholar 

  • Yamada, S., Tokuno, K., & Osaki, S. (1992). Imperfect debugging models with fault introduction rate for software reliability assessment. International Journal of Systems Science, 23(12), 2241–2252.

    Article  Google Scholar 

  • Yi, Z., Wen, Y., & Wu, X. (2019). Impacts of networking effects on software reliability growth processes: A multi-attribute utility theory approach. Quality and Reliability Engineering International, 35(6), 1952–1972.

    Article  Google Scholar 

  • Zhang, J., Lu, Y., Yang, S., & Xu, C. (2016). NHPP-based software reliability model considering testing effort and multivariate fault detection rate. Journal of Systems Engineering and Electronics, 27(1), 260–270.

    Google Scholar 

  • Zhang, X. M., Li, Q. W., Liu, Z., & Chang, C. T. (2021). Optimal pricing and remanufacturing mode in a closed-loop supply chain of WEEE under government fund policy. Computers & Industrial Engineering, 151, 106951.

    Article  Google Scholar 

  • Zhang, X., Teng, X., & Pham, H. (2003). Considering fault removal efficiency in software reliability assessment. IEEE Transactions on Systems, Man, and Cybernetics-Part a: Systems and Humans, 33(1), 114–120.

    Article  Google Scholar 

  • Zhao, J., Liu, H. W., Cui, G., & Yang, X. Z. (2006). Software reliability growth model with change-point and environmental function. Journal of Systems and Software, 79(11), 1578–1587.

    Article  Google Scholar 

  • Zhao, M., & Xie, M. (1993, November). Robustness of optimum software release policies. In Proceedings of 1993 IEEE international symposium on software reliability engineering (pp. 218–225). IEEE.

  • Zhu, M., & Pham, H. (2018). A two-phase software reliability modeling involving with software fault dependency and imperfect fault removal. Computer Languages, Systems & Structures, 53, 27–42.

    Article  Google Scholar 

  • Zhu, M., & Pham, H. (2020). A generalized multiple environmental factors software reliability model with stochastic fault detection process. Annals of Operations Research, 1–22.

Download references

Author information

Authors and Affiliations

  1. Fortune Institute of International Business (FIIB), New Delhi, Delhi, India

    Shakshi Singhal

  2. Amity Center for Interdisciplinary Research, Amity University, Uttar Pradesh, Noida, India

    P. K. Kapur

  3. Department of Operational Research, University of Delhi, New Delhi, Delhi, India

    Vivek Kumar & Saurabh Panwar

Authors
  1. Shakshi Singhal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. K. Kapur
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vivek Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saurabh Panwar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saurabh Panwar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singhal, S., Kapur, P.K., Kumar, V. et al. Stochastic debugging based reliability growth models for Open Source Software project. Ann Oper Res (2023). https://doi.org/10.1007/s10479-023-05240-6

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10479-023-05240-6

Keywords

Search

Navigation