Skip to main content

Advertisement

SpringerLink
Log in

Testing coverage based software reliability assessment incorporating effort expenditure and error generation

  • Theoretical Article
  • Published:
OPSEARCH Aims and scope Submit manuscript

Abstract

Due to the complexity of software systems, the testing team may be unable to completely eliminate a problem after observing a failure and the participation of numerous external elements in their development, and another fault may replace the identified fault, also known as error generation. Software developers and customers can benefit from test coverage as a software statistic since it can assist them improve the functionality of tested software and identify what additional effort is necessary to increase the reliability of the software. Software reliability growth models (SRGMs) have been cited as one of the effective methods for quantitative evaluation of software quality. The concept of effort spent and error generation are integrated into a model for testing coverage-based software reliability evaluation. Testing effort spent is supposed to follow the Weibull distribution, whereas testing coverage is described by exponential, delayed S-shaped, and logistic functions respectively. Additionally, we look into the cost requirement-based software release time for exponential functions with a reliability constraint. We introduce the genetic algorithm, which is a powerful tool for dealing with search and optimization issues. Two real failure datasets have been used to test different goodness of fit criteria for the model and their performance is evaluated using four goodness-of-fit metrics, including coefficient of determination \(\left({\mathrm{R}}^{2}\right)\), mean square error (MSE), predictive power (PP) and predictive ratio risk (PRR). The acquired results outperform the perfect debugging model and demonstrate notable advancements that are fairly encouraging.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Arora, R., Aggarwal, A.: Testing effort based software reliability assessment incorporating FRF and change point. Yugoslav J. Op. Res. 30(3), 273–288 (2020). https://doi.org/10.2298/YJOR190315022A

    Article  Google Scholar 

  2. Brooks, W., Motley, R.: Analysis of Discrete Software Reliability Models. IBM Federal Systems Division, Gaithersburg (1980)

    Book  Google Scholar 

  3. Chatterjee, S., Singh, J.B.: A NHPP based software reliability model and optimal release policy with logistic–exponential test coverage under imperfect debugging. Int. J. Syst. Assur. Eng. Manag. 5(3), 399–406 (2014). https://doi.org/10.1007/s13198-013-0181-6

    Article  Google Scholar 

  4. Chatterjee, S., Shukla, A.: A unified approach of testing coverage-based software reliability growth modelling with fault detection probability, imperfect debugging, and change point. J. Softw. Evolut. Process 31(3), e2150 (2019). https://doi.org/10.1002/smr.2150

    Article  Google Scholar 

  5. Chaudhary, S., Sinha, A., Singh, D., Mishra, S.: A review on software realibility growth modelling. Int. J. Eng. Res. Technol. 8(10), 113–116 (2020)

    Google Scholar 

  6. Huang, C.-Y., Lo, J.-H., Kuo, S.-Y., Lyu, M. R.: Optimal allocation of testing resources for modular software systems. In: 13th International Symposium on Software Reliability Engineering, 2002. Proceedings., 129–138. https://doi.org/10.1109/ISSRE.2002.1173228(2002)

  7. Gupta, R., Jain, M., Jain, A.: (2019) Software Reliability Growth Model in Distributed Environment Subject to Debugging Time Lag (pp. 105–118). https://doi.org/10.1007/978-981-13-0857-4_7

  8. Huang, C.-Y.: Performance analysis of software reliability growth models with testing-effort and change-point. J. Syst. Softw. 76(2), 181–194 (2005). https://doi.org/10.1016/j.jss.2004.04.024

    Article  Google Scholar 

  9. Huang, Y.S., Chiu, K.C., Chen, W.M.: A software reliability growth model for imperfect debugging. J. Syst. Softw. 188, 111267 (2022). https://doi.org/10.1016/j.jss.2022.111267

    Article  Google Scholar 

  10. Inoue, S., Yamada, S.: Two-dimensional software reliability assessment with testing-coverage. Second Int. Conf. Secur. Syst. Integr. Reliab. Improvem. 2008, 150–157 (2008). https://doi.org/10.1109/SSIRI.2008.21

    Article  Google Scholar 

  11. Jain, M., Jain, A., Gupta, R.: Analysis of Module-Based Software Reliability Growth Model Incorporating Imperfect Debugging and Fault Reduction Factor (pp. 69–80). https://doi.org/10.1007/978-981-10-5577-5_6 (2018)

  12. Kapur, P.K., Garg, R.B.: Optimum release policy for an inflection s-shaped software reliability growth model. Microelectron. Reliab. 31(1), 39–41 (1991). https://doi.org/10.1016/0026-2714(91)90343-6

    Article  Google Scholar 

  13. Kapur, P.K., Garg, R.B., Aggarwal, A.G., Tandon, A.: General framework for change point problem in software reliability and related release time problem. Int. J. Reliab. Qual. Saf. Eng. 16(06), 567–579 (2009). https://doi.org/10.1142/S0218539309003587

    Article  Google Scholar 

  14. Kapur, P.K., Gupta, A., Shatnawi, O., Yadavalli, V.S.S.: Testing effort control using flexible software reliability growth model with change point. Int. J. Perform. Eng. 2(3), 245–263 (2006)

    Google Scholar 

  15. Kapur, P.K., Pham, H., Aggarwal, A.G., Kaur, G.: Two dimensional multi-release software reliability modeling and optimal release planning. IEEE Trans. Reliab. 61(3), 758–768 (2012). https://doi.org/10.1109/TR.2012.2207531

    Article  Google Scholar 

  16. Kapur, P.K., Shatnawi, O., Aggarwal, A.G., Kumar, R.: Unified framework for developing testing effort dependent software reliability growth models. WSEAS Trans. Syst. 8(4), 521–531 (2009)

    Google Scholar 

  17. Khurshid, S., Shrivastava, A.K., Iqbal, J.: Effort based software reliability model with fault reduction factor, change point and imperfect debugging. Int. J. Inf. Technol. 13(1), 331–340 (2021). https://doi.org/10.1007/s41870-019-00286-x

    Article  Google Scholar 

  18. Kumar, V., Kapur, P. K., Sahni, R., Shrivastava, A. K.: Testing Time and Effort-Based Successive Release Modeling of a Software in the Presence of Imperfect Debugging (pp. 421–434). https://doi.org/10.1007/978-981-10-5577-5_33 (2018)

  19. Li, Q., Pham, H.: NHPP software reliability model considering the uncertainty of operating environments with imperfect debugging and testing coverage. Appl. Math. Model. 51, 68–85 (2017). https://doi.org/10.1016/j.apm.2017.06.034

    Article  Google Scholar 

  20. Malaiya, Y.K., Li, M.N., Bieman, J.M., Karcich, R.: Software reliability growth with test coverage. IEEE Trans. Reliab. 51(4), 420–426 (2002). https://doi.org/10.1109/TR.2002.804489

    Article  Google Scholar 

  21. Panwar, S., Kumar, V., Kapur, P.K., Singh, O.: Software reliability prediction and release time management with coverage. Int. J. Quality Reliab. Manag. 39(3), 741–761 (2022). https://doi.org/10.1108/IJQRM-05-2021-0139

    Article  Google Scholar 

  22. Peng, R., Li, Y.F., Zhang, W.J., Hu, Q.P.: Testing effort dependent software reliability model for imperfect debugging process considering both detection and correction. Reliab. Eng. Syst. Saf. 126, 37–43 (2014). https://doi.org/10.1016/j.ress.2014.01.004

    Article  Google Scholar 

  23. Pham, H.: Loglog fault-detection rate and testing coverage software reliability models subject to random environments. Vietnam J. Comput. Sci. 1(1), 39–45 (2014). https://doi.org/10.1007/s40595-013-0003-4

    Article  Google Scholar 

  24. Pham, H., Zhang, X.: NHPP software reliability and cost models with testing coverage. Eur. J. Oper. Res. 145(2), 443–454 (2003). https://doi.org/10.1016/S0377-2217(02)00181-9

    Article  Google Scholar 

  25. Song, K.Y., Chang, I.H., Pham, H.: A testing coverage model based on NHPP software reliability considering the software operating environment and the sensitivity analysis. Mathematics 7(450), 1–21 (2019). https://doi.org/10.3390/math7050450

    Article  Google Scholar 

  26. Tandon, A., Aggarwal, A.G., Nijhawan, N.: An NHPP SRGM with change point and multiple releases. Int. J. Inform. Syst. Service Sector 8(4), 57–68 (2016). https://doi.org/10.4018/IJISSS.2016100105

    Article  Google Scholar 

  27. Neha, T.A., Aggarwal, A.G.: Testing coverage based reliability modeling for multi-release open-source software incorporating fault reduction factor. Life Cycle Reliab. Safety Eng. 9(4), 425–435 (2020). https://doi.org/10.1007/s41872-020-00148-7

    Article  Google Scholar 

  28. Verma, V., Anand, S., Kapur, P.K., Aggarwal, A.G.: Unified framework to assess software reliability and determine optimal release time in presence of fault reduction factor, error generation and fault removal efficiency. Int. J. Syst. Assur. Eng. Manag. (2022). https://doi.org/10.1007/s13198-022-01653-x

    Article  Google Scholar 

  29. Wood, A.: Predicting software reliability. Computer 29(11), 69–77 (1996). https://doi.org/10.1109/2.544240

    Article  Google Scholar 

  30. Yamada, S., Ohtera, H., Narihisa, H.: A testing-effort dependent software reliability model and its application. Microelectron. Reliab. 27(3), 507–522 (1987). https://doi.org/10.1016/0026-2714(87)90469-0

    Article  Google Scholar 

  31. Zhu, M., Pham, H.: Environmental factors analysis and comparison affecting software reliability in development of multi-release software. J. Syst. Softw. 132, 72–84 (2017). https://doi.org/10.1016/J.JSS.2017.05.097

    Article  Google Scholar 

Download references

Acknowledgements

The author (Sudeep Kumar) gratefully acknowledges the funding support provided by the Council of Scientific and Industrial Research (CSIR), New Delhi (File no. 09/915(0018)/2019-EMR-I).

Author information

Authors and Affiliations

  1. AIAS, Amity University Uttar Pradesh, Noida, 201303, India

    Sudeep Kumar

  2. Department of Operational Research, University of Delhi, Delhi, India

    Anu G. Aggarwal

  3. T A Pai Management Institute, Manipal, Karnataka, 576104, India

    Ritu Gupta

Authors
  1. Sudeep Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anu G. Aggarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ritu Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sudeep Kumar.

Ethics declarations

Conflict of interest

There are no conflicts of interest to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, S., Aggarwal, A.G. & Gupta, R. Testing coverage based software reliability assessment incorporating effort expenditure and error generation. OPSEARCH 60, 1888–1901 (2023). https://doi.org/10.1007/s12597-023-00680-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12597-023-00680-x

Keywords

Search

Navigation