Skip to main content
SpringerLink
Log in

Optimal stopping time in software testing based on failure size approach

  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

When to stop testing is an important aspect in software test management. This paper discusses a new approach to this problem. There are two major improvements in this approach in comparison to the earlier studies. It uses failure size proportional model to estimate the number of failures and the corresponding cost of repair. Instead of reliability it uses a new measure called attained failure size. Failure size is the probability of finding a failure causing input in the input domain. Attained failure size is the failure size at the termination of testing. The advantages of these improvements are also discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Abbreviations

SR:

Software reliability

NHPP:

Non homogeneous Poisson process

References

  • Boland, P. J., Singh, H., & Cukic, B. (2003). Comparing partition and random testing via. Majorization and Schur Functions. IEEE Transactions on Software Engineering, 29(1), 88–93.

    Article  Google Scholar 

  • Chatterjee, S., Misra, R. B., & Alam, S. S. (1997). Joint effect of test effort and learning factor on software reliability and optimal release policy. International Journal of System Science, 28(4), 391–396.

    Article  Google Scholar 

  • Chen, T. Y., & Yu, Y. T. (1994). On the relationship between partition and random testing. IEEE Transactions on Software Engineering, 20(12), 997–980.

    Article  Google Scholar 

  • Dalal, S. R., & Mallows, C. L. (1988). When should one stop testing software. JASA, 83(403), 872–879.

    Article  Google Scholar 

  • Dalal, S. R., & Mallows, C. L. (2008). Optimal stopping with exact confidence on remaining defects. Technometrics, 50(3), 397–406.

    Article  Google Scholar 

  • Dalal, S. R., & McIntosh, A. A. (1994). When to stop testing for large software systems with changing code. IEEE Transactions on Software Engineering, 20(4), 318–323.

    Article  Google Scholar 

  • Frankl, P. G., Hamlet, R. G., Littlewood, B., & Strigini, L. (1998). Evaluating testing methods by delivered reliability. IEEE Transactions on Software Engineering, 24(8), 586–601.

    Article  Google Scholar 

  • Fakhre-Zakeri, I., & Slud, E. (1995). Mixture models for reliability of software with imperfect debugging: Identifiably of parameters. IEEE Transactions on Reliability, 44(1), 104–113.

    Article  Google Scholar 

  • Goel, L., & Okumoto, K. (1979). Time-dependent error-detection rate model for software reliability and other performance measures. IEEE Transactions on Reliability, R–28(3), 206–211.

    Article  Google Scholar 

  • Hou, R., & Chang, Y. (1997). Optimal release times for software systems with scheduled delivery time based on HGDM. IEEE Transactions on Computers, 46(2), 216–221.

    Article  Google Scholar 

  • Hu, Q. P., Peng, R., Xie, M., Ng, S. H., & Levitin, G. (2011). Software reliability modelling and optimization for multi-release software development processes. 2011 IEEE international conference on Industrial Engineering and Engineering Management (IEEM) (pp. 1534–1538).

  • Jelinski, Z., & Moranda, P. B. (1972). Software reliability research. In W. Freiberger (Ed.), Statistical computer performance evaluation (pp. 465–484). New York: Academic.

    Google Scholar 

  • Kimura, M., & Fujiwara, T. (2010). Practical optimal software release decision making by bootstrap moving-average quality control chart. International Journal of Software Engineering and Its Applications, 4(1), 29–41.

    Google Scholar 

  • Moranda, P. B. (1975). Prediction of software reliability and its applications. Proceedings of the annual reliability and maintainability symposium (pp. 327–332). Washington, DC.

  • Musa, J. D., & Okumoto, K. (1984). A logarithmic poisson execution time model for software reliability measurement. In Proceedings of the 7th international conference on software Engineering (pp. 230–237).

  • Ntafos, S. C. (2001). On comparisons of random, partition and proportional partition testing. IEEE Transactions on Software Engineering, 27(10), 949–960.

    Article  Google Scholar 

  • Okumoto, K., & Goal, A. L. (1980). Optimum release time for software systems based on reliability and cost criteria. Journal of Systems and Software, 1, 315–318.

    Article  Google Scholar 

  • Ping, C., Zhao, D., & Ke, L. (2010). An optimal release policy for software testing process. In 29th Chinese Control Conference (CCC) (pp. 6037–6042).

  • Singpurwalla, N. D. (1991). Determining the optimal time interval for testing and debugging software. IEEE Transactions on Software Engineering, 17(4), 313–319.

    Article  Google Scholar 

  • Srivastava, P. R. (2012). Optimal software release using time and cost benefits via fuzzy multi-criteria and fault tolerance. Journal of Information Processing Systems, 8(1), 21–54.

    Article  Google Scholar 

  • Stutzke, M. A., & Smidts, C. S. (2001). A stochastic model for fault introduction and removal during software development. IEEE Transactions on Reliability, 50(2), 184–193.

    Article  Google Scholar 

  • Trachtenberg, M. (1990). A general theory of software reliability modeling. IEEE Transactions on Reliability, 39(1), 92–96.

    Article  Google Scholar 

  • Weyuker, E. J., & Jeng, B. (1991). Analyzing partition testing strategies. IEEE Transactions on Software Engineering, 179(7), 703–711.

    Article  Google Scholar 

  • Xie, M., & Yang, B. (2003). A study of the effect of imperfect debugging on software development cost. IEEE Transactions on Software Engineering, 299(5), 471–473.

    Google Scholar 

  • Yamada, S., & Ohtera, H. (1990). Software reliability growth models for testing effort control. European Journal of Operational Research, 46(3), 343–349.

    Article  Google Scholar 

  • Yang, M. C. K., & Chao, A. (1995). Reliability-estimation and stopping- rules for software testing, based on repeated appearances of bugs. IEEE Transactions on Reliability, 44(2), 315–321.

    Article  Google Scholar 

  • Yun, W. Y., & Bai, D. S. (1990). Optimum software release policy with random life cycle. IEEE Transactions on Reliability, 39(2), 167–170.

    Article  Google Scholar 

  • Zachariah, B., & Rattihalli, R. N. (2007). Failure size proportional models and an analysis of failure detection abilities of software testing strategies. IEEE Transactions on Reliability, 56(2), 246–253.

    Article  Google Scholar 

  • Zachariah, B. (2012). Analysis of software testing strategies through attained failure size. IEEE Transactions on Reliability, 61(2), 569–579.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chh. Shahu Institute of Business Education and Research, University Road, Kolhapur, 416004, India

    Babu Zachariah

Authors
  1. Babu Zachariah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Babu Zachariah.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zachariah, B. Optimal stopping time in software testing based on failure size approach. Ann Oper Res 235, 771–784 (2015). https://doi.org/10.1007/s10479-015-1959-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-015-1959-5

Keywords

Search

Navigation