A replicated empirical study of a selection method for software reliability growth models

  • Carina Andersson


Replications are commonly considered to be important contributions to investigate the generality of empirical studies. By replicating an original study it may be shown that the results are either valid or invalid in another context, outside the specific environment in which the original study was launched. The results of the replicated study show how much confidence we could possibly have in the original study. We present a replication of a method for selecting software reliability growth models to decide whether to stop testing and release software. We applied the selection method in an empirical study, conducted in a different development environment than the original study. The results of the replication study show that with the changed values of stability and curve fit, the selection method works well on the empirical system test data available, i.e., the method was applicable in an environment that was different from the original one. The application of the SRGMs to failures during functional testing resulted in predictions with low relative error, thus providing a useful approach in giving good estimates of the total number of failures to expect during functional testing.


Replication Software reliability 



The author would like to thank Prof. Catherine Stringfellow for being generous with her time and willing to answer my questions about the selection method. Thanks also to Prof. Anneliese Amschler Andrews and Prof. Per Runeson who provided valuable comments on the paper.


  1. Ehrlich W, Lee S, Molisanim R (1990) Applying reliability measurement: a case study. IEEE Softw 7(2):56–64CrossRefGoogle Scholar
  2. Ehrlich W, Prasanna B, Stanpfel J, Wu J (1993) Determining the cost of a stop-test decision. IEEE Softw 10(2):33–42CrossRefGoogle Scholar
  3. Fenton NE, Pfleeger SL (1997) Software metrics: a rigorous and practical approach, 2nd edn. PWS Publishing Company, BostonGoogle Scholar
  4. Fujiwara T, Yamada S (2003) A testing-domain-dependent software reliability growth model for imperfect debugging environment and its evaluation of goodness-of-fit. Elec Commun Jap Part 3 86(1):11–18CrossRefGoogle Scholar
  5. Gaudoin O, Yang B, Xie M (2003) A simple goodness-of-fit test for the power-law process, based on the Duane plot. IEEE Trans Reliab 52(1):69–74CrossRefGoogle Scholar
  6. Goel AL, Okumoto L (1979) A time dependent error detection rate model for software reliability and other performance measures. IEEE Trans Reliab 28(3):206–211zbMATHGoogle Scholar
  7. Huang CY (2005) Cost-reliability-optimal release policy for software reliability models incorporating improvements in testing efficiency. J Syst Softw 77(2):139–155CrossRefGoogle Scholar
  8. Institute of Electrical and Electronics Engineers (1990) IEEE standard glossary of software engineering terminology, IEEE Std 610.12-1990Google Scholar
  9. International Standards Organisation (2000) Information technology—software product evaluation—quality characteristics and guidelines for their use, ISO/IEC FDIS 9126-1. Geneva, SwitzerlandGoogle Scholar
  10. Jeske DR, Zhang X (2005) Some successful approaches to software reliability modeling in industry. J Syst Softw 74(1):85–99CrossRefGoogle Scholar
  11. Kececioglu D (1991) Reliability engineering handbook, vol. 2. Prentice-Hall, Englewood Cliffs, NJGoogle Scholar
  12. Lyu MR (ed) (1996) Handbook of software reliability engineering. McGraw-Hill, New YorkGoogle Scholar
  13. Miller J (2005) Replicating software engineering experiments: a poisoned chalice or the Holy Grail. Inf Softw Technol 47(4):233–244CrossRefGoogle Scholar
  14. Montgomery DC (2001) Design and analysis of experiments, 5th edn. Wiley, New YorkGoogle Scholar
  15. Musa J (1999) Software reliability engineering. McGraw-Hill, New YorkGoogle Scholar
  16. Musa J, Ackerman A (1989) Quantifying software validation: when to stop testing? IEEE Softw 6(3):19–27CrossRefGoogle Scholar
  17. Musa J, Iannino A, Okumoto L (1987) Software reliability measurement, prediction, application. McGraw-Hill, New YorkGoogle Scholar
  18. Robson C (2002) Real world research. Blackwell Publishers, UKGoogle Scholar
  19. Siegel S, Castellan NJ (1988) Nonparametric statistics for the behavioral sciences. McGraw-Hill, SingaporeGoogle Scholar
  20. Stringfellow C (2000) An integrated method for improving testing effectiveness and efficiency. PhD Dissertation, Colorado State UniversityGoogle Scholar
  21. Stringfellow C, Amschler Andrews A (2002) An empirical method for selecting software reliability growth models. Empir Softw Eng 7(4):319–343zbMATHCrossRefGoogle Scholar
  22. Wood A (1996) Predicting software reliability. IEEE Comput 29(11):69–78Google Scholar
  23. Wood A (1997) Software reliability growth models: assumptions vs. reality. Proceedings of the Eighth International Symposium on Software Reliability Engineering, pp136–141Google Scholar
  24. Yamada S, Ohba M, Osaki S (1983) S-shaped reliability growth modeling for software error detection. IEEE Trans Reliab 32(5):475–478CrossRefGoogle Scholar
  25. Yamada S, Ohtera H, Narihisa H (1986) Software reliability growth models with testing effort. IEEE Trans Reliab 35(1):19–23CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2006

Authors and Affiliations

  1. 1.Department of Communication SystemsLund UniversityLundSweden

Personalised recommendations