Skip to main content
SpringerLink
Log in
Book cover

Electronic Systems Effectiveness and Life Cycle Costing pp 441–460Cite as

On the Quality of Software Reliability Prediction

  • Conference paper

Part of the book series: NATO ASI Series ((NATO ASI F,volume 3))

Abstract

We suggest that users are interested solely in the quality of predictions which can be obtained from software reliability models. Some ways of analysing the quality of predictions are proposed and several models and inference procedures are compared on real software failure data sets. We conclude that some predictions are extremely poor: notably those arising from ML analysis of the Jelinski-Moranda model. Others seem quite good. We suggest promising areas for future work.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. B. Littlewood, “How to measure software reliability and how not to”,IEEE Trans. Reliability, Vol R-28, 1979 June, pp. 103–110.

    Article  Google Scholar 

  2. Z. Jelinski and P.B. Moranda, “Software reliability research”, inStatistical Computer Performance Evaluation(W. Freiberger, ed.). New York: Academic Press, 1972, pp. 465–484.

    Google Scholar 

  3. M. Shooman, “Operational testing and software reliability during program development”, inRecord, 1973 IEEE Symp. Computer Software Reliability(New York, NY, 1973, April 30-May 2) pp. 51–57.

    Google Scholar 

  4. J.D. Musa, “A theory of software reliability and its application”,IEEE Trans.Software Engineering, Vol SE-1, 1975 Sept, pp. 312–327.

    Google Scholar 

  5. M.B. Kline, “Software and hardware R & M: what are the differences?”Proc. 1980 R & M Symp. (San Francisco, Jan 22–24, 1980) pp. 179–185.

    Google Scholar 

  6. A.K. Goel and K. Okumoto, “Bayesian software prediction models, Vol 1: An imperfect debugging model for reliability and other quantitative measures of software systems”, RADC-TR-78–155, Rome Air Development Center, NY, 1978.

    Google Scholar 

  7. A.K. Goel, “Software error detection model with applications”,J. Systems and Software, 1, 1980, pp. 243–249.

    Article  Google Scholar 

  8. B. Littlewood and J.L. Verrall, “A Bayesian reliability growth model for computer software”,J. Royal Statist. Soc., C(Applied Statistics), 22, 1973, pp. 332–346.

    MathSciNet  Google Scholar 

  9. B. Littlewood, “What makes a reliable program: few bugs or a small failure rate?”Proc. 1980 National Computer Conference, AFIPS Press, Arlington, VA, 1980, pp. 707–713.

    Google Scholar 

  10. R.E. Barlow and F. Proschan,Statistical Theory of Reliability and Life Testing,Probability Models, Holt, Rinehart and Winston, New York, 1975.

    MATH  Google Scholar 

  11. N.R. Mann, R.E. Schafer and N.D. Singpurwalla,Methods for Statistical Analysis of Reliability and Life Data, Wiley, New York, 1974.

    MATH  Google Scholar 

  12. B. Littlewood, “Software reliability model for modular program structure”,IEEE Trans. Reliability, Vol R-28, No.3, August 1979, pp. 241–246.

    Google Scholar 

  13. M. Shooman, “Structural models for software reliability prediction”, inProc.2nd Intern. Conf. Software Engineering(San Francisco, CA; 1976 Oct] pp. 268–280.

    Google Scholar 

  14. J.P. Buzen, P.P. Chen and R.P. Goldberg, “Virtual machine techniques for improving system reliability”,Record, 1973 IEEE Symp. Computer Software Reliability, (New York, April 30-May 2, 1973) pp. 12–17.

    Google Scholar 

  15. B. Randell, P.A. Lee and P.C. Treleaven, “Reliability issues in computing system design”,Computing Surveys, Vol.10, No.2, June 1978, pp. 123–165.

    Article  MATH  Google Scholar 

  16. A. Avizienis and L. Chen, “On the implementation of N-version programming for software fault-tolerance during program execution”,Proc. COMPSAC 1977(Chicago, 1977) pp. 149–155.

    Google Scholar 

  17. H. Ascher and H. Feingold, “Repairable systems reliability: Probabilistic modelling and statistical analysis. Misconceptions and their causes”, unpublished manuscript (author address: NRL, code 1434, Washington DC 20375).

    Google Scholar 

  18. G. Campbell and K.O. Ott, “Statistical evaluation of major human errors during the development of new technological systems”,Nuclear Science and Engineering,71, 1979, pp. 267–279.

    Google Scholar 

  19. L.H. Crow, “Confidence interval procedures for reliability growth analysis”, Tech. Report No.197, US Army Material Systems Analysis Activity, Aberdeen, Md., 1977.

    Google Scholar 

  20. G.J. Schick and R.W. Wolverton, “An analysis of competing software reliability models”,IEEE Trans. Software Eng., Vol. SE-4, 1978, pp. 104–120. (see also A.L. Goel,IEEE Trans. Software Eng., Vol. SE-6, 1980, pp. 501–2).

    Google Scholar 

  21. R. Troy and R. Moawad, “Assessment of software reliability models”, unpublished manuscript (author address: Agence de 1’Informatique, 7 Avenue du Colonel Roche, 31400 Toulouse, France).

    Google Scholar 

  22. J.D. Musa and K. Okumoto, “Software reliability models: concepts classifications, comparisons and practice” (in these proceedings).

    Google Scholar 

  23. J. Aitchison and I.R. Dunsmore,Statistical Prediction Analysis, Cambridge University Press, Cambridge, 1975.

    Book  MATH  Google Scholar 

  24. D.R. Cox and P.A.W. Lewis,Statistical Analysis of Series of Events, Methuen, London, 1966.

    MATH  Google Scholar 

  25. P.M. Nagel and J.A. Skrivan, “Software reliability: repetitive run experimentation and modelling”, BCS-40399, Boeing Computer Services Company, Seattle, Washington, December 1981.

    Google Scholar 

  26. B. Littlewood and J.L. Verrall, “On the likelihood function of a debugging model for computer software reliability”,IEEE Trans. Reliability, Vol. R-30, 1981 June, pp. 145–148.

    Article  MATH  Google Scholar 

  27. E.H. Forman, “Statistical models and methods for measuring software reliability”, D.Sc. dissertation, SEAS, George Washington U., Washington DC, 1974.

    Google Scholar 

  28. E.H. Forman and N.D. Singpurwalla, “An empirical stopping rule for debugging and testing computer software”,J. Amer. Statist. Assoc, Vol. 72, 1977 Dec, pp. 750–757.

    Article  Google Scholar 

  29. B. Littlewood and A. Sofer, “A Bayesian modification to the Jelinski-Moranda software reliability growth model”, unpublished manuscript (author address: Maths. Dept., City University, Northampton Square, London, EC1V 0HB).

    Google Scholar 

  30. J.D. Musa, “Validation of the execution time theory of software reliability”,IEEE Trans. Reliability, Vol. R-28, No.3, 1979, pp. 181–191.

    Article  MATH  Google Scholar 

  31. M. Dyer and H.D. Mills, “Developing electronic systems with certifiable reliability” (in these proceedings).

    Google Scholar 

  32. B. Littlewood, “Stochastic reliability growth: a model for fault-removal in computer programs and hardware designs”,IEEE Trans. Reliability, Vol. R-30, 4, Oct. 1981, pp. 313–320.

    Article  MathSciNet  Google Scholar 

  33. J.D. Musa, “Software reliability data”, report available from Data and Analysis Center for Software, Rome Air Development Center, Rome, NY.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IBM, USA

    P. A. Keiller

  2. City University, London, UK

    B. Littlewood

  3. George Washington University, USA

    D. R. Miller & A. Sofer

Authors
  1. P. A. Keiller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. B. Littlewood
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. R. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. Sofer
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Theoretical Support Services, Marconi Research Laboratories, GEC Marconi Electronics Ltd., Chelmsford, Essex, CM2 8HN, UK

    J. K. Skwirzynski (Manager, Director of the Institute) (Manager, Director of the Institute)

Rights and permissions

Reprints and permissions

Copyright information

© 1983 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Keiller, P.A., Littlewood, B., Miller, D.R., Sofer, A. (1983). On the Quality of Software Reliability Prediction. In: Skwirzynski, J.K. (eds) Electronic Systems Effectiveness and Life Cycle Costing. NATO ASI Series, vol 3. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-82014-4_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-82014-4_24

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-82016-8

  • Online ISBN: 978-3-642-82014-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation