The Use of Exploratory Data Analysis Techniques for Software Reliability Assessment and Prediction

  • Tony Bendell
Part of the NATO ASI Series book series (volume 22)


As is still largely the case with hardware, much of current software reliability prediction is based upon established models which are largely black-box in nature. The structure of these models reflect supposed scientific observation of the physical processes of software failure, together with the conventional desire for modelling brevity. It is, unfortunately, the case that such models, if automatically applied, may neglect important but unknown structure in the failure data, and may squeeze out of the data much of its information content. For this reason such model validation as may subsequently be used, may not be effective.

In this paper the author introduces the nature of exploratory data analysis (e.d.a.), and discusses its use for software reliability assessment. Emphasis is on searching all the available data, in its greatest generality, for pattern to exploit in its analysis. As well as more basic techniques, the use of time series analysis and proportional hazards modelling in this context will be discussed. Examples will be presented of the types of results obtained from the application of e.d.a. to software reliability data and, in particular, of the discovery of structure ignored by established models.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Akiyama, A. (1972). An example of software system debugging, Information Processing 71, North Holland, pp 353–359.Google Scholar
  2. Ascher, H.E. and Feingold, H. (1984). Repairable Systems Reliability: Modelling, Inference, Misconceptions and Their Causes, Marcel Dekker, New York.Google Scholar
  3. Bendell, A. (1935a). Proportional hazards modelling in reliability assessment, Reliability Engineering, 11, 175–183.CrossRefGoogle Scholar
  4. Bendell, A. (1985b). Minimising misconceived models. IOS International Conference on Statistical Modelling, Cambridge. To appear in the Statistician.Google Scholar
  5. Bendell, A. and Samson, W.B. U985). Software quality and reliability, State of the Art Report on the Software Process, Pergamon Infotech.Google Scholar
  6. Bendell, A. and Walls, L.A. (1985a). Exploring reliability data, Qual. and Rel. Eng. Int., 1, 35–51.Google Scholar
  7. Bendell A. and Walls, L.A. (1985b). Multivariate Methods in Reliability — A Case Study concerning Fire Detectors. RSS Multivariate Study Group Workshop.Google Scholar
  8. Boehm, B.W., Brown, B. and Lipow, M. (1976). Quantitative evaluation of software quality, Proc. Int. Conf. on Software Eng., 592–605.Google Scholar
  9. Box, B.J. and Jenkins, G. (1976). Time Series Analysis, Holder Day.MATHGoogle Scholar
  10. Buckland, W.R. (1976). Reliability of nuclear power plants; statistical techniques for analysis, Task Force on Problems of Rare Events in the Reliability of Nuclear Power Plants, JRC ISPRA 8-10 June 1976; CSNI Report No. 10.Google Scholar
  11. Crow, L.H. and Singpurwalla, N.D. (1984). An empirically derived Fourier series model for describing software failures. IEEE Trans. Rel., R-33, 176–183.CrossRefGoogle Scholar
  12. Dale, C.J. (1982). Software Reliability Evaluation Methods, British Aerospace Dynamics Group, Report No. ST26750.Google Scholar
  13. Dale, C.J. (1983). Application of the proportional hazards model in the reliability field. Proc. 4th Nat. Rel. Conf., Birmingham, 5B/1/1–9.Google Scholar
  14. Dale, C.J. and Harris, L.N. (1982). Approaches to software reliability prediction, Proc. Ann. Rel. Maint. Syrnp.Google Scholar
  15. Jardine, A.K.S. and Anderson, M. (1984). Use of concomitant variables for reliability estimations and setting component replacement policies, Proc. 8th ARTS, B3/2/1–6.Google Scholar
  16. Jelinski, Z. and Moranda, P.B. (1972). Software reliability research, In W. Freiberger (Ed.) Statistical Computer Performance Evaluation, Academic Press, pp 465–484.Google Scholar
  17. Kalbfleisch, J.D. and Prentice, R.L. (1980). The Statistical Analysis of Failure Time Data, Wiley,New York.MATHGoogle Scholar
  18. Keiller, D.A. and Littlewood, B. (1984). Adaptive software reliability modelling, FTCS-14.Google Scholar
  19. Lawless, J.F. (1982). Statistical Models and Methods for Lifetime Data, Wiley, New York.MATHGoogle Scholar
  20. Lipow, M. and Thayer, T.A. (1977). Prediction of software failures, Proc. Ann. Rel. Maint. Symp., 489–494.Google Scholar
  21. Littlewood, B. (1981). Stochastic reliability growth: a model for fault removal in computer programs and hardware designs, IEEE Trans. Rel., R-30, 313–320.CrossRefGoogle Scholar
  22. Littlewood, B. (1985). Software reliability prediction. Proc. NATO Advanced Study Institute on the Challenge of Advanced Computing Technology to System Design Methods, Springer-Verlag.Google Scholar
  23. Littlewood, B. and Verrall, J.L. (1973). A Bayesian reliability growth model for computer software, Applied Statistics, 22, 332–346.CrossRefMathSciNetGoogle Scholar
  24. McCall, J.A., Richards, F.R. and Walters, G.F. (1977). Factors in Software Quality, Report No. RADC-TR-77-369, Rome Air Development Centre (3 vols.)Google Scholar
  25. Motley, R.W. and Brooks, W.D. (1977). Statistical Prediction of Programming Errors, USAF Report RADC-TR-77-175.Google Scholar
  26. Musa, J.D. (1975). A theory of software reliability and its application. IEEE Trans. Software Eng., SE-1, 312–327.Google Scholar
  27. Musa, J.D. (1980). Software Reliability Data Submitted to the DACS, Bell Telephone Labs.Google Scholar
  28. Nagel, P.M. and Skrivan, J.A. (1981). Software Reliability: Repetitive Run Experimentation and Modelling, Boeng Computer Services Co. Report BCS-40366, NASA Report No. CR-165836.Google Scholar
  29. Singpurwalla, N.D. (1978). Time series analysis of failure data, Proc. Ann. Rel. Maint. Symp., 107–112.Google Scholar
  30. Singpurwalla, N.D. (1980). Analysing availability using transfer function models and cross spectral analysis, Nav. Res. Logist. Qtly., 27, 1–16.CrossRefMATHGoogle Scholar
  31. Thayer, T.A., Lipow, M. and Nelson, W. (1978). Software Reliability (Vol. No. 2 of TRW Series of Software Technology), North Holland.Google Scholar
  32. Tukey, J.W. (1977). Exploratory Data Analysis, Addison-Wesley, New York.MATHGoogle Scholar
  33. Walls, L.A. and Bendell, A. (1985). The structure and exploration of reliability field data; what to look for and how to analyse it, Proc. 5th Nat. Rel. Conf., Birmingham, 5B/5/1–18.Google Scholar
  34. Walters, G.F. and McCall, J.A. (1978). The development of metrics for software R & M, Proc. Ann. Rel. Maint. Symp., 79–85.Google Scholar
  35. Wightman, D.W. and Bendell, A. (1985). The practical application of proportional hazards modelling, Proc. 5th Nat. Rel. Conf., Birmingham, 2B/3/1–16.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1986

Authors and Affiliations

  • Tony Bendell
    • 1
    • 2
  1. 1.Statistics and Operational ResearchTrent PolytechnicNottinghamEngland
  2. 2.Services Ltd.UK

Personalised recommendations