Acceptance Criteria for Critical Software Based on Testability Estimates and Test Results

  • Antonia Bertolino
  • Lorenzo Strigini
Conference paper


Testability is defined as the probability that a program will fail a test, conditional on the program containing some fault. In this paper, we show that statements about the testability of a program can be more simply described in terms of assumptions on the probability distribution of the failure intensity of the program. We can thus state general acceptance conditions in clear mathematical terms using Bayesian inference. We develop two scenarios, one for software for which the reliability requirements are that the software must be completely fault-free, and another for requirements stated as an upper bound on the acceptable failure probability.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Musa JD. Operational profiles in software-reliability engineering. IEEE Software 1993; March: 14–32.Google Scholar
  2. [2]
    Parnas DL, van Schouwen AJ, Kwan SP. Evaluation of safety-critical software. Communications of the ACM 1990; 33: 636–648.CrossRefGoogle Scholar
  3. [3]
    Miller KW, Morell LJ, Noonan RE, et al. Estimating the probability of failure when testing reveals no failures. IEEE Transactions on Software Engineering 1992; 18: 33–43.CrossRefGoogle Scholar
  4. [4]
    Littlewood B, Strigini L. Validation of ultra-high dependability for software-based systems. Communications of the ACM 1993; 36: 69–80.CrossRefGoogle Scholar
  5. [5]
    Butler RW, Finelli GB. The infeasibility of experimental quantification of life-critical software reliability. In Proc. ACM Conference on Software for Critical Systems, in ACM SIGSOFT Software Eng. Notes, Vol. 16 (5). New Orleans, Louisiana, 1991, pp 66–76.Google Scholar
  6. [6]
    Hamlet D, Voas J. Faults on its sleeve: amplifying software reliability testing. In Proc. 1993 Int. Symposium on Software Testing and Analysis (ISSTA), in ACM SIGSOFT Software Eng. Notes, Vol. 18 (3). Cambridge, Massachusetts, U.S.A., 1993, pp 89–98.Google Scholar
  7. [7]
    Voas JM, Miller KW. Improving the software development process using testability research. In Proc. of the Third Int. Symposium on Software Reliability Engineering. 1992, pp 114–121.Google Scholar
  8. [8]
    Voas JM, Michael CC, Miller KW. Confidently assessing a zero probability of software failure. In Proc. SAFECOMP ’93 12th International Conference on Computer Safety, Reliability and Security. Poznan-Kiekrz, Poland, 1993, pp 197–206.Google Scholar
  9. [9]
    Voas JM, Michael CC, Miller KW. Confidently assessing a zero probability of software failure. High Integrity Systems 1995; 1: 269–275.Google Scholar
  10. [10]
    Voas JM, Miller KW. Software testability: The new verification. IEEE Software 1995; May: 17–28.Google Scholar
  11. [11]
    Bertolino A, Strigini L. On the use of testability measures for dependability assessment. IEEE Transactions on Software Engineering 1996; 22: 97–108.CrossRefGoogle Scholar
  12. [12]
    Bertolino A, Strigini L. Predicting software reliability from testing taking into account other knowledge about a program. In Proc. Quality Week ’96. San Francisco, 1996.Google Scholar
  13. [13]
    Bertolino A, Strigini L. Is it more convenient to assess a probability of failure or of correctness? Submitted for publication 1996.Google Scholar
  14. [14]
    Kahnemann D, Slovic P, Tversky A (ed). Judgment under uncertainty: heuristics and biases. Cambridge University Press, 1982.Google Scholar
  15. [15]
    Strigini L. Engineering judgement in reliability and safety and its limits: what can we learn from research in psychology? SHIP project Technical Report T/030, July, 1994.Google Scholar
  16. [16]
    Littlewood B, Wright D. On a stopping rule for the operational testing of safety critical software. In Proc. FTCS25 (25th Annual International Symposium on Fault -Tolerant Computing). Pasadena, 1995, pp 444–451.Google Scholar

Copyright information

© Springer-Verlag London Limited 1997

Authors and Affiliations

  • Antonia Bertolino
    • 1
  • Lorenzo Strigini
    • 2
  1. 1.Istituto di Elaborazione della InformazioneCNRPisaItaly
  2. 2.Centre for Software ReliabilityCity UniversityLondonUK

Personalised recommendations