On the Effectiveness of Run-Time Checks

  • Meine J. P. van der Meulen
  • Lorenzo Strigini
  • Miguel A. Revilla
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3688)


Run-time checks are often assumed to be a cost-effective way of improving the dependability of software components, by checking required properties of their outputs and flagging an output as incorrect if it fails the check. However, evaluating how effective they are going to be in a future application is difficult, since the effectiveness of a check depends on the unknown faults of the program to which it is applied. A programming contest, providing thousands of programs written to the same specifications, gives us the opportunity to systematically test run-time checks to observe statistics of their effects on actual programs. In these examples, run-time checks turn out to be most effective for unreliable programs. For more reliable programs, the benefit is relatively low as compared to the gain that can be achieved by other (more expensive) measures, most notably multiple-version diversity.


Equivalence Class False Alarm Software Reliability Prime Time Plausibility Check 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blum, M., Wasserman, H.: Software reliability via run-time result-checking. Technical Report TR-94-053, International Computer Science Institute (October 1994)Google Scholar
  2. 2.
    Jhumka, A., Gärtner, F.C., Fetzer, C., Suri, N.: On systematic design of fast and perfect detectors. Technical Report 200263, École Polytechnique Fédérale de Lausanne (EPFDL), School of Computer and Communication Sciences (September 2002)Google Scholar
  3. 3.
    Lee, P.A., Anderson, T.: Fault Tolerance; Principles and Practice, 2nd edn. Dependable Computing and Fault-Tolerant Systems, vol. 3. Springer, Heidelberg (1981)Google Scholar
  4. 4.
    Leveson, N.G., Cha, S.S., Knight, J.C., Shimeall, T.J.: The use of self checks and voting in software error detection: An empirical study. IEEE Transactions on Software Engineering 16(4), 432–443 (1990)CrossRefGoogle Scholar
  5. 5.
    Meyer, B.: Design by contract. Computer (IEEE) 25(10), 40–51 (1992)CrossRefGoogle Scholar
  6. 6.
    Popov, P., Strigini, L.: The reliability of diverse systems: A contribution using modelling of the fault creation process. In: DSN 2001, International Conference on Dependable Systems and Networks, Göteborg, Sweden (July 2001)Google Scholar
  7. 7.
    Rela, M., Madeira, H., Silva, J.G.: Experimental evaluation of the fail-silent behavior of programs with consistency checks. In: FTCS-26, Sendai, Japan, pp. 394–403 (1996)Google Scholar
  8. 8.
    Skiena, S., Revilla, M.: Programming Challenges, March 2003. Springer, Heidelberg (2003)MATHGoogle Scholar
  9. 9.
    van der Meulen, M.J.P., Bishop, P.G., Revilla, M.: An exploration of software faults and failure behaviour in a large population of programs. In: The 15th IEEE International Symposium of Software Reliability Engineering, St. Malo, France, November 2–5, pp. 101–12 (2004)Google Scholar
  10. 10.
    van der Meulen, M.J.P., Revilla, M.: The effectiveness of choice of programming language as a diversity seeking decision. In: Dal Cin, M., Kaâniche, M., Pataricza, A. (eds.) EDCC 2005. LNCS, vol. 3463, pp. 199–209. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Vinter, J., Aidemark, J., Folkesson, P., Karlsson, J.: Reducing critical failures for control algorithms using executable assertions and best effort recovery. In: DSN 2001, International Conference on Dependable Systems and Networks, Goteborg, Sweden (2001)Google Scholar
  12. 12.
    Wasserman, H., Blum, M.: Software reliability via run-time result-checking. Journal of the ACM 44(6), 826–849 (1997)MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Meine J. P. van der Meulen
    • 1
  • Lorenzo Strigini
    • 1
  • Miguel A. Revilla
    • 2
  1. 1.Centre for Software ReliabilityCity UniversityLondonUK
  2. 2.University of ValladolidValladolidSpain

Personalised recommendations