Modeling the Impact of Testing on Diverse Programs

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9337)


This paper presents a model of diverse programs that assumes there are a common set of potential software faults that are more or less likely to exist in a specific program version. Testing is modeled as a specific ordering of the removal of faults from each program version. Different models of testing are examined where common and diverse test strategies are used for the diverse program versions. Under certain assumptions, theory suggests that a common test strategy could leave the proportion of common faults unchanged, while diverse test strategies are likely to reduce the proportion of common faults. A review of the available empirical evidence gives some support to the assumptions made in the fault-based model. We also consider how the proportion of common faults can be related to the expected reliability improvement.


Software diversity Multi-version programs Diverse test strategies 



The author wishes to acknowledge the support of the UK Control and Instrumentation Nuclear Industry Forum (CINIF) who funded the research presented in this paper.


  1. 1.
    Bentley, J.G., Bishop, P.G., van der Meulen, M.J.: An empirical exploration of the difficulty function. In: Heisel, M., Liggesmeyer, P., Wittmann, S. (eds.) SAFECOMP 2004. LNCS, vol. 3219, pp. 60–71. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Brilliant, S.S., Knight, J.C., Leveson, N.G.: Analysis of faults in an N-version software experiment. IEEE Trans. Softw. Eng. 16(2), 238–247 (1990)CrossRefGoogle Scholar
  3. 3.
    Eckhardt, D.E., Lee, L.D.: A theoretical basis for the analysis of multiversion software subject to coincident errors. IEEE Trans. Softw. Eng. 12, 1511–1517 (1985)CrossRefMATHGoogle Scholar
  4. 4.
    Littlewood, B., Miller, D.R.: Conceptual modeling of coincident failures in multi-version software. IEEE Trans. Softw. Eng. 15(12), 1596–1614 (1989)CrossRefGoogle Scholar
  5. 5.
    Littlewood, B., Popov, P.T., Strigini, L., Shryane, N.: Modeling the effects of combining diverse software fault detection techniques. IEEE Trans. Softw. Eng. 26(12), 1157–1167 (2000)CrossRefGoogle Scholar
  6. 6.
    Littlewood, B., Rushby, J.: Reasoning about the reliability of diverse two-channel systems in which one channel is “possibly perfect”. IEEE Trans. Softw. Eng. 38(5), 1178–1194 (2012)CrossRefGoogle Scholar
  7. 7.
    Popov, P., Littlewood, B.: The effect of testing on reliability of fault-tolerant software. In: International Conference on Dependable Systems and Networks, DSN 2004, pp. 265–274. IEEE (2004)Google Scholar
  8. 8.
    Popov, P., Strigini, L.: The reliability of diverse systems: a contribution using modelling of the fault creation process. In: International Conference on Dependable Systems and Networks, DSN 2001, pp. 5–14). IEEE (2001)Google Scholar
  9. 9.
    Popov, P., Stankovic, V., Strigini, L.: An empirical study of the effectiveness of “forcing” diversity based on a large population of diverse programs. In International Conference on Software Reliability Engineering, ISSRE 2012, pp. 41–50. IEEE (2012)Google Scholar
  10. 10.
    Revilla, M., Skiena, S.: Programming Challenges: The Programming Contest Training Manual. Springer, New York (2003)MATHGoogle Scholar
  11. 11.
    Van der Meulen, M.J.P., Bishop, P.G., Villa, R.: An exploration of software faults and failure behaviour in a large population of programs. In: International Symposium on Software Reliability Engineering, ISSRE 2004, pp. 101–112. IEEE (2004)Google Scholar
  12. 12.
    Van der Meulen, M.J., Revilla, M.A.: The effectiveness of software diversity in a large population of programs. IEEE Trans. Softw. Eng. 34(6), 753–764 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.City University and Adelard LLPLondonUK

Personalised recommendations