Skip to main content

Differential Monitoring

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 12974)


We argue that the time is ripe to investigate differential monitoring, in which the specification of a program’s behavior is implicitly given by a second program implementing the same informal specification. Similar ideas have been proposed before, and are currently implemented in restricted form for testing and specialized run-time analyses, aspects of which we combine. We discuss the challenges of implementing differential monitoring as a general-purpose, black-box run-time monitoring framework, and present promising results of a preliminary implementation, showing low monitoring overheads for diverse programs.


  • Run-time verification
  • Software engineering
  • Implicit specification

Supported in part by Austrian Science Fund (FWF) grant Z211-N23 (Wittgenstein Award).

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-88494-9_12
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   59.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-88494-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   79.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.


  1. Barringer, H., Falcone, Y., Havelund, K., Reger, G., Rydeheard, D.: Quantified event automata: towards expressive and efficient runtime monitors. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012. LNCS, vol. 7436, pp. 68–84. Springer, Heidelberg (2012).

    CrossRef  Google Scholar 

  2. Bartocci, E., Falcone, Y. (eds.): Lectures on Runtime Verification - Introductory and Advanced Topics. Lecture Notes in Computer Science, vol. 10457. Springer, Heidelberg (2018).

    CrossRef  Google Scholar 

  3. Bauer, A., Leucker, M., Schallhart, C.: Runtime verification for LTL and TLTL. ACM Trans. Softw. Eng. Methodol. 20(4) (2011).

  4. Berger, E.D., Zorn, B.G.: DieHard: probabilistic memory safety for unsafe languages. In: PLDI 2006, p. 158–168. Association for Computing Machinery, New York (2006).

  5. Bonakdarpour, B., Navabpour, S., Fischmeister, S.: Time-triggered runtime verification. Formal Methods Syst. Design 43(1), 29–60 (2013).

    CrossRef  MATH  Google Scholar 

  6. Cadar, C., Hosek, P.: Multi-version software updates. In: HotSWUp 2012, pp. 36–40 (2012).

  7. Chen, F., Roşu, G.: Parametric trace slicing and monitoring. In: Kowalewski, S., Philippou, A. (eds.) TACAS 2009. LNCS, vol. 5505, pp. 246–261. Springer, Heidelberg (2009).

    CrossRef  MATH  Google Scholar 

  8. Chen, L., Avizienis, A.: N-version programming: a fault-tolerance approach to reliability of software operation. In: FTCS 1978, vol. 1, pp. 3–9 (1978)

    Google Scholar 

  9. Chen, L., Avizienis, A.: N-version programming: a fault-tolerance approach to reliability of software operation. In: FTCS 1995, ‘Highlights from Twenty-Five Years’, p. 113ff (1995).

  10. Clarkson, M.R., Finkbeiner, B., Koleini, M., Micinski, K.K., Rabe, M.N., Sánchez, C.: Temporal logics for hyperproperties. In: Abadi, M., Kremer, S. (eds.) POST 2014. LNCS, vol. 8414, pp. 265–284. Springer, Heidelberg (2014).

    CrossRef  Google Scholar 

  11. Coppens, B., Sutter, B.D., Volckaert, S.: Multi-variant execution environments. In: Larsen, P., Sadeghi, A. (eds.) The Continuing Arms Race: Code-Reuse Attacks and Defenses, pp. 211–258. ACM/Morgan & Claypool (2018).

  12. Cox, B., et al.: N-Variant systems: a secretless framework for security through diversity. In: USENIX-SS 2006. USENIX Association, USA (2006).

  13. Demri, S., Lazic, R.: LTL with the freeze quantifier and register automata. ACM Trans. Comput. Log. 10(3), 16:1–16:30 (2009).

  14. Elmendorf, W.: Fault-tolerant programming. In: FTCS 1972, pp. 79–83 (1972)

    Google Scholar 

  15. Evans, R.B., Savoia, A.: Differential testing: a new approach to change detection. In: ESEC-FSE companion 2007, pp. 549–552. Association for Computing Machinery, New York (2007).

  16. Falcone, Y., Fernandez, J., Mounier, L.: What can you verify and enforce at runtime? Int. J. Softw. Tools Technol. Transf. 14(3), 349–382 (2012).

    CrossRef  Google Scholar 

  17. Falcone, Y., Mariani, L., Rollet, A., Saha, S.: Runtime failure prevention and reaction. In: Bartocci, E., Falcone, Y. (eds.) Lectures on Runtime Verification. LNCS, vol. 10457, pp. 103–134. Springer, Cham (2018).

    CrossRef  Google Scholar 

  18. Fischler, M.A. et al.: Distinct software: an approach to reliable computing. In: 2nd USA-Japan Computer Conference, pp. 1–7 (1975)

    Google Scholar 

  19. Fokkink, W.J.: Introduction to Process Algebra. Texts in Theoretical Computer Science. An EATCS Series. Springer, Heidelberg (2000).

  20. Girard, E., Rault, J.: A programming technique for software reliability. In: IEEE Symposium on Computer Software Reliability, pp. 44–50 (1973)

    Google Scholar 

  21. Glabbeek, R.J.: The linear time - branching time spectrum. In: Baeten, J.C.M., Klop, J.W. (eds.) CONCUR 1990. LNCS, vol. 458, pp. 278–297. Springer, Heidelberg (1990).

    CrossRef  Google Scholar 

  22. van Glabbeek, R.J., et al.: The linear time—branching time spectrum II. In: Best, E. (ed.) CONCUR 1993. LNCS, vol. 715, pp. 66–81. Springer, Heidelberg (1993).

    CrossRef  Google Scholar 

  23. Grigore, R., Distefano, D., Petersen, R.L., Tzevelekos, N.: Runtime verification based on register automata. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013. LNCS, vol. 7795, pp. 260–276. Springer, Heidelberg (2013).

    CrossRef  Google Scholar 

  24. Groce, A., Holzmann, G., Joshi, R.: Randomized differential testing as a prelude to formal verification. In: ICSE 2007, pp. 621–631. IEEE Computer Society, USA (2007).

  25. Havelund, K., Reger, G., Roşu, G.: Runtime verification past experiences and future projections. In: Steffen, B., Woeginger, G. (eds.) Computing and Software Science. LNCS, vol. 10000, pp. 532–562. Springer, Cham (2019).

    CrossRef  Google Scholar 

  26. Hosek, P., Cadar, C.: VARAN the unbelievable: an efficient N-version execution framework. In: ASPLOS 2015, pp. 339–353. Association for Computing Machinery, New York (2015).

  27. Kallas, K., Niksic, F., Stanford, C., Alur, R.: DiffStream: differential output testing for stream processing programs. PACMPL 4(OOPSLA) (2020).

  28. Knight, J.C., Leveson, N.G.: An experimental evaluation of the assumption of independence in multiversion programming. IEEE Trans. Softw. Eng. 12(1), 96–109 (1986).

    CrossRef  Google Scholar 

  29. Knight, J.C., Leveson, N.G.: A reply to the criticisms of the Knight & Leveson experiment. ACM SIGSOFT Softw. Eng. Notes 15(1), 24–35 (1990).

    CrossRef  Google Scholar 

  30. Kopetz, H.: Software redundancy in real time systems. In: IFIP Congress 1974, pp. 182–186. North-Holland (1974)

    Google Scholar 

  31. Lehmann, D., Pradel, M.: Feedback-directed differential testing of interactive debuggers. In: ESEC/FSE 2018, pp. 610–620. Association for Computing Machinery, New York (2018).

  32. McKeeman, W.M.: Differential testing for software. Digit. Tech. J. 10(1), 100–107 (1998).

  33. Muehlboeck, F., Henzinger, T.A.: Differential monitoring. Technical report 9946, IST Austria (2021).

  34. Pina, L., Andronidis, A., Hicks, M., Cadar, C.: MVEDSUA: higher availability dynamic software updates via multi-version execution. In: ASPLOS 2019, pp. 573–585. ACM (2019).

  35. Pnueli, A., Zaks, A.: PSL model checking and run-time verification via testers. In: Misra, J., Nipkow, T., Sekerinski, E. (eds.) FM 2006. LNCS, vol. 4085, pp. 573–586. Springer, Heidelberg (2006).

    CrossRef  Google Scholar 

  36. Pomerance, C., Selfridge, J.L., Wagstaff, S.S.: The pseudoprimes to \(25 \cdot 10^9\). Math. Comput. 35(151), 1003–1026 (1980)

    MATH  Google Scholar 

  37. Rigger, M., Su, Z.: Detecting optimization bugs in database engines via non-optimizing reference engine construction. In: ESEC/FSE 2020, pp. 1140–1152. Association for Computing Machinery, New York (2020).

  38. Rigger, M., Su, Z.: Finding bugs in database systems via query partitioning. PACMPL 4(OOPSLA) (2020).

  39. Sánchez, C.: Online and offline stream runtime verification of synchronous systems. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 138–163. Springer, Cham (2018).

    CrossRef  MATH  Google Scholar 

  40. Slutz, D.R.: Massive stochastic testing of SQL. In: VLDB 1998, pp. 618–622. Morgan Kaufmann (1998).

  41. Volckaert, S., De Sutter, B., De Baets, T., De Bosschere, K.: GHUMVEE: efficient, effective, and flexible replication. In: Garcia-Alfaro, J., Cuppens, F., Cuppens-Boulahia, N., Miri, A., Tawbi, N. (eds.) FPS 2012. LNCS, vol. 7743, pp. 261–277. Springer, Heidelberg (2013).

    CrossRef  Google Scholar 

  42. Yang, X., Chen, Y., Eide, E., Regehr, J.: Finding and understanding bugs in C compilers. In: PLDI 2011, pp. 283–294. Association for Computing Machinery, New York (2011).

Download references


The authors would like to thank Borzoo Bonakdarpour, Derek Dreyer, Adrian Francalanza, Owolabi Legunsen, Matthew Milano, Manuel Rigger, Cesar Sanchez, and the members of the IST Verification Seminar for their helpful comments and insights on various stages of this work, as well as the reviewers of RV’21 for their helpful suggestions on the actual paper.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Fabian Muehlboeck .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Muehlboeck, F., Henzinger, T.A. (2021). Differential Monitoring. In: Feng, L., Fisman, D. (eds) Runtime Verification. RV 2021. Lecture Notes in Computer Science(), vol 12974. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88493-2

  • Online ISBN: 978-3-030-88494-9

  • eBook Packages: Computer ScienceComputer Science (R0)