Skip to main content
SpringerLink
Log in

A taxonomy for classifying runtime verification tools

  • General
  • Special Issue: RV 2018
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Over the last 20 years, runtime verification (RV) has grown into a diverse and active field, which has stimulated the development of numerous theoretical frameworks and practical tools. Many of the tools are at first sight very different and challenging to compare. Yet, there are similarities. In this work, we classify RV tools within a high-level taxonomy of concepts. We first present this taxonomy and discuss its different dimensions. Then, we survey the existing RV tools and, where possible with the support of tool authors, classify them according to the taxonomy. While the classification continually evolves, this article presents a snapshot with 60 state-of-the-art RV tools. We believe that this work is an important step in establishing a common terminology in RV and enabling a meaningful comparison of existing RV tools.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. We note that the total absence of influence is impossible because of the need for instrumentation; see Sect. 3.6.

  2. A transformational program is a program that takes some input, processes it, delivers some output, and terminates (e.g., a compiler), as opposed to an interactive or a reactive program.

  3. Available at https://forms.gle/mUiTZK3egSPr3Wsd9.

  4. Available at https://goo.gl/Mmuhdd#gid=795731900.

  5. Available at https://goo.gl/Mmuhdd.

References

  1. Ahishakiye, F., Jaksic, S., Lange, F.D., Schmitz, M., Stolz, V., Thoma, D.: Non-intrusive MC/DC measurement based on traces. In: Méry, D., Qin, S. (eds.) 2019 International Symposium on Theoretical Aspects of Software Engineering, TASE 2019, Guilin, China, July 29–31, 2019, pp. 86–92. IEEE (2019). https://doi.org/10.1109/TASE.2019.00-15

  2. Ahrendt, W., Artho, C., Colombo, C., Falcone, Y., Krstić, S., Leucker, M., Lorber, F., Lourenço, J., Mariani, L., Sánchez, C., Schneider, G., Stolz, V.: COST action IC1402 ARVI: runtime verification beyond monitoring—activity report of working group 1. CoRR (2019). arXiv:1902.03776

  3. Akidau, T., Bradshaw, R., Chambers, C., Chernyak, S., Fernández-Moctezuma, R., Lax, R., McVeety, S., Mills, D., Perry, F., Schmidt, E., Whittle, S.: The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. PVLDB 8(12), 1792–1803 (2015)

    Google Scholar 

  4. Ancona, D., Ferrando, A., Mascardi, V.: Comparing trace expressions and linear temporal logic for runtime verification. In: Ábrahám, E., Bonsangue, M.M., Johnsen E.B. (eds.) Theory and Practice of Formal Methods—Essays Dedicated to Frank de Boer on the Occasion of His 60th Birthday, LNCS, vol. 9660, pp. 47–64. Springer (2016)

  5. Ancona, D., Ferrando, A., Mascardi, V.: Parametric runtime verification of multiagent systems. In: Larson, K., Winikoff, M., Das, S., Durfee, E.H. (eds.) Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, AAMAS 2017, São Paulo, Brazil, May 8–12, 2017, pp. 1457–1459. ACM (2017)

  6. Artho, C., Seidl, M., Gros, Q., Choi, E., Kitamura, T., Mori, A., Ramler, R., Yamagata, Y.: Model-based testing of stateful APIs with modbat. In: Cohen, M.B., Grunske, L., Whalen, M. (eds.) 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9–13, 2015, pp. 858–863. IEEE Computer Society (2015)

  7. Artho, C.V., Biere, A., Hagiya, M., Platon, E., Seidl, M., Tanabe, Y., Yamamoto, M.: Modbat: a model-based API tester for event-driven systems. In: Bertacco, V., Legay, A. (eds.) Hardware and Software: Verification and Testing—9th International Haifa Verification Conference, HVC 2013, Haifa, Israel, November 5–7, 2013, Proceedings, LNCS, vol. 8244, pp. 112–128. Springer (2013)

  8. Attard, D.P., Francalanza, A.: A monitoring tool for a branching-time logic. In: Falcone, Y., Sánchez, C. (eds.) Runtime Verification—16th International Conference, RV 2016, Madrid, Spain, September 23–30, 2016, Proceedings, LNCS, vol. 10012, pp. 473–481. Springer (2016)

  9. Attard, D.P., Francalanza, A.: Trace partitioning and local monitoring for asynchronous components. In: Cimatti, A., Sirjani, M. (eds.) Software Engineering and Formal Methods—15th International Conference, SEFM 2017, Trento, Italy, September 4–8, 2017, Proceedings, LNCS, vol. 10469, pp. 219–235. Springer (2017)

  10. Azzopardi, S., Colombo, C., Ebejer, J.P., Mallia, E., Pace, G.: Runtime verification using VALOUR. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 10–18. EasyChair (2017)

  11. Babenko, A., Mariani, L., Pastore, F.: AVA: automated interpretation of dynamically detected anomalies. In: Rothermel, G., Dillon, L.K. (eds.) Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, ISSTA 2009, Chicago, IL, USA, July 19–23, 2009, pp. 237–248. ACM (2009)

  12. Barringer, H., Falcone, Y., Havelund, K., Reger, G., Rydeheard, D.E.: Quantified event automata: towards expressive and efficient runtime monitors. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012, LNCS, vol. 7436, pp. 68–84. Springer (2012)

  13. Barringer, H., Goldberg, A., Havelund, K., Sen, K.: Program monitoring with LTL in EAGLE. In: 18th International Parallel and Distributed Processing Symposium (IPDPS 2004), CD-ROM/Abstracts Proceedings, 26–30 April 2004, Santa Fe, New Mexico, USA. IEEE Computer Society (2004)

  14. Barringer, H., Goldberg, A., Havelund, K., Sen, K.: Rule-based runtime verification. In: Steffen, B., Levi, G. (eds.) Verification, Model Checking, and Abstract Interpretation, 5th International Conference, VMCAI 2004, Venice, Italy, January 11–13, 2004, Proceedings, LNCS, vol. 2937, pp. 44–57. Springer (2004)

  15. Barringer, H., Groce, A., Havelund, K., Smith, M.H.: An entry point for formal methods: Specification and analysis of event logs. In: Bujorianu, M.L., Fisher, M. (eds.) Proceedings FM-09 Workshop on Formal Methods for Aerospace, FMA 2009, Eindhoven, The Netherlands, 3rd November 2009, EPTCS, vol. 20, pp. 16–21 (2009)

  16. Barringer, H., Groce, A., Havelund, K., Smith, M.H.: Formal analysis of log files. JACIC 7(11), 365–390 (2010)

    Article  Google Scholar 

  17. Barringer, H., Havelund, K.: Tracecontract: a Scala DSL for trace analysis. In: Butler, M.J., Schulte, W. (eds.) FM 2011: Formal Methods—17th International Symposium on Formal Methods, Limerick, Ireland, June 20–24, 2011. Proceedings, LNCS, vol. 6664, pp. 57–72. Springer (2011)

  18. Barringer, H., Rydeheard, D.E., Havelund, K.: Rule systems for run-time monitoring: from Eagle to RuleR. J. Log. Comput. 20(3), 675–706 (2010)

    Article  MathSciNet  Google Scholar 

  19. Bartocci, E., Bonakdarpour, B., Falcone, Y.: First international competition on software for runtime verification. In: Bonakdarpour, B., Smolka, S.A. (eds.) Runtime Verification—5th International Conference, RV 2014, Toronto, ON, Canada, September 22–25, 2014. Proceedings, LNCS, vol. 8734, pp. 1–9. Springer (2014)

  20. Bartocci, E., Falcone, Y. (eds.): Lectures on Runtime Verification–Iintroductory and Advanced Topics, LNCS, vol. 10457. Springer (2018)

  21. Bartocci, E., Falcone, Y., Bonakdarpour, B., Colombo, C., Decker, N., Havelund, K., Joshi, Y., Klaedtke, F., Milewicz, R., Reger, G., Rosu, G., Signoles, J., Thoma, D., Zalinescu, E., Zhang, Y.: First international competition on runtime verification: rules, benchmarks, tools, and final results of CRV 2014. STTT, pp. 1–40 (2017)

  22. Bartocci, E., Falcone, Y., Francalanza, A., Reger, G.: Introduction to runtime verification. In: Bartocci, E., Falcone, Y. (eds.) Lectures on Runtime Verification—Introductory and Advanced Topics, LNCS, vol. 10457, pp. 1–33. Springer (2018)

  23. Bartocci, E., Manjunath, N., Mariani, L., Mateis, C., Nickovic, D.: Automatic failure explanation in CPS models. CoRR (2019). arXiv:1903.12468

  24. Basin, D., Bhatt, B.N., Krstić, S., Traytel, D.: Almost event-rate independent monitoring. Formal Methods Syst. Des. 54, 449–478 (2019)

    Article  Google Scholar 

  25. Basin, D., Bhatt, B.N., Traytel, D.: Almost event-rate independent monitoring of metric temporal logic. In: Legay, A., Margaria, T. (eds.) TACAS 2017, LNCS, vol. 10206, pp. 94–112. Springer (2017)

  26. Basin, D., Harvan, M., Klaedtke, F., Zalinescu, E.: MONPOLY: monitoring usage-control policies. In: Khurshid, S., Sen, K. (eds.) RV 2011, LNCS, vol. 7186, pp. 360–364. Springer (2011)

  27. Basin, D., Klaedtke, F., Marinovic, S., Zalinescu, E.: Monitoring of temporal first-order properties with aggregations. Formal Methods Syst. Des. 46(3), 262–285 (2015)

    Article  Google Scholar 

  28. Basin, D., Klaedtke, F., Müller, S., Zalinescu, E.: Monitoring metric first-order temporal properties. J. ACM 62(2), 15:1–15:45 (2015)

    Article  MathSciNet  Google Scholar 

  29. Basin, D., Klaedtke, F., Zalinescu, E.: The MonPoly monitoring tool. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 19–28. EasyChair (2017)

  30. Basin, D., Krstić, S., Traytel, D.: AERIAL: almost event-rate independent algorithms for monitoring metric regular properties. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 29–36. EasyChair (2017)

  31. Basin, D., Krstić, S., Traytel, D.: Almost event-rate independent monitoring of metric dynamic logic. In: Lahiri, S.K. , Reger, G. (eds.) RV 2017, LNCS, vol. 10548, pp. 85–102. Springer (2017)

  32. Basin, D.A., Klaedtke, F., Marinovic, S., Zalinescu, E.: Monitoring compliance policies over incomplete and disagreeing logs. In: Qadeer, S., Tasiran, S. (eds.) Runtime Verification, Third International Conference, RV 2012, Istanbul, Turkey, September 25–28, 2012, Revised Selected Papers, LNCS, vol. 7687, pp. 151–167. Springer (2012)

  33. Basin, D.A., Klaedtke, F., Marinovic, S., Zalinescu, E.: On real-time monitoring with imprecise timestamps. In: Bonakdarpour, B., Smolka, S.A. (eds.) Runtime Verification—5th International Conference, RV 2014, Toronto, ON, Canada, September 22–25, 2014. Proceedings, LNCS, vol. 8734, pp. 193–198. Springer (2014)

  34. Bauer, A., Falcone, Y.: Decentralised LTL monitoring. Formal Methods Syst. Des. 48(1–2), 46–93 (2016)

    Article  Google Scholar 

  35. Bersani, M.M., Bianculli, D., Ghezzi, C., Krstić, S., San Pietro, P.: SMT-based checking of SOLOIST over sparse traces. In: Gnesi, S., Rensink, A. (eds.) Fundamental Approaches to Software Engineering—17th International Conference, FASE 2014, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2014, Grenoble, France, April 5–13, 2014, Proceedings, LNCS, vol. 8411, pp. 276–290. Springer (2014)

  36. Bersani, M.M., Bianculli, D., Ghezzi, C., Krstić, S., San Pietro, P.: Efficient large-scale trace checking using MapReduce. In: Dillon, L.K., Visser, W., Williams, L. (eds.) Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14–22, 2016, pp. 888–898. ACM (2016)

  37. Bianculli, D., Ghezzi, C., Krstić, S.: Trace checking of metric temporal logic with aggregating modalities using MapReduce. In: Giannakopoulou, D., Salaün, G. (eds.) Software Engineering and Formal Methods—12th International Conference, SEFM 2014, Grenoble, France, September 1–5, 2014. Proceedings, LNCS, vol. 8702, pp. 144–158. Springer (2014)

  38. Bianculli, D., Ghezzi, C., Krstić, S., San Pietro, P.: Offline trace checking of quantitative properties of service-based applications. In: 7th IEEE International Conference on Service-Oriented Computing and Applications, SOCA 2014, Matsue, Japan, November 17–19, 2014, pp. 9–16. IEEE Computer Society (2014)

  39. Bianculli, D., Ghezzi, C., San Pietro, P.: The tale of SOLOIST: a specification language for service compositions interactions. In: Pasareanu, C.S. , Salaün, G. (eds.) Formal Aspects of Component Software, 9th International Symposium, FACS 2012, Mountain View, CA, USA, September 12–14, 2012. Revised Selected Papers, LNCS, vol. 7684, pp. 55–72. Springer (2012)

  40. Blein, Y., Ledru, Y., du Bousquet, L., Groz, R.: Extending specification patterns for verification of parametric traces. In: Gnesi, S., Plat, N., Spoletini, P., Pelliccione, P. (eds.) Proceedings of the 6th Conference on Formal Methods in Software Engineering, FormaliSE 2018, collocated with ICSE 2018, Gothenburg, Sweden, June 2, 2018, pp. 10–19. ACM (2018)

  41. Bonakdarpour, B., Navabpour, S., Fischmeister, S.: Time-triggered runtime verification. Formal Methods Syst. Des. 43(1), 29–60 (2013)

    Article  Google Scholar 

  42. Carle, P., Choppy, C., Kervarc, R.: Behaviour recognition using chronicles. In: Duan, Z., Ong, C.L. (eds.) 5th IEEE International Symposium on Theoretical Aspects of Software Engineering, TASE 2011, Xi’an, China, 29–31 August 2011, pp. 100–107. IEEE Computer Society (2011)

  43. Carle, P., Choppy, C., Kervarc, R., Piel, A.: Handling breakdowns in unmanned aircraft systems. In: FM 2012: Formal Methods—18th International Symposium—Doctoral Symposium, Paris, France, August 27–31, 2012. Proceedings (2012)

  44. Carle, P., Choppy, C., Kervarc, R., Piel, A.: Safety of unmanned aircraft systems facing multiple breakdowns. In: Choppy, C., Sun, J. (eds.) 1st French Singaporean Workshop on Formal Methods and Applications, FSFMA 2013, July 15–16, 2013, Singapore, OASICS, vol. 31, pp. 86–91. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2013)

  45. Cassar, I., Francalanza, A.: Runtime adaptation for actor systems. In: Bartocci, E., Majumdar, R. (eds.) Runtime Verification—6th International Conference, RV 2015 Vienna, Austria, September 22–25, 2015. Proceedings, LNCS, vol. 9333, pp. 38–54. Springer (2015)

  46. Cassar, I., Francalanza, A.: On implementing a monitor-oriented programming framework for actor systems. In: Ábrahám, E., Huisman, M. (eds.) Integrated Formal Methods—12th International Conference, IFM 2016, Reykjavik, Iceland, June 1–5, 2016, Proceedings, LNCS, vol. 9681, pp. 176–192. Springer (2016)

  47. Cassar, I., Francalanza, A., Attard, D.P., Aceto, L., Ingólfsdóttir, A.: A generic instrumentation tool for Erlang. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017. An International Workshop on Competitions, Usability, Benchmarks, Evaluation, and Standardisation for Runtime Verification Tools, September 15, 2017, Seattle, WA, USA, Kalpa Publications in Computing, vol. 3, pp. 48–54. EasyChair (2017)

  48. Cassar, I., Francalanza, A., Attard, D.P., Aceto, L., Ingólfsdóttir, A.: A suite of monitoring tools for Erlang. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 41–47. EasyChair (2017)

  49. Cheikh, A.B., Blein, Y., Chehida, S., Vega, G., Ledru, Y., du Bousquet, L.: An environment for the partrap trace property language (tool demonstration). In: Colombo, C., Leucker, M. (eds.) Runtime Verification—18th International Conference, RV 2018, Limassol, Cyprus, November 10–13, 2018, Proceedings, LNCS, vol. 11237, pp. 437–446. Springer (2018)

  50. Cohen, S.: JTrek (2000). Developed by Compaq

  51. Cok, D.R.: OpenJML: software verification for Java 7 using JML, OpenJDK, and Eclipse. In: Dubois, C., Giannakopoulou, D., Méry, D. (eds.) Proceedings 1st Workshop on Formal Integrated Development Environment, F-IDE 2014, Grenoble, France, April 6, 2014, EPTCS, vol. 149, pp. 79–92 (2014)

  52. Colombo, C., Falcone, Y.: First international summer school on runtime verification—as part of the ARVI COST action 1402. In: Falcone, Y., Sánchez, C. (eds.) Runtime Verification—16th International Conference, RV 2016, Madrid, Spain, September 23–30, 2016, Proceedings, LNCS, vol. 10012, pp. 17–20. Springer (2016)

  53. Colombo, C., Falcone, Y.: Organising LTL monitors over distributed systems with a global clock. Formal Methods Syst. Des. 49(1–2), 109–158 (2016)

    Article  Google Scholar 

  54. Colombo, C., Pace, G.J.: Runtime verification using LARVA. In: G. Reger, K. Havelund (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 55–63. EasyChair (2017)

  55. Colombo, C., Pace, G.J., Schneider, G.: Dynamic event-based runtime monitoring of real-time and contextual properties. In: Cofer, D.D., Fantechi, A. (eds.) FMICS 2008, LNCS, vol. 5596, pp. 135–149. Springer (2008)

  56. Colombo, C., Pace, G.J., Schneider, G.: LARVA—safer monitoring of real-time Java programs (tool paper). In: Hung, D.V., Krishnan ,P. (eds.) SEFM 2009, pp. 33–37. IEEE Computer Society (2009)

  57. DeAntoni, J., Mallet, F.: Timesquare: treat your models with logical time. In: Furia, C.A., Nanz, S. (eds.) TOOLS 2012, LNCS, vol. 7304, pp. 34–41. Springer (2012)

  58. Decker, N., Harder, J., Scheffel, T., Schmitz, M., Thoma, D.: Runtime monitoring with union-find structures. In: Chechik, M., Raskin, J. (eds.) TACAS 2016, LNCS, vol. 9636, pp. 868–884. Springer (2016)

  59. Decker, N., Leucker, M., Thoma, D.: jUnit\({}^{\text{rv}}\)-adding runtime verification to jUnit. In: Brat, G., Rungta, N., Venet, A. (eds.) NFM 2013, LNCS, vol. 7871, pp. 459–464. Springer (2013)

  60. Decker, N., Leucker, M., Thoma, D.: Monitoring modulo theories. STTT 18(2), 205–225 (2016)

    Article  Google Scholar 

  61. Delahaye, M., Kosmatov, N., Signoles, J.: Common specification language for static and dynamic analysis of C programs. In: Shin, S.Y., Maldonado, J.C. (eds.) SAC 2013, pp. 1230–1235. ACM (2013)

  62. Delgado, N., Gates, A.Q., Roach, S.: A taxonomy and catalog of runtime software-fault monitoring tools. IEEE Trans. Softw. Eng. 30(12), 859–872 (2004)

    Article  Google Scholar 

  63. Díaz, A., Merino, P., Salmerón, A.: Obtaining models for realistic mobile network simulations using real traces. IEEE Commun. Lett. 15(7), 782–784 (2011)

    Article  Google Scholar 

  64. Donzé, A.: Breach, A toolbox for verification and parameter synthesis of hybrid systems. In: Touili, T., Cook, B., Jackson, P.B. (eds.) Computer Aided Verification, 22nd International Conference, CAV 2010, Edinburgh, UK, July 15–19, 2010. Proceedings, LNCS, vol. 6174, pp. 167–170. Springer (2010)

  65. Donzé, A., Ferrère, T., Maler, O.: Efficient robust monitoring for STL. In: Sharygina, N., Veith, H. (eds.) Computer Aided Verification—25th International Conference, CAV 2013, Saint Petersburg, Russia, July 13–19, 2013. Proceedings, LNCS, vol. 8044, pp. 264–279. Springer (2013)

  66. Dou, W., Bianculli, D., Briand, L.: TemPsy-Check: a tool for model-driven trace checking of pattern-based temporal properties. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 64–70. EasyChair (2017)

  67. Dou, W., Bianculli, D., Briand, L.C.: A model-driven approach to trace checking of pattern-based temporal properties. In: 20th ACM/IEEE International Conference on Model Driven Engineering Languages and Systems, MODELS 2017, Austin, TX, USA, September 17–22, 2017, pp. 323–333. IEEE Computer Society (2017)

  68. Drabek, C., Weiss, G.: DANA—description and analysis of networked applications. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 71–80. EasyChair (2017)

  69. Drusinsky, D.: Modeling and verification using UML Statecharts: a working guide to reactive system design. Runtime Monitoring and Execution-based Model Checking. Newnes, Newton, MA, USA (2006)

  70. El-Hokayem, A., Falcone, Y.: Monitoring decentralized specifications. In: Bultan, T., Sen, K. (eds.) Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, Santa Barbara, CA, USA, July 10–14, 2017, pp. 125–135. ACM (2017)

  71. El-Hokayem, A., Falcone, Y.: THEMIS: a tool for decentralized monitoring algorithms. In: Bultan, T., Sen, K. (eds.) Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, Santa Barbara, CA, USA, July 10–14, 2017, pp. 372–375. ACM (2017)

  72. El-Hokayem, A., Falcone, Y.: On the monitoring of decentralized specifications: semantics, properties, analysis, and simulation. ACM Trans. Softw. Eng. Methodol. 29(1), 1:1–1:57 (2020). https://doi.org/10.1145/3355181

    Article  Google Scholar 

  73. Ellul, J., Pace, G.J.: Runtime verification of ethereum smart contracts. In: 14th European Dependable Computing Conference, EDCC 2018, Iaşi, Romania, September 10–14, 2018, pp. 158–163. IEEE Computer Society (2018)

  74. Falcone, Y.: You should better enforce than verify. In: Barringer, H., Falcone, Y., Finkbeiner, B., Havelund, K., Lee, I., Pace, G.J., Rosu, G., Sokolsky, O., Tillmann, N. (eds.) Runtime Verification—First International Conference, RV 2010, St. Julians, Malta, November 1–4, 2010. Proceedings, LNCS, vol. 6418, pp. 89–105. Springer (2010)

  75. Falcone, Y.: Second school on runtime verification, as part of the arvi COST action 1402—overview and reflections. In: Runtime Verification—18th International Conference, RV 2018, Limassol, Cyprus, November 10–13, 2018, Proceedings, pp. 27–32 (2018). https://doi.org/10.1007/978-3-030-03769-7_3

  76. Falcone, Y., Havelund, K., Reger, G.: A tutorial on runtime verification. In: Broy, M., Peled, D.A., Kalus, G. (eds.) Engineering Dependable Software Systems, NATO SPS D: Information and Communication Security, vol. 34, pp. 141–175. IOS Press (2013)

  77. Falcone, Y., Krstić, S., Reger, G., Traytel, D.: A taxonomy for classifying runtime verification tools. In: International Conference on Runtime Verification, pp. 241–262. Springer (2018)

  78. Falcone, Y., Mariani, L., Rollet, A., Saha, S.: Runtime failure prevention and reaction. In: Bartocci and Falcone [22], pp. 103–134. https://doi.org/10.1007/978-3-319-75632-5_4

  79. Falcone, Y., Nickovic, D., Reger, G., Thoma, D.: Second international competition on runtime verification CRV 2015. In: Bartocci, E., Majumdar, R. (eds.) RV 2015, LNCS, vol. 9333, pp. 405–422. Springer (2015)

  80. Faymonville, P., Finkbeiner, B., Schledjewski, M., Schwenger, M., Stenger, M., Tentrup, L., Torfah, H.: Streamlab: stream-based monitoring of cyber-physical systems. In: Dillig, I., Tasiran, S. (eds.) Computer Aided Verification—31st International Conference, CAV 2019, New York City, NY, USA, July 15–18, 2019, Proceedings, Part I, LNCS, vol. 11561, pp. 421–431. Springer (2019)

  81. Finkbeiner, B., Hahn, C., Stenger, M., Tentrup, L.: RVHyper: a runtime verification tool for temporal hyperproperties. In: Tools and Algorithms for the Construction and Analysis of Systems—24th International Conference, TACAS 2018, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2018, Thessaloniki, Greece, April 14–20, 2018, Proceedings, Part II, pp. 194–200 (2018)

  82. Francalanza, A., Aceto, L., Ingólfsdóttir, A.: Monitorability for the Hennessy–Milner logic with recursion. Formal Methods Syst. Des. 51(1), 87–116 (2017)

    Article  Google Scholar 

  83. Francalanza, A., Pérez, J.A., Sánchez, C.: Runtime verification for decentralised and distributed systems. In: Bartocci, E., Falcone, Y. (eds.) Lectures on Runtime Verification—Introductory and Advanced Topics, LNCS, vol. 10457, pp. 176–210. Springer (2018)

  84. Goldberg, A., Havelund, K.: Automated runtime verification with Eagle. In: Ultes-Nitsche, U., Augusto, J.C., Barjis, J. (eds.) Modelling, Simulation, Verification and Validation of Enterprise Information Systems, Proceedings of the 3rd International Workshop on Modelling, Simulation, Verification and Validation of Enterprise Information Systems, MSVVEIS 2005, In conjunction with ICEIS 2005, Miami, FL, USA, May 2005. INSTICC Press (2005)

  85. Gorostiaga, F., Sánchez, C.: Striver: Stream runtime verification for real-time event-streams. In: Colombo, C., Leucker ,M. (eds.) Runtime Verification—18th International Conference, RV 2018, Limassol, Cyprus, November 10–13, 2018, Proceedings, LNCS, vol. 11237, pp. 282–298. Springer (2018)

  86. Goubault-Larrecq, J., Lachance, J.: On the complexity of monitoring orchids signatures, and recurrence equations. Formal Methods Syst. Des. 53(1), 6–32 (2018)

    Article  Google Scholar 

  87. Goubault-Larrecq, J., Olivain, J.: A smell of orchids. In: Leucker, M. (ed.) Runtime Verification, 8th International Workshop, RV 2008, Budapest, Hungary, March 30, 2008. Selected Papers, LNCS, vol. 5289, pp. 1–20. Springer (2008)

  88. Groce, A., Havelund, K., Smith, M.H.: From scripts to specifications: the evolution of a flight software testing effort. In: Kramer, J., Bishop, J., Devanbu, P.T., Uchitel, S. (eds.) Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering—Volume 2, ICSE 2010, Cape Town, South Africa, 1–8 May 2010, pp. 129–138. ACM (2010)

  89. Hallé, S.: When RV meets CEP. In: Falcone, Y., Sánchez, C. (eds.) RV 2016, LNCS, vol. 10012, pp. 68–91. Springer (2016)

  90. Hallé, S., Khoury, R.: Event stream processing with BeepBeep 3. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 81–88. EasyChair (2017)

  91. Havelund, K.: Runtime verification of C programs. In: Suzuki, K., Higashino, T., Ulrich, A., Hasegawa, T. (eds.) Testing of Software and Communicating Systems, 20th IFIP TC 6/WG 6.1 International Conference, TestCom 2008, 8th International Workshop, FATES 2008, Tokyo, Japan, June 10–13, 2008, Proceedings, LNCS, vol. 5047, pp. 7–22. Springer (2008)

  92. Havelund, K.: Rule-based runtime verification revisited. STTT 17(2), 143–170 (2015)

    Article  Google Scholar 

  93. Havelund, K., Leucker, M., Reger, G., Stolz, V.: A shared challenge in behavioural specification (Dagstuhl seminar 17462). Dagstuhl Rep. 7(11), 59–85 (2017)

    Google Scholar 

  94. Havelund, K., Peled, D.: Bdds on the run. In: Margaria, T., Steffen, B. (eds.) Leveraging Applications of Formal Methods, Verification and Validation. Industrial Practice—8th International Symposium, ISoLA 2018, Limassol, Cyprus, November 5–9, 2018, Proceedings, Part IV, LNCS, vol. 11247, pp. 58–69. Springer (2018)

  95. Havelund, K., Peled, D.: Efficient runtime verification of first-order temporal properties. In: Gallardo, M., Merino, P. (eds.) Model Checking Software—25th International Symposium, SPIN 2018, Malaga, Spain, June 20–22, 2018, Proceedings, LNCS, vol. 10869, pp. 26–47. Springer (2018)

  96. Havelund, K., Peled, D.: Runtime verification: from propositional to first-order temporal logic. In: Colombo, C., Leucker, M. (eds.) Runtime Verification—18th International Conference, RV 2018, Limassol, Cyprus, November 10–13, 2018, Proceedings, LNCS, vol. 11237, pp. 90–112. Springer (2018)

  97. Havelund, K., Peled, D., Ulus, D.: First order temporal logic monitoring with BDDs. In: 2017 Formal Methods in Computer Aided Design (FMCAD), pp. 116–123 (2017)

  98. Havelund, K., Peled, D., Ulus, D.: Dejavu: A monitoring tool for first-order temporal logic. In: 3rd Workshop on Monitoring and Testing of Cyber-Physical Systems, MT@CPSWeek 2018, Porto, Portugal, April 10, 2018, pp. 12–13. IEEE (2018)

  99. Havelund, K., Reger, G.: Runtime verification logics A language design perspective. In: Models, Algorithms, Logics and Tools—Essays Dedicated to Kim Guldstrand Larsen on the Occasion of His 60th Birthday, pp. 310–338 (2017)

  100. Havelund, K., Rosu, G.: Monitoring Java programs with Java PathExplorer. Electr. Notes Theor. Comput. Sci. 55(2), 200–217 (2001)

    Article  Google Scholar 

  101. Havelund, K., Rosu, G.: Synthesizing monitors for safety properties. In: Katoen, J., Stevens, P. (eds.) Tools and Algorithms for the Construction and Analysis of Systems, 8th International Conference, TACAS 2002, Held as Part of the Joint European Conference on Theory and Practice of Software, ETAPS 2002, Grenoble, France, April 8–12, 2002, Proceedings, LNCS, vol. 2280, pp. 342–356. Springer (2002)

  102. Havelund, K., Rosu, G.: Efficient monitoring of safety properties. STTT 6(2), 158–173 (2004)

    Article  Google Scholar 

  103. Havelund, K., Rosu, G.: An overview of the runtime verification tool java pathexplorer. Formal Methods Syst. Des. 24(2), 189–215 (2004)

    Article  Google Scholar 

  104. IC1402 Runtime Verification beyond Monitoring (ARVI). https://www.cost-arvi.eu/

  105. Jakse, R., Falcone, Y., Méhaut, J., Pouget, K.: Interactive runtime verification—when interactive debugging meets runtime verification. In: 28th IEEE International Symposium on Software Reliability Engineering, ISSRE 2017, Toulouse, France, October 23–26, 2017, pp. 182–193. IEEE Computer Society (2017). https://doi.org/10.1109/ISSRE.2017.19

  106. Jin, D., Meredith, P.O., Lee, C., Rosu, G.: JavaMOP: Efficient parametric runtime monitoring framework. In: Glinz, M., Murphy, G.C., Pezzè, M. (eds.) ICSE 2012, pp. 1427–1430. IEEE Computer Society (2012)

  107. Kane, A.: Runtime monitoring for safety-critical embedded systems. Ph.D. thesis, Carnegie Mellon University, Pittsburgh, USA (2015). https://doi.org/10.1184/R1/6721376.v1

  108. Kane, A., Chowdhury, O., Datta, A., Koopman, P.: A case study on runtime monitoring of an autonomous research vehicle (ARV) system. In: Bartocci, E., Majumdar, R. (eds.) Runtime Verification—6th International Conference, RV 2015 Vienna, Austria, September 22–25, 2015. Proceedings, LNCS, vol. 9333, pp. 102–117. Springer (2015)

  109. Kauffman, S., Fischmeister, S.: Event stream abstraction using nfer: demo abstract. In: Liu, X., Tabuada, P., Pajic, M., Bushnell, L. (eds.) Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems, ICCPS 2019, Montreal, QC, Canada, April 16–18, 2019, pp. 332–333. ACM (2019)

  110. Kauffman, S., Havelund, K., Joshi, R.: nfer—a notation and system for inferring event stream abstractions. In: Falcone, Y., Sánchez, C. (eds.) Runtime Verification—16th International Conference, RV 2016, Madrid, Spain, September 23–30, 2016, Proceedings, LNCS, vol. 10012, pp. 235–250. Springer (2016)

  111. Kauffman, S., Havelund, K., Joshi, R., Fischmeister, S.: Inferring event stream abstractions. Formal Methods Syst. Des. 53(1), 54–82 (2018)

    Article  Google Scholar 

  112. Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., Griswold, W.G.: An overview of AspectJ. In: Knudsen, J.L. (ed.) ECOOP 2001—Object-Oriented Programming, 15th European Conference, Budapest, Hungary, June 18–22, 2001, Proceedings, LNCS, vol. 2072, pp. 327–353. Springer (2001)

  113. Leucker, M., Sánchez, C., Scheffel, T., Schmitz, M., Schramm, A.: Tessla: runtime verification of non-synchronized real-time streams. In: Haddad, H.M., Wainwright, R.L., Chbeir, R. (eds.) Proceedings of the 33rd Annual ACM Symposium on Applied Computing, SAC 2018, Pau, France, April 09–13, 2018, pp. 1925–1933. ACM (2018)

  114. Leucker, M., Schallhart, C.: A brief account of runtime verification. J. Log. Algebr. Program. 78(5), 293–303 (2009)

    Article  Google Scholar 

  115. Lourenço, J.M., Fiedor, J., Krena, B., Vojnar, T.: Discovering concurrency errors. In: Bartocci, E., Falcone, Y. (eds.) Lectures on Runtime Verification—Introductory and Advanced Topics, LNCS, vol. 10457, pp. 34–60. Springer (2018)

  116. Luo, Q., Zhang, Y., Lee, C., Jin, D., Meredith, P.O., Serbanuta, T., Rosu, G.: RV-Monitor: efficient parametric runtime verification with simultaneous properties. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014, LNCS, vol. 8734, pp. 285–300. Springer (2014)

  117. Mariani, L., Pastore, F., Pezzè, M.: Dynamic analysis for diagnosing integration faults. IEEE Trans. Softw. Eng. 37(4), 486–508 (2011)

    Article  Google Scholar 

  118. Meredith, P.O., Jin, D., Griffith, D., Chen, F., Rosu, G.: An overview of the MOP runtime verification framework. STTT 14(3), 249–289 (2012)

    Article  Google Scholar 

  119. Milewicz, R., Vanka, R., Tuck, J., Quinlan, D., Pirkelbauer, P.: Lightweight runtime checking of C programs with RTC. Comput. Lang. Syst. Str. 45, 191–203 (2016)

    Google Scholar 

  120. Moosbrugger, P., Rozier, K.Y., Schumann, J.: R2U2: monitoring and diagnosis of security threats for unmanned aerial systems. Formal Methods Syst. Des. 51(1), 31–61 (2017)

    Article  Google Scholar 

  121. Navabpour, S., Joshi, Y., Wu, C.W.W., Berkovich, S., Medhat, R., Bonakdarpour, B., Fischmeister, S.: RiTHM: a tool for enabling time-triggered runtime verification for C programs. In: Meyer, B., Baresi, L., Mezini, M. (eds.) ESEC/FSE 2013, pp. 603–606. ACM (2013)

  122. Olivain, J., Goubault-Larrecq, J.: The orchids intrusion detection tool. In: Etessami, K., Rajamani, S.K. (eds.) Computer Aided Verification, 17th International Conference, CAV 2005, Edinburgh, Scotland, UK, July 6–10, 2005, Proceedings, LNCS, vol. 3576, pp. 286–290. Springer (2005)

  123. Pastore, F., Mariani, L.: AVA: supporting debugging with failure interpretations. In: Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013, Luxembourg, Luxembourg, March 18–22, 2013, pp. 416–421. IEEE Computer Society (2013)

  124. Pastore, F., Mariani, L., Goffi, A., Oriol, M., Wahler, M.: Dynamic analysis of upgrades in C/C++ software. In: 23rd IEEE International Symposium on Software Reliability Engineering, ISSRE 2012, Dallas, TX, USA, November 27–30, 2012, pp. 91–100. IEEE Computer Society (2012)

  125. Pastore, F., Mariani, L., Goffi, A., Oriol, M., Wahler, M.: RADAR: dynamic analysis of upgrades in C/C++ software. In: Chockler, H., Kroening, D., Mariani, L., Sharygina, N. (eds.) Validation of Evolving Software, pp. 85–105. Springer (2015)

  126. Piel, A.: Reconnaissance de comportements complexes par traitement en ligne de flux d’événements. (Online event flow processing for complex behaviour recognition). Ph.D. thesis, Paris 13 University, Villetaneuse, Saint-Denis, Bobigny, France (2014). https://tel.archives-ouvertes.fr/tel-01262245

  127. Pinisetty, S., Falcone, Y., Jéron, T., Marchand, H.: TiPeX: a tool chain for timed property enforcement during execution. In: Bartocci, E., Majumdar, R. (eds.) Runtime Verification—6th International Conference, RV 2015 Vienna, Austria, September 22–25, 2015. Proceedings, LNCS, vol. 9333, pp. 306–320. Springer (2015)

  128. Pinisetty, S., Falcone, Y., Jéron, T., Marchand, H., Rollet, A., Nguena-Timo, O.: Runtime enforcement of timed properties revisited. Formal Methods Syst. Des. 45(3), 381–422 (2014)

    Article  Google Scholar 

  129. Pnueli, A., Zaks, A.: PSL model checking and run-time verification via testers. In: Misra, J., Nipkow, T., Sekerinski, E. (eds.) FM 2006: Formal Methods, 14th International Symposium on Formal Methods, Hamilton, Canada, August 21–27, 2006, Proceedings, LNCS, vol. 4085, pp. 573–586. Springer (2006). https://doi.org/10.1007/11813040_38

  130. Pnueli, A., Zaks, A.: On the merits of temporal testers. In: Grumberg, O., Veith, H. (eds.) 25 Years of Model Checking—History, Achievements, Perspectives, LNCS, vol. 5000, pp. 172–195. Springer (2008)

  131. Rapin, N.: Reactive property monitoring of hybrid systems with aggregation. In: Falcone, Y., Sánchez, C. (eds.) Runtime Verification—16th International Conference, RV 2016, Madrid, Spain, September 23–30, 2016, Proceedings, LNCS, vol. 10012, pp. 447–453. Springer (2016)

  132. Rapin, N.: ARTiMon monitoring tool, the time domains. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 106–122. EasyChair (2017)

  133. Raszyk, M., Basin, D., Traytel, D.: Multi-head monitoring of metric dynamic logic. In: Hung, D.V., Sokolsky, O. (eds.) ATVA 2020, LNCS, vol. 12302. Springer (2020). To appear

  134. Raszyk, M., Basin, D.A., Krstic, S., Traytel, D.: Multi-head monitoring of metric temporal logic. In: Chen, Y., Cheng, C., Esparza, J. (eds.) ATVA 2019, LNCS, vol. 11781, pp. 151–170. Springer (2019)

  135. Reger, G.: An overview of MarQ. In: Falcone, Y., Sánchez, C. (eds.) RV 2016, LNCS, vol. 10012, pp. 498–503. Springer (2016)

  136. Reger, G.: A report of RV-CuBES 2017. In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 1–9. EasyChair (2017)

  137. Reger, G., Cruz, H.C., Rydeheard, D.E.: MarQ: monitoring at runtime with QEA. In: Baier, C., Tinelli, C. (eds.) TACAS 2015, LNCS, vol. 9035, pp. 596–610. Springer (2015)

  138. Reger, G., Hallé, S., Falcone, Y.: Third international competition on runtime verification—CRV 2016. In: Falcone, Y., Sánchez, C. (eds.) RV 2016, LNCS, vol. 10012, pp. 21–37. Springer (2016)

  139. Reger, G., Havelund, K. (eds.): RV-CuBES 2017. An International Workshop on Competitions, Usability, Benchmarks, Evaluation, and Standardisation for Runtime Verification Tools, Kalpa Publications in Computing, vol. 3. EasyChair (2017)

  140. Reinbacher, T., Rozier, K.Y., Schumann, J.: Temporal-logic based runtime observer pairs for system health management of real-time systems. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014, LNCS, vol. 8413, pp. 357–372. Springer (2014)

  141. Renard, M., Rollet, A., Falcone, Y.: GREP: games for the runtime enforcement of properties. In: Yevtushenko, N., Cavalli, A.R., Yenigün, H. (eds.) Testing Software and Systems—29th IFIP WG 6.1 International Conference, ICTSS 2017, St. Petersburg, Russia, October 9–11, 2017, Proceedings, LNCS, vol. 10533, pp. 259–275. Springer (2017). https://doi.org/10.1007/978-3-319-67549-7_16

  142. Riganelli, O., Micucci, D., Mariani, L.: Policy enforcement with proactive libraries. In: 12th IEEE/ACM International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS@ICSE 2017, Buenos Aires, Argentina, May 22–23, 2017, pp. 182–192. IEEE Computer Society (2017)

  143. Riganelli, O., Micucci, D., Mariani, L.: Increasing the reusability of enforcers with lifecycle events. In: Margaria, T., Steffen, B. (eds.) Leveraging Applications of Formal Methods, Verification and Validation. Industrial Practice—8th International Symposium, ISoLA 2018, Limassol, Cyprus, November 5–9, 2018, Proceedings, Part IV, LNCS, vol. 11247, pp. 51–57. Springer (2018)

  144. Salmerón, A.: Model checking techniques for runtime testing and QoS analysis. Ph.D. thesis

  145. Sánchez, C.: Online and offline stream runtime verification of synchronous systems. In: Colombo, C., Leucker, M. (eds.) Runtime Verification—18th International Conference, RV 2018, Limassol, Cyprus, November 10–13, 2018, Proceedings, LNCS, vol. 11237, pp. 138–163. Springer (2018)

  146. Sánchez, C., Schneider, G., Ahrendt, W., Bartocci, E., Bianculli, D., Colombo, C., Falcone, Y., Francalanza, A., Krstic, S., Lourenço, J.M., Nickovic, D., Pace, G.J., Rufino, J., Signoles, J., Traytel, D., Weiss, A.: A survey of challenges for runtime verification from advanced application domains (beyond software). Formal Methods Syst. Des. 54(3), 279–335 (2019). https://doi.org/10.1007/s10703-019-00337-w

    Article  MATH  Google Scholar 

  147. Santos, J.F., Jensen, T., Rezk, T., Schmitt, A.: Hybrid typing of secure information flow in a JavaScript-like language. In: Ganty, P., Loreti, M. (eds.) Trustworthy Global Computing—10th International Symposium, TGC 2015, Madrid, Spain, August 31–September 1, 2015 Revised Selected Papers, LNCS, vol. 9533, pp. 63–78. Springer (2015)

  148. Schumann, J., Moosbrugger, P., Rozier, K.Y.: Runtime analysis with R2U2: a tool exhibition report. In: Falcone, Y., Sánchez, C. (eds.) RV 2016, LNCS, vol. 10012, pp. 504–509. Springer (2016)

  149. Serebryany, K., Bruening, D., Potapenko, A., Vyukov, D.: Addresssanitizer: a fast address sanity checker. In: Heiser, G., Hsieh, W.C. (eds.) 2012 USENIX Annual Technical Conference, Boston, MA, USA, June 13–15, 2012, pp. 309–318. USENIX Association (2012)

  150. Seyster, J., Dixit, K., Huang, X., Grosu, R., Havelund, K., Smolka, S.A., Stoller, S.D., Zadok, E.: Interaspect: aspect-oriented instrumentation with GCC. Formal Methods Syst. Des. 41(3), 295–320 (2012)

    Article  Google Scholar 

  151. Signoles, J., Kosmatov, N., Vorobyov, K.: E-ACSL, a runtime verification tool for safety and security of C programs (tool paper). In: Reger, G., Havelund, K. (eds.) RV-CuBES 2017, Kalpa Publications in Computing, vol. 3, pp. 164–173. EasyChair (2017)

  152. Song, D., Lettner, J., Rajasekaran, P., Na, Y., Volckaert, S., Larsen, P., Franz, M.: Sok: sanitizing for security. CoRR (2018). arXiv:1806.04355

  153. Soueidi, C., Kassem, A., Falcone, Y.: BISM: bytecode-level instrumentation for software monitoring. In: Runtime Verification—20th International Conference, RV 2020, Los Angeles, CA, USA, October 6–9, 2020, Proceedings, pp. 323–335 (2020). https://doi.org/10.1007/978-3-030-60508-7_18

  154. Stoller, S.D., Bartocci, E., Seyster, J., Grosu, R., Havelund, K., Smolka, S.A., Zadok, E.: Runtime verification with state estimation. In: Khurshid, S., Sen, K. (eds.) Runtime Verification—Second International Conference, RV 2011, San Francisco, CA, USA, September 27–30, 2011, Revised Selected Papers, LNCS, vol. 7186, pp. 193–207. Springer (2011). https://doi.org/10.1007/978-3-642-29860-8_15

  155. Szekeres, L., Payer, M., Wei, T., Song, D.: Sok: Eternal war in memory. In: 2013 IEEE Symposium on Security and Privacy, SP 2013, Berkeley, CA, USA, May 19–22, 2013, pp. 48–62. IEEE Computer Society (2013)

  156. Ulus, D.: Montre: a tool for monitoring timed regular expressions. In: Majumdar, R., Kuncak, V. (eds.) Computer Aided Verification—29th International Conference, CAV 2017, Heidelberg, Germany, July 24–28, 2017, Proceedings, Part I, LNCS, vol. 10426, pp. 329–335. Springer (2017)

  157. Ulus, D.: Sequential circuits from regular expressions revisited. CoRR (2018). arXiv:1801.08979

  158. Ulus, D.: Online monitoring of metric temporal logic using sequential networks. CoRR (2019). arXiv:1901.00175

  159. Ulus, D., Ferrère, T., Asarin, E., Maler, O.: Timed pattern matching. In: Legay, A., Bozga, M.(eds.) Formal Modeling and Analysis of Timed Systems—12th International Conference, FORMATS 2014, Florence, Italy, September 8–10, 2014. Proceedings, LNCS, vol. 8711, pp. 222–236. Springer (2014)

  160. Ulus, D., Ferrère, T., Asarin, E., Maler, O.: Online timed pattern matching using derivatives. In: Chechik, M., Raskin, J. (eds.) Tools and Algorithms for the Construction and Analysis of Systems—22nd International Conference, TACAS 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings, LNCS, vol. 9636, pp. 736–751. Springer (2016)

  161. van der Veen, V., dutt-Sharma, N., Cavallaro, L., Bos, H.: Memory errors: the past, the present, and the future. In: Balzarotti, D., Stolfo, S.J., Cova, M. (eds.) Research in Attacks, Intrusions, and Defenses—15th International Symposium, RAID 2012, Amsterdam, The Netherlands, September 12–14, 2012. Proceedings, LNCS, vol. 7462, pp. 86–106. Springer (2012)

  162. Walkinshaw, N., Taylor, R., Derrick, J.: Inferring extended finite state machine models from software executions. Empir. Softw. Eng. 21(3), 811–853 (2016)

    Article  Google Scholar 

  163. Wang, L., Stoller, S.D.: Accurate and efficient runtime detection of atomicity errors in concurrent programs. In: Torrellas, J., Chatterjee, S. (eds.) Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP 2006, New York, New York, USA, March 29–31, 2006, pp. 137–146. ACM (2006)

  164. Wang, L., Stoller, S.D.: Runtime analysis of atomicity for multithreaded programs. IEEE Trans. Softw. Eng. 32(2), 93–110 (2006)

    Article  Google Scholar 

  165. Xposed. https://repo.xposed.info/

Download references

Acknowledgements

The authors warmly thank Martin Leucker for the early discussions on the taxonomy and its mind map representation. This article is based on work from COST Action ARVI IC1402 [2], supported by COST (European Cooperation in Science and Technology). In particular, the taxonomy and classification benefited from discussions within working groups one and two of this action. We would also like to acknowledge input from participants of Dagstuhl seminar 17462 [93]. We thank anonymous RV and STTT reviewers for their input that helped us to improve the presentation of this work and to refine the taxonomy. Finally, we would like to thank all the tool authors who contributed to the tool classification. The research is partially supported by the Swiss National Science Foundation Grant “Big Data Monitoring” (167162), the US Air Force Grant “Monitoring at Any Cost” (FA9550-17-1-0306), and by H2020-ECSEL grants CPS4EU 2018-IA call - Grant Agreement Number 826276.

Author information

Authors and Affiliations

  1. Univ. Grenoble Alpes, CNRS, Inria, Grenoble INP, Laboratoire d’Informatique de Grenoble, 38000, Grenoble, France

    Yliès Falcone

  2. Institute of Information Security, Department of Computer Science, ETH Zürich, Zurich, Switzerland

    Srđan Krstić

  3. University of Manchester, Manchester, UK

    Giles Reger

  4. Department of Computer Science, University of Copenhagen, Copenhagen, Denmark

    Dmitriy Traytel

Authors
  1. Yliès Falcone
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Srđan Krstić
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giles Reger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dmitriy Traytel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yliès Falcone.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Falcone, Y., Krstić, S., Reger, G. et al. A taxonomy for classifying runtime verification tools. Int J Softw Tools Technol Transfer 23, 255–284 (2021). https://doi.org/10.1007/s10009-021-00609-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-021-00609-z

Keywords

Search

Navigation