Second International Competition on Runtime Verification

CRV 2015
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9333)


We report on the Second International Competition on Runtime Verification (CRV-2015). The competition was held as a satellite event of the 15th International Conference on Runtime Verification (RV’15). The competition consisted of three tracks: offline monitoring, online monitoring of C programs, and online monitoring of Java programs. This report describes the format of the competition, the participating teams and submitted benchmarks. We give an example illustrating the two main inputs expected from the participating teams, namely a benchmark (i.e. a program and a property on this program) and a monitor for this benchmark. We also propose some reflection based on the lessons learned.


Specification Language Linear Temporal Logic Online Monitoring Execution Trace Concrete Event 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The organizers would like to thank Christian Seguy from the IT team of Laboratoire d’Informatique de Grenoble for his help on setting up the the repository hosting the benchmarks. The organizers are also grateful to Yuguang Zhang from the DataMill team for setting up a convenient and powerful evaluation infrastructure on DataMill. This work has been partly done in the context of the ICT COST Action IC1402 Runtime Verification beyond Monitoring (ARVI).


  1. 1.
    Barringer, H., Falcone, Y., Havelund, K., Reger, G., Rydeheard, D.: Quantified event automata: towards expressive and efficient runtime monitors. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012. LNCS, vol. 7436, pp. 68–84. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  2. 2.
    Bartocci, E., Bonakdarpour, B., Falcone, Y.: First international competition on software for runtime verification. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 1–9. Springer, Heidelberg (2014) Google Scholar
  3. 3.
    DeAntoni, J., Mallet, F.: Timesquare: treat your models with logical time. In: Furia, C.A., Nanz, S. (eds.) TOOLS 2012. LNCS, vol. 7304, pp. 34–41. Springer, Heidelberg (2012). CrossRefGoogle Scholar
  4. 4.
    Decker, N., Leucker, M., Thoma, D.: Monitoring modulo theories. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014 (ETAPS). LNCS, vol. 8413, pp. 341–356. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  5. 5.
    Delahaye, M., Kosmatov, N., Signoles, J.: Common specification language for static and dynamic analysis of c programs. In: Proceedings of SAC 2013: the 28th Annual ACM Symposium on Applied Computing, pp. 1230–1235. ACM (2013)Google Scholar
  6. 6.
    Díaz, A., Merino, P., Salmeron, A.: Obtaining models for realistic mobile network simulations using real traces. IEEE Commun. Lett. 15(7), 782–784 (2011)CrossRefGoogle Scholar
  7. 7.
    Dou, W., Bianculli, D., Briand, L.: A model-driven approach to offline trace checking of temporal properties with ocl. Technical report. SnT-TR-2014-5, Interdisciplinary Centre for Security, Reliability and Trust (2014).
  8. 8.
    Falcone, Y., Havelund, K., Reger, G.: A tutorial on runtime verification. In: Broy, M., Peled, D., Kalus, G., (eds.) Summer School Marktoberdorf 2012 - Engineering Dependable Software Systems, vol. 34, pp. 141–175. IOS Press (2013)Google Scholar
  9. 9.
    Forgy, C.: Rete: a fast algorithm for the many patterns/many objects match problem. Artif. Intell. 19(1), 17–37 (1982)CrossRefGoogle Scholar
  10. 10.
    Havelund, K.: Rule-based runtime verification revisited. Int. J. Softw. Tools Technol Transfer (STTT) 17, 143–170 (2014)CrossRefGoogle Scholar
  11. 11.
    Jin, D., Meredith, P.O., Lee, C., Roşu, G.: JavaMOP: efficient parametric runtime monitoring framework. In: Proceedings of ICSE 2012: The 34th International Conference on Software Engineering, Zurich, Switzerland, June 2–9, pp. 1427–1430. IEEE Press (2012)Google Scholar
  12. 12.
    Juniwal, G., Donzé, A., Jensen, J.C., Seshia, S.A.: Cpsgrader: synthesizing temporal logic testers for auto-grading an embedded systems laboratory. In: Mitra, T., Reineke, J. (eds.) 2014 International Conference on Embedded Software, EMSOFT 2014, New Delhi, India, October 12–17, 2014, pp. 24:1–24:10. ACM (2014)Google Scholar
  13. 13.
    Kane, A., Fuhrman, T.E., Koopman, P.: Monitor based oracles for cyber-physical system testing: practical experience report. In: 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2014, Atlanta, GA, USA, June 23–26, 2014, pp. 148–155. IEEE (2014)Google Scholar
  14. 14.
    Leucker, M., Schallhart, C.: A brief account of runtime verification. J. Logic Algebraic Program. 78(5), 293–303 (2008). CrossRefzbMATHGoogle Scholar
  15. 15.
    Luo, Q., Zhang, Y., Lee, C., Jin, D., Meredith, P.O., Serbanuta, T., Rosu, G.: Rv-monitor: efficient parametric runtime verification with simultaneous properties. In: Proceedings of Runtime Verification - 5th International Conference, RV 2014, Toronto, ON, Canada, September 22–25, 2014, pp. 285–300 (2014)Google Scholar
  16. 16.
    Navabpour, S., Joshi, Y., Wu, C.W.W., Berkovich, S., Medhat, R., Bonakdarpour, B., Fischmeister, S.: RiTHM: a tool for enabling time-triggered runtime verification for c programs. In: ACM Symposium on the Foundations of Software Engineering (FSE), pp. 603–606 (2013)Google Scholar
  17. 17.
    Quinlan, D.J., Schordan, M., Miller, B., Kowarschik, M.: Parallel object-oriented framework optimization. Concurr. Comput.: Pract. Exp. 16(2–3), 293–302 (2004)CrossRefGoogle Scholar
  18. 18.
    Reger, G., Cruz, H.C., Rydeheard, D.: MarQ: monitoring at runtime with QEA. In: Baier, C., Tinelli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 596–610. Springer, Heidelberg (2015). Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Univ. Grenoble Alpes, Inria, LIGGrenobleFrance
  2. 2.AIT Austrian Institute of Technology GmbHGrazAustria
  3. 3.University of ManchesterManchesterUK
  4. 4.University of LübeckLübeckGermany

Personalised recommendations