Abstract
Runtime monitoring is generally considered a light-weight alternative to formal verification. In safety-critical systems, however, the monitor itself is a critical component. For example, if the monitor is responsible for initiating emergency protocols, as proposed in a recent aviation standard, then the safety of the entire system critically depends on the correctness of the monitor. In this paper, we present a verification extension to the Lola monitoring language that extends the efficient specification of the monitor with Hoare-style annotations that guarantee the correctness of the monitor specification. We add two new operators, assume and assert, which specify assumptions of the monitor and expectations on its output, respectively. The validity of the annotations is established by an integrated SMT solver. We report on experience in applying the approach to specifications from the avionics domain, where the annotation with assumptions and assertions has lead to the discovery of safety-critical errors in specifications. The errors range from incorrect default values in offset computations to complex algorithmic errors that result in unexpected temporal patterns. We also report how verified specifications can be monitored efficiently at runtime.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Baumeister, J., Finkbeiner, B., Schwenger, M., Torfah, H.: FFGA stream-monitoring of real-time properties. ACM Trans. Embed. Comput. Syst. 18(5s), 88 (2019). https://doi.org/10.1145/3358220
Baumeister, J., Finkbeiner, B., Kruse, M., Schwenger, M.: Automatic optimizations for stream-based monitoring languages. In: Deshmukh, J., Ničković, D. (eds.) Runtime Verification, pp. 451–461. Springer, Cham (2020)
Baumeister, J., Finkbeiner, B., Schirmer, S., Schwenger, M., Torens, C.: RTLola cleared for take-off: monitoring autonomous aircraft. In: Lahiri, S.K., Wang, C. (eds.) Computer Aided Verification, pp. 28–39. Springer, Cham (2020)
Beckert, B., Hähnle, R., Schmitt, P.H. (eds.): Verification of Object-Oriented Software. The KeY Approach. LNCS, vol. 4334. Springer, Berlin (2007). https://doi.org/10.1007/978-3-540-69061-0
Berry, G.: The foundations of ESTEREL. In: Proof, Language and Interaction: Essays in Honour of Robin Milner, pp. 425–454. MIT Press, Cambridge (2000)
Bobot, F., Filliâtre, J.C., Marché, C., Paskevich, A.: Let’s verify this with why3. Int. J. Softw. Tools Technol. Transf. 17, 709–727 (2015)
Cluzeau, J.M., Henriquel, X., van Dijk, L., Gronskiy, A.: Concepts of design assurance for neural networks (CoDANN). Tech. Rep., EASA European Union Aviation Safety Agency (Mar 2020)
D’Angelo, B., Sankaranarayanan, S., Sanchez, C., Robinson, W., Finkbeiner, B., Sipma, H.B., Mehrotra, S., Manna, Z.: Lola: runtime monitoring of synchronous systems. In: 12th International Symposium on Temporal Representation and Reasoning (TIME’05), pp. 166–174 (2005). https://doi.org/10.1109/TIME.2005.26
Dauer, J.C., Finkbeiner, B., Schirmer, S.: Monitoring with verified guarantees. In: Feng, L., Fisman, D. (eds.) Runtime Verification, pp. 62–80. Springer, Cham (2021)
Faymonville, P., Finkbeiner, B., Schirmer, S., Torfah, H.: A stream-based specification language for network monitoring. In: Falcone, Y., Sánchez, C. (eds.) Runtime Verification, pp. 152–168. Springer, Cham (2016)
Finkbeiner, B., Oswald, S., Passing, N., Schwenger, M.: Verified rust monitors for lola specifications. In: Deshmukh, J., Ničković, D. (eds.) Runtime Verification, pp. 431–450. Springer, Cham (2020)
Floyd, R.W.: Assigning meanings to programs. In: Program Verification, pp. 65–81. Springer, Dordrecht (1993). https://doi.org/10.1007/978-94-011-1793-7_4
Gautier, T., Le Guernic, P., Besnard, L.: SIGNAL: a declarative language for synchronous programming of real-time systems. In: Kahn, G. (ed.) Functional Programming Languages and Computer Architecture, pp. 257–277. Springer, Berlin (1987)
Hagen, G., Tinelli, C.: Scaling up the formal verification of LUSTRE programs with SMT-based techniques. In: 2008 Formal Methods in Computer-Aided Design, pp. 1–9 (2008). https://doi.org/10.1109/FMCAD.2008.ECP.19
Halbwachs, N., Caspi, P., Raymond, P., Pilaud, D.: The synchronous data flow programming language LUSTRE. Proc. IEEE 79(9), 1305–1320 (1991). https://doi.org/10.1109/5.97300
Hoare, C.A.R.: An axiomatic basis for computer programming. Commun. ACM 12(10), 576–580 (1969). https://doi.org/10.1145/363235.363259
Jagadeesan, L.J., Puchol, C., Von Olnhausen, J.E.: Safety property verification of ESTEREL programs and applications to telecommunications software. In: Wolper, P. (ed.) Computer Aided Verification, pp. 127–140. Springer, Berlin (1995)
Leino, K.R.M.: Dafny: an automatic program verifier for functional correctness. In: Clarke, E.M., Voronkov, A. (eds.) Logic for Programming, Artificial Intelligence, and Reasoning, pp. 348–370. Springer, Berlin (2010)
Müller, P., Schwerhoff, M., Summers, A.J.: Viper: a verification infrastructure for permission-based reasoning. In: Jobstmann, B., Leino, K.R.M. (eds.) Verification, Model Checking, and Abstract Interpretation, pp. 41–62. Springer, Berlin (2016)
Nagarajan, P., Kannan, S.K., Torens, C., Vukas, M.E., Wilber, G.F.: ASTM F3269 – an Industry Standard on Run Time Assurance for Aircraft Systems. https://arc.aiaa.org/doi/abs/10.2514/6.2021-0525
Nenzi, L., Bortolussi, L., Ciancia, V., Loreti, M., Massink, M.: Qualitative and quantitative monitoring of spatio-temporal properties. In: Bartocci, E., Majumdar, R. (eds.) Runtime Verification, pp. 21–37. Springer, Cham (2015)
Pike, L., Goodloe, A., Morisset, R., Niller, S.: Copilot: a hard real-time runtime monitor. In: Barringer, H., Falcone, Y., Finkbeiner, B., Havelund, K., Lee, I., Pace, G., Roşu, G., Sokolsky, O., Tillmann, N. (eds.) Runtime Verification, pp. 345–359. Springer, Berlin (2010)
Reactive Systems Group, C.: RTLola. https://github.com/reactive-systems/RTLola-Frontend, https://github.com/reactive-systems/RTLola-Interpreter (2023)
Reinbacher, T., Rozier, K.Y., Schumann, J.: Temporal-logic based runtime observer pairs for system health management of real-time systems. In: Ábrahám, E., Havelund, K. (eds.) Tools and Algorithms for the Construction and Analysis of Systems, pp. 357–372. Springer, Berlin (2014)
Schirmer, S.: Runtime monitoring with Lola. Master’s thesis, Saarland University (Dec. 2016)
Schirmer, S., Torens, C., Adolf, F.: Formal monitoring of risk-based geofences. In: 2018 AIAA Information Systems-AIAA Infotech @ Aerospace (2018). https://arc.aiaa.org/doi/abs/10.2514/6.2018-1986
Seto, D., Krogh, B., Sha, L., Chutinan, A.: The simplex architecture for safe online control system upgrades. In: Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207), vol. 6, pp. 3504–3508 (1998). https://doi.org/10.1109/ACC.1998.703255
Song, Y., Chin, W.N.: A synchronous effects logic for temporal verification of pure ESTEREL. In: Henglein, F., Shoham, S., Vizel, Y. (eds.) Verification, Model Checking, and Abstract Interpretation, pp. 417–440. Springer, Cham (2021)
Funding
Open Access funding enabled and organized by Projekt DEAL. This work was partially supported by the German Research Foundation (DFG) as part of the Collaborative Research Center Foundations of Perspicuous Software Systems (TRR 248, 389792660), by the European Research Council (ERC) Grant OSARES (No. 683300), and by the Aviation Research Program LuFo of the German Federal Ministry for Economic Affairs and Energy as part of “Volocopter Sicherheitstechnologie zur robusten eVTOL Flugzustandsabsicherung durch formales Monitoring” (No. 20Q1963C).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Baumeister, J., Dauer, J.C., Finkbeiner, B. et al. Monitoring with verified guarantees. Int J Softw Tools Technol Transfer 25, 593–616 (2023). https://doi.org/10.1007/s10009-023-00712-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10009-023-00712-3