Abstract
Runtime verification (RV) enables monitoring systems at runtime, to detect property violations early and limit their potential consequences. This paper presents an end-to-end framework to capture requirements in structured natural language and generate monitors that capture their semantics faithfully. We leverage NASA’s Formal Requirement Elicitation Tool (fret), and the RV system Copilot. We extend fret with mechanisms to capture additional information needed to generate monitors, and introduce Ogma, a new tool to bridge the gap between fret and Copilot. With this framework, users can write requirements in an intuitive format and obtain real-time C monitors suitable for use in embedded systems. Our toolchain is available as open source.
Author contributed to this work prior to joining AWS.
Chapter PDF
References
Copilot. https://github.com/Copilot-Language/copilot/. Accessed Oct 04, 2021
FRET: Formal Requirements Elicitation Tool. https://github.com/NASA-SW-VnV/fret/. Accessed Oct 04, 2021
FRET: Formal Requirements Elicitation Tool - User Manual. https://github.com/NASA-SW-VnV/fret/blob/master/fret-electron/docs/_media/userManual.md. See Section “Exporting for Analysis”. Accessed Oct 04, 2021
Ogma. https://github.com/nasa/ogma/. Accessed Oct 04, 2021
E. Bartocci, Y. Falcone, A. Francalanza, and G. Reger. Introduction to runtime verification. In Lectures on Runtime Verification - Introductory and Advanced Topics, volume 10457 of Lecture Notes in Computer Science, pages 1–33. Springer, 2018
J. Baumeister, B. Finkbeiner, S. Schirmer, M. Schwenger, and C. Torens. RTLola cleared for take-off: Monitoring autonomous aircraft. In S. K. Lahiri and C. Wang, editors, Computer Aided Verification, pages 28–39, Cham, 2020. Springer International Publishing
S. Biewer, B. Finkbeiner, H. Hermanns, M. A. Köhl, Y. Schnitzer, and M. Schwenger. RTLola on board: Testing real driving emissions on your phone. In J. F. Groote and K. G. Larsen, editors, Tools and Algorithms for the Construction and Analysis of Systems, pages 365–372, Cham, 2021. Springer International Publishing
A. Boteanu, T. Howard, J. Arkin, and H. Kress-Gazit. A model for verifiable grounding and execution of complex natural language instructions. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2649–2654, 2016
K. Claessen and J. Hughes. QuickCheck: a lightweight tool for random testing of Haskell programs. ACM Sigplan Notices, 46(4):53–64, 2011
D. Cohen, M. S. Feather, K. Narayanaswamy, and S. S. Fickas. Automatic monitoring of software requirements. In Proceedings of the 19th International Conference on Software Engineering, pages 602–603, 1997
C. Elliott. On example models and challenges ahead for the evaluation of complex cyber-physical systems with state of the art formal methods V&V, Lockheed Martin Skunk Works. In A. F. R. Laboratory, editor, Safe & Secure Systems and Software Symposium (S5), 2015
C. Elliott. An example set of cyber-physical V&V challenges for S5, Lockheed Martin Skunk Works. In A. F. R. Laboratory, editor, Safe & Secure Systems and Software Symposium (S5), 2016
P. Faymonville, B. Finkbeiner, M. Schledjewski, M. Schwenger, M. Stenger, L. Tentrup, and H. Torfah. StreamLAB: Stream-based monitoring of cyber-physical systems. In I. Dillig and S. Tasiran, editors, Computer Aided Verification, pages 421–431, Cham, 2019. Springer International Publishing
A. Fifarek, L. G. Wagner, J. A. Hoffman, B. D. Rodes, M. A. Aiello, and J. A. Davis. SpeAR v2.0: Formalized past LTL specification and analysis of requirements. In NASA Formal Methods - 9th International Symposium, NFM 2017, Moffett Field, CA, USA, May 16–18, 2017, Proceedings, pages 420–426, 2017
A. Gacek, J. Backes, M. Whalen, L. Wagner, and E. Ghassabani. The jk ind model checker. In International Conference on Computer Aided Verification, pages 20–27. Springer, 2018
S. Ghosh, D. Elenius, W. Li, P. Lincoln, N. Shankar, and W. Steiner. ARSENAL: automatic requirements specification extraction from natural language. In S. Rayadurgam and O. Tkachuk, editors, NASA Formal Methods - 8th International Symposium, NFM 2016, Minneapolis, MN, USA, June 7-9, 2016, Proceedings, volume 9690 of Lecture Notes in Computer Science, pages 41–46. Springer, 2016
D. Giannakopoulou, T. Pressburger, A. Mavridou, J. Rhein, J. Schumann, and N. Shi. Formal requirements elicitation with FRET. In Joint Proceedings of REFSQ-2020 Workshops, Doctoral Symposium, Live Studies Track, and Poster Track co-located with the 26th International Conference on Requirements Engineering: Foundation for Software Quality (REFSQ 2020), 2020
D. Giannakopoulou, T. Pressburger, A. Mavridou, and J. Schumann. Automated formalization of structured natural language requirements. Inf. Softw. Technol., 137:106590, 2021
A. Goodloe and L. Pike. Monitoring distributed real-time systems: A survey and future directions. Technical Report NASA/CR-2010-216724, NASA Langley Research Center, July 2010
N. Halbwachs, P. Caspi, P. Raymond, and D. Pilaud. The synchronous data flow programming language LUSTRE. Proceedings of the IEEE, 79(9):1305–1320, 1991
K. Havelund and A. Goldberg. Verify Your Runs, pages 374–383. Springer, Berlin Heidelberg, Berlin, Heidelberg, 2008
B. Jeannet and F. Gaucher. Debugging embedded systems requirements with STIMULUS: an automotive case-study. In 8th European Congress on Embedded Real Time Software and Systems (ERTS 2016), Toulouse, France, Jan. 2016
J. C. Knight. Safety critical systems: Challenges and directions. In Proceedings of the 24th International Conference on Software Engineering, ICSE ’02, pages 547–550. ACM, 2002
C. Lignos, V. Raman, C. Finucane, M. Marcus, and H. Kress-Gazit. Provably correct reactive control from natural language. Auton. Robots, 38(1):89–105, jan 2015
L. Lúcio, S. Rahman, C.-H. Cheng, and A. Mavin. Just formal enough? Automated analysis of EARS requirements. In NASA Formal Methods - 9th International Symposium, NFM 2017, Moffett Field, CA, USA, May 16-18, 2017, Proceedings, pages 427–434, May 2017
A. Mavridou, H. Bourbouh, P. L. Garoche, and M. Hejase. Evaluation of the FRET and CoCoSim tools on the ten Lockheed Martin cyber-physical challenge problems. Technical Report TM-2019-220374, National Aeronautics and Space Administration, February 2020
A. Mavridou, H. Bourbouh, D. Giannakopoulou, T. Pressburger, M. Hejase, P.-L. Garoche, and J. Schumann. The ten Lockheed Martin cyber-physical challenges: Formalized, analyzed, and explained. In 2020 IEEE 28th International Requirements Engineering Conference (RE), pages 300–310, 2020
P. Moosbrugger, K. Y. Rozier, and J. Schumann. R2U2: monitoring and diagnosis of security threats for unmanned aerial systems. Formal Methods in System Design, 51(1):31–61, 2017
I. Perez, F. Dedden, and A. Goodloe. Copilot 3. Technical Report NASA/TM–2020–220587, NASA Langley Research Center, April 2020
I. Perez, A. Goodloe, and W. Edmonson. Fault-tolerant swarms. In 2019 IEEE International Conference on Space Mission Challenges for Information Technology (SMC-IT), pages 47–54. IEEE, 2019
I. Perez and A. E. Goodloe. Fault-tolerant functional reactive programming (extended version). Journal of Functional Programming, 30, 2020
I. Perez, A. Mavridou, T. Pressburger, A. Goodloe, and D. Giannakopoulou. Artifact for Automated Translation of Natural Language Requirements to Runtime Monitors. https://doi.org/10.5281/zenodo.5888956. Accessed Jan 21, 2022
I. Perez, A. Mavridou, T. Pressburger, A. Goodloe, and D. Giannakopoulou. Integrating FRET with Copilot: Automated Translation of Natural Language Requirements to Runtime Monitors. Technical Report NASA/TM–20220000049, NASA, January 2022
I. Perez and H. Nilsson. Runtime verification and validation of functional reactive systems. Journal of Functional Programming, 30:e28, 2020
L. Pike, A. Goodloe, R. Morisset, and S. Niller. Copilot: A hard real-time runtime monitor. In Proceedings of the 1st Intl. Conference on Runtime Verification, LNCS. Springer, November 2010
L. Pike, N. Wegmann, S. Niller, and A. Goodloe. Copilot: Monitoring embedded systems. Innov. Syst. Softw. Eng., 9(4):235–255, Dec. 2013
T. Reinbacher, K. Y. Rozier, and J. Schumann. Temporal-logic based runtime observer pairs for system health management of real-time systems. In E. Ábrahám and K. Havelund, editors, Tools and Algorithms for the Construction and Analysis of Systems, pages 357–372, Berlin, Heidelberg, 2014. Springer, Berlin Heidelberg
J. Schumann, P. Moosbrugger, and K. Y. Rozier. R2u2: monitoring and diagnosis of security threats for unmanned aerial systems. In Runtime Verification, pages 233–249. Springer, 2015
H. Torfah. Stream-based monitors for real-time properties. In B. Finkbeiner and L. Mariani, editors, Runtime Verification - 19th International Conference, RV 2019, Porto, Portugal, October 8-11, 2019, Proceedings, volume 11757 of Lecture Notes in Computer Science, pages 91–110. Springer, 2019
J. Wilmot. A core flight software system. In Proceedings of the 3rd IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis, CODES+ISSS ’05, pages 13–14, New York, NY, USA, 2005. ACM
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2022 The Author(s)
About this paper
Cite this paper
Perez, I., Mavridou, A., Pressburger, T., Goodloe, A., Giannakopoulou, D. (2022). Automated Translation of Natural Language Requirements to Runtime Monitors. In: Fisman, D., Rosu, G. (eds) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2022. Lecture Notes in Computer Science, vol 13243. Springer, Cham. https://doi.org/10.1007/978-3-030-99524-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-99524-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-99523-2
Online ISBN: 978-3-030-99524-9
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://www.etaps.org/
