Abstract
Falsification testing is a popular method for efficiently identifying inputs that witness the violation of cyber-physical system (CPS) specifications. The generated counterexamples are used to locate and explain faults and debug the system. However, CPS rarely operate in unconstrained environments and not all counterexamples have the same value. On one hand, faults resulting from common system usage are more likely to happen in practice than faults triggered by esoteric inputs. On the other hand, some faults can have more severe consequences than others. Together, the probability and the severity of a fault determine its risk, an important aspect that is neglected by the existing falsification testing techniques. We propose a new falsification testing methodology that is aware of the system’s expected usage and the severity associated to different faulty behaviors. Given a user profile in the form of a stochastic hybrid automaton, an associated severity degree measure, an executable black-box implementation of the CPS and its formalized requirements, we provide a test generation method that (1) uses efficient randomized methods to generate multiple violating traces, and (2) estimates the probability and the expected severity, and hence the expected risk of each counterexample, thus providing their ranking to the engineer.
This research received funding from the Villum Investigator Grant S4OS of Professor Kim G. Larsen.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ábrahám, E., Becker, B., Dehnert, C., Jansen, N., Katoen, J., Wimmer, R.: Counterexample generation for discrete-time markov models: An introductory survey. In: Formal Methods for Executable Software Models - 14th International School on Formal Methods for the Design of Computer, Communication, and Software Systems, SFM 2014, Bertinoro, Italy, June 16–20, 2014, Advanced Lectures, pp. 65–121 (2014)
Aljazzar, H., Leitner-Fischer, F., Leue, S., Simeonov, D.: Dipro - A tool for probabilistic counterexample generation. In: Model Checking Software - 18th International SPIN Workshop, Snowbird, UT, USA, July 14–15, 2011. Proceedings, pp. 183–187 (2011)
Bartocci, E., Bloem, R., Maderbacher, B., Manjunath, N., Nickovic, D.: Adaptive testing for specification coverage in CPS models. In: 7th IFAC Conference on Analysis and Design of Hybrid Systems, ADHS 2021, Brussels, Belgium, July 7–9, 2021, pp. 229–234 (2021)
Bogomolov, S., Frehse, G., Gurung, A., Li, D., Martius, G., Ray, R.: Falsification of hybrid systems using symbolic reachability and trajectory splicing. In: Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control, HSCC 2019, Montreal, QC, Canada, April 16–18, 2019, pp. 1–10 (2019)
Bulychev, P.E., et al.: UPPAAL-SMC: statistical model checking for priced timed automata. In: Proceedings 10th Workshop on Quantitative Aspects of Programming Languages and Systems, QAPL 2012, Tallinn, Estonia, 31 March and 1 April 2012, pp. 1–16 (2012)
Chaturvedi, D.K.: Modeling and simulation of systems using MATLAB® and Simulink®. CRC Press (2017)
Clarke, E.M., Zuliani, P.: Statistical model checking for cyber-physical systems. In: Automated Technology for Verification and Analysis, 9th International Symposium, ATVA 2011, Taipei, Taiwan, October 11–14, 2011. Proceedings, pp. 1–12 (2011)
David, A., et al.: Statistical model checking for stochastic hybrid systems. Electron. Proc. Theor. Comput. Sci. 92, 122–136 (2012). https://doi.org/10.4204/eptcs.92.9
Donzé, A., Maler, O.: Robust satisfaction of temporal logic over real-valued signals. In: Chatterjee, K., Henzinger, T.A. (eds.) Formal Modeling and Analysis of Timed Systems, pp. 92–106. Springer, Berlin Heidelberg, Berlin, Heidelberg (2010)
Han, T., Katoen, J.: Counterexamples in probabilistic model checking. In: Tools and Algorithms for the Construction and Analysis of Systems, 13th International Conference, TACAS 2007, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2007 Braga, Portugal, March 24 - April 1, 2007, Proceedings, pp. 72–86 (2007)
Jégourel, C., Legay, A., Sedwards, S.: Importance splitting for statistical model checking rare properties. In: Computer Aided Verification - 25th International Conference, CAV 2013, Saint Petersburg, Russia, July 13–19, 2013. Proceedings, pp. 576–591 (2013)
Kiviriga, A., Larsen, K.G., Nyman, U.: Randomized reachability analysis in uppaal: Fast error detection in timed systems. In: Lluch Lafuente, A., Mavridou, A. (eds.) Formal Methods for Industrial Critical Systems, pp. 149–166. Springer International Publishing, Cham (2021)
Larsen, K.G., Legay, A., Mikucionis, M., Poulsen, D.B.: Importance splitting in uppaal. In: Margaria, T., Steffen, B. (eds.) Leveraging Applications of Formal Methods, Verification and Validation. Adaptation and Learning - 11th International Symposium, ISoLA 2022, Rhodes, Greece, October 22–30, 2022, Proceedings, Part III. Lecture Notes in Computer Science, vol. 13703, pp. 433–447. Springer (2022). https://doi.org/10.1007/978-3-031-19759-8_26
Lindemann, L., Jiang, L., Matni, N., Pappas, G.J.: Risk of stochastic systems for temporal logic specifications. ACM Trans. Embed. Comput. Syst. 22(3), 54:1–54:31 (2023). https://doi.org/10.1145/3580490
Lindemann, L., Matni, N., Pappas, G.J.: STL robustness risk over discrete-time stochastic processes. In: 2021 60th IEEE Conference on Decision and Control (CDC), Austin, TX, USA, December 14–17, 2021, pp. 1329–1335 (2021)
Maler, O., Nickovic, D.: Monitoring temporal properties of continuous signals. In: Lakhnech, Y., Yovine, S. (eds.) Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems, pp. 152–166. Springer, Berlin Heidelberg, Berlin, Heidelberg (2004)
Nghiem, T., Sankaranarayanan, S., Fainekos, G., Ivancic, F., Gupta, A., Pappas, G.J.: Monte-carlo techniques for falsification of temporal properties of non-linear hybrid systems. In: Proceedings of the 13th ACM International Conference on Hybrid Systems: Computation and Control, HSCC 2010, Stockholm, Sweden, April 12–15, 2010, pp. 211–220 (2010)
Ničković, D., Yamaguchi, T.: Rtamt: online robustness monitors from STL. In: Hung, D.V., Sokolsky, O. (eds.) Automated Technology for Verification and Analysis, pp. 564–571. Springer International Publishing, Cham (2020)
Rubino, G., Tuffin, B.: Rare event simulation using Monte Carlo methods. John Wiley & Sons (2009)
Sen, K., Viswanathan, M., Agha, G.: Statistical model checking of black-box probabilistic systems. In: Computer Aided Verification, 16th International Conference, CAV 2004, Boston, MA, USA, July 13–17, 2004, Proceedings, pp. 202–215 (2004)
Sen, K., Viswanathan, M., Agha, G.: Statistical model checking of black-box probabilistic systems. In: Alur, R., Peled, D.A. (eds.) Computer Aided Verification, pp. 202–215. Springer, Berlin Heidelberg, Berlin, Heidelberg (2004)
Younes, H.L.S.: Verification and Planning for Stochastic Processes with Asynchronous Events. Ph.D. thesis (2004)
Younes, H.L.S., Simmons, R.G.: Probabilistic verification of discrete event systems using acceptance sampling. In: Computer Aided Verification, 14th International Conference, CAV 2002, Copenhagen, Denmark, July 27–31, 2002, Proceedings, pp. 223–235 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kiviriga, A., Larsen, K.G., Nickovic, D., Nyman, U. (2023). Usage-and Risk-Aware Falsification Testing for Cyber-Physical Systems. In: Petrucci, L., Sproston, J. (eds) Formal Modeling and Analysis of Timed Systems. FORMATS 2023. Lecture Notes in Computer Science, vol 14138. Springer, Cham. https://doi.org/10.1007/978-3-031-42626-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-42626-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42625-4
Online ISBN: 978-3-031-42626-1
eBook Packages: Computer ScienceComputer Science (R0)