Abstract
The formal verification and validation of real-world, industrial critical hybrid flight controllers remains a very challenging task. An increasingly popular and quite successful alternative to formal verification is the use of optimization and reinforcement learning techniques to maximize some real-valued reward function encoding the robustness margin to the falsification of a property. In this paper we present an evaluation of a simple Monte-Carlo Tree Search property falsification algorithm, applied to select properties of a longitudinal hybrid flight control law: a threshold overshoot property, two frequential properties, and a discrete event-based property.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
By lack of space and since we are only considering planning problems in the deterministic and finite-horizon case, the definitions of policy, state-value function S, action value function Q and optimal policy synthesis are deliberately omitted.
- 2.
Disclaimer: some quantitative aspects of the results presented in this section have been omitted for industrial confidentiality reasons.
References
Akazaki, T., Liu, S., Yamagata, Y., Duan, Y., Hao, J.: Falsification of cyber-physical systems using deep reinforcement learning. In: Havelund, K., Peleska, J., Roscoe, B., de Vink, E. (eds.) FM 2018. LNCS, vol. 10951, pp. 456–465. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95582-7_27
Annpureddy, Y., Liu, C., Fainekos, G., Sankaranarayanan, S.: S-TaLiRo: a tool for temporal logic falsification for hybrid systems. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS 2011. LNCS, vol. 6605, pp. 254–257. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19835-9_21
Bouissou, O., Mimram, S., Chapoutot, A.: Hyson: set-based simulation of hybrid systems. In: Proceedings of the 23rd IEEE International Symposium on Rapid System Prototyping, RSP 2012, Tampere, Finland, 11–12 October, pp. 79–85 (2012). https://doi.org/10.1109/RSP.2012.6380694
Chen, X., Sankaranarayanan, S., Abraham, E.: Flow* 1.2 - more effective to play with hybrid systems. In: 2nd International Workshop on Applied Verification for Continuous and Hybrid Systems, ARCH@CPSWeek 2015, Seattle, WA, USA, 13 April, pp. 152–159 (2015). https://doi.org/10.29007/1w4t
Coulom, R.: Efficient selectivity and backup operators in Monte-Carlo tree search. In: van den Herik, H.J., Ciancarini, P., Donkers, H.H.L.M.J. (eds.) CG 2006. LNCS, vol. 4630, pp. 72–83. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-75538-8_7
Donzé, A.: Breach, a toolbox for verification and parameter synthesis of hybrid systems. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 167–170. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14295-6_17
Frehse, G., et al.: SpaceEx: scalable verification of hybrid systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 379–395. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22110-1_30
Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006). https://doi.org/10.1007/11871842_29
Kong, S., Gao, S., Chen, W., Clarke, E.: dReach: \(\delta \)-reachability analysis for hybrid systems. In: Baier, C., Tinelli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 200–205. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-46681-0_15
Lee, K., Kim, S.A., Choi, J., Lee, S.W.: Deep reinforcement learning in continuous action spaces: a case study in the game of simulated curling. In: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, 10–15 July, pp. 2943–2952 (2018). http://proceedings.mlr.press/v80/lee18b.html
Lee, R., et al.: Adaptive stress testing: finding failure events with reinforcement learning. CoRR abs/1811.02188 (2018). http://arxiv.org/abs/1811.02188
Schrammel, P., Jeannet, B.: From hybrid data-flow languages to hybrid automata: a complete translation. In: Hybrid Systems: Computation and Control, HSCC 2012, Beijing, China, 17–19 April, pp. 167–176 (2012). https://doi.org/10.1145/2185632.2185658
Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016). https://doi.org/10.1038/nature16961
Silvetti, S., Policriti, A., Bortolussi, L.: An active learning approach to the falsification of black box cyber-physical systems. In: Polikarpova, N., Schneider, S. (eds.) IFM 2017. LNCS, vol. 10510, pp. 3–17. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66845-1_1
Xiao, C., Mei, J., Müller, M.: Memory-augmented Monte Carlo tree search. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, 2–7 February, pp. 1455–1462 (2018). https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17139
Zhang, Z., Ernst, G., Sedwards, S., Arcaini, P., Hasuo, I.: Two-layered falsification of hybrid systems guided by Monte Carlo tree search. IEEE Trans. CAD Integr. Circ. Syst. 37(11), 2894–2905 (2018). https://doi.org/10.1109/TCAD.2018.2858463
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Delmas, R., Loquen, T., Boada-Bauxell, J., Carton, M. (2019). An Evaluation of Monte-Carlo Tree Search for Property Falsification on Hybrid Flight Control Laws. In: Zamani, M., Zufferey, D. (eds) Numerical Software Verification. NSV 2019. Lecture Notes in Computer Science(), vol 11652. Springer, Cham. https://doi.org/10.1007/978-3-030-28423-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-28423-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28422-0
Online ISBN: 978-3-030-28423-7
eBook Packages: Computer ScienceComputer Science (R0)