Model Checking at Scale: Automated Air Traffic Control Design Space Exploration

  • Marco GarioEmail author
  • Alessandro Cimatti
  • Cristian Mattarei
  • Stefano Tonetta
  • Kristin Yvonne RozierEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9780)


Many possible solutions, differing in the assumptions and implementations of the components in use, are usually in competition during early design stages. Deciding which solution to adopt requires considering several trade-offs. Model checking represents a possible way of comparing such designs, however, when the number of designs is large, building and validating so many models may be intractable.

During our collaboration with NASA, we faced the challenge of considering a design space with more than 20,000 designs for the NextGen air traffic control system. To deal with this problem, we introduce a compositional, modular, parameterized approach combining model checking with contract-based design to automatically generate large numbers of models from a possible set of components and their implementations. Our approach is fully automated, enabling the generation and validation of all target designs. The 1,620 designs that were most relevant to NASA were analyzed exhaustively. To deal with the massive amount of data generated, we apply novel data-analysis techniques that enable a rich comparison of the designs, including safety aspects. Our results were validated by NASA system designers, and helped to identify novel as well as known problematic configurations.


Model Check Design Space Linear Temporal Logic Software Product Line Fault Tree 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Eric & Wendy Schmidt Data Science for Social Good, University of Chicago.
  2. 2.
    NASA airspace operations and safety program.
  3. 3.
  4. 4.
    Project webpage: Formal methods for automated airspace concepts.
  5. 5.
    NextGen, May 2016.
  6. 6.
    Abadi, M., Lamport, L.: Composing specifications. ACM Trans. Program. Lang. Syst. 15(1), 73–132 (1993)CrossRefGoogle Scholar
  7. 7.
    ARP4754A guidelines for development of civil aircraft and systems. In: SAE, December 2010Google Scholar
  8. 8.
    ARP4761 guidelines and methods for conducting the safety assessment process on civil airborne systems and equipment. In: SAE, December 1996Google Scholar
  9. 9.
    Bauer, C., Lagadec, K., Bès, C., Mongeau, M.: Flight control system architecture optimization for fly-by-wire airliners. J. Guidance Control Dyn. 30(4), 1023–1029 (2007)CrossRefGoogle Scholar
  10. 10.
    Bittner, B., Bozzano, M., Cavada, R., Cimatti, A., Gario, M., Griggio, A., Mattarei, C., Micheli, A., Zampedri, G.: The xSAP safety analysis platform. In: Chechik, M., Raskin, J.-F. (eds.) TACAS 2016. LNCS, vol. 9636, pp. 533–539. Springer, Heidelberg (2016)CrossRefGoogle Scholar
  11. 11.
    Bozzano, M., Cimatti, A., Griggio, A., Mattarei, C.: Efficient anytime techniques for model-based safety analysis. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 603–621. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  12. 12.
    Bozzano, M., Cimatti, A., Mattarei, C.: Automated analysis of reliability architectures. In: 18th International Conference on Engineering of Complex Computer Systems (ICECCS), pp. 198–207. IEEE, July 2013Google Scholar
  13. 13.
    Bryant, R.E.: Graph-based algorithms for Boolean function manipulation. IEEE Trans. Comput. 100(8), 677–691 (1986)CrossRefzbMATHGoogle Scholar
  14. 14.
    Butler, R.W., Hagen, G., Maddalon, J.M.: The Chorus conflict and loss of separation resolution algorithms. Technical report, Technical Memorandum NASA/TM-2013-218030, NASA, Langley Research Center, Hampton VA 23681–2199, USA (2013)Google Scholar
  15. 15.
    Can, A.B., Bultan, T., Lindvall, M., Lux, B., Topp, S.: Eliminating synchronization faults in air traffic control software via design for verification with concurrency controllers. Autom. Softw. Eng. 14(2), 129–178 (2007)CrossRefGoogle Scholar
  16. 16.
    Cavada, R., Cimatti, A., Dorigatti, M., Griggio, A., Mariotti, A., Micheli, A., Mover, S., Roveri, M., Tonetta, S.: The nuXmv symbolic model checker. In: Biere, A., Bloem, R. (eds.) CAV 2014. LNCS, vol. 8559, pp. 334–342. Springer, Heidelberg (2014)Google Scholar
  17. 17.
    Cimatti, A., Dorigatti, M., Tonetta, S.: OCRA: a tool for checking the refinement of temporal contracts. In: ASE, pp. 702–705. IEEE (2013)Google Scholar
  18. 18.
    Cimatti, A., Tonetta, S.: Contracts-refinement proof system for component-based embedded systems. Sci. Comput. Program. 97, 333–348 (2015)CrossRefGoogle Scholar
  19. 19.
    Classen, A., Heymans, P., Schobbens, P.Y., Legay, A.: Symbolic model checking of software product lines. In: Proceedings of the 33rd International Conference on Software Engineering, pp. 321–330. ACM (2011)Google Scholar
  20. 20.
    von Essen, C., Giannakopoulou, D.: Analyzing the next generation airborne collision avoidance system. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014 (ETAPS). LNCS, vol. 8413, pp. 620–635. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  21. 21.
    Gario, M., Micheli, A.: pySMT: a solver-agnostic library for fast prototyping of SMT-based algorithms. In: SMT-Workshop (2015)Google Scholar
  22. 22.
    Hagen, G., Butler, R., Maddalon, J.: Stratway: a modular approach to strategic conflict resolution. In: Preceedings of 11th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference, Virgina Beach, VA (2011)Google Scholar
  23. 23.
    Idris, H.R., Shen, N., Wing, D.J.: Improving separation assurance stability through trajectory flexibility preservation. In: 10th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference, p. 9011 (2010)Google Scholar
  24. 24.
    Jeannin, J.-B., Ghorbal, K., Kouskoulas, Y., Gardner, R., Schmidt, A., Zawadzki, E., Platzer, A.: A formally verified hybrid system for the next-generation airborne collision avoidance system. In: Baier, C., Tinelli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 21–36. Springer, Heidelberg (2015)Google Scholar
  25. 25.
    Karr, D.A., Vivona, R.A., Roscoe, D.A., DePascale, S.M., Wing, D.J.: Autonomous operations planner: a flexible platform for research in flight-deck support for airborne self-separation. In: 12th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference and 14th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, p. 5417 (2012)Google Scholar
  26. 26.
    Lauderdale, T., Lewis, T., Prevot, T., Ballin, M., Aweiss, A., Guerreiro, N.: Function allocation for separation assurance: research plan, NASA HQ Project Overview, August 2014Google Scholar
  27. 27.
    Loos, S.M., Renshaw, D., Platzer, A.: Formal verification of distributed aircraft controllers. In: Proceedings of the 16th International Conference on Hybrid Systems: Computation and Control, HSCC 2013, pp. 125–130. ACM, New York (2013).
  28. 28.
    Mattarei, C., Cimatti, A., Gario, M., Tonetta, S., Kristin Yvonne, R.: Comparing different functional allocations in automated air traffic control design. In: Formal Methods in Computer-Aided Design (FMCAD15) (2015)Google Scholar
  29. 29.
    Mehlitz, P.: Trust your model-verifying aerospace system models with Java PathFinder. In: IEEE/Aero (2008)Google Scholar
  30. 30.
    Muñoz, C., Carreño, V.A., Dowek, G.: Formal analysis of the operational concept for the small aircraft transportation system. In: Butler, M., Jones, C.B., Romanovsky, A., Troubitsyna, E. (eds.) Rigorous Development of Complex Fault-Tolerant Systems. LNCS, vol. 4157, pp. 306–325. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  31. 31.
    Muñoz, C., Siminiceanu, R., Carreño, V., Dowek, G.: KB3D Reference Manual-Version 1. NASA (2005)Google Scholar
  32. 32.
    Vesely, W., Goldberg, F., Roberts, N., Haasl, D.: Fault Tree Handbook. Technical report NUREG-0492, Systems and Reliability Research Office of Nuclear Regulatory Research U.S. (1981)Google Scholar
  33. 33.
    Wing, D.J., Ballin, M.G., Krishnamurthy, K.: Pilot in command: a feasibility assessment of autonomous flight management operations. In: 24th International Congress of the Aeronautical Sciences (2004)Google Scholar
  34. 34.
    Zhao, Y., Rozier, K.Y.: Formal specification and verification of a coordination protocol for an automated air traffic control system. Sci. Comput. Program. J. 96(3), 337–353 (2014)CrossRefGoogle Scholar
  35. 35.
    Zhao, Y., Rozier, K.Y.: Probabilistic model checking for comparative analysis of automated air traffic control systems. In: Proceedings of the 33rd IEEE/ACM International Conference On Computer-Aided Design (ICCAD 2014), pp. 690–695. IEEE/ACM, San Jose, November 2014Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Fondazione Bruno KesslerTrentoItaly
  2. 2.Iowa State UniversityAmesUSA

Personalised recommendations