Qualification of a Model Checker for Avionics Software Verification

  • Lucas Wagner
  • Alain Mebsout
  • Cesare Tinelli
  • Darren Cofer
  • Konrad Slind
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10227)


Formal methods tools have been shown to be effective at finding defects in safety-critical systems, including avionics systems in commercial aircraft. The publication of DO-178C and the accompanying formal methods supplement DO-333 provide guidance for aircraft manufacturers and equipment suppliers who wish to obtain certification credit for the use of formal methods for software development and verification.

However, there are still a number of issues that must be addressed before formal methods tools can be injected into the design process for avionics systems. DO-178C requires that a tool used to meet certification objectives be qualified to demonstrate that its output can be trusted. The qualification of formal methods tools is a relatively new concept presenting unique challenges for both formal methods researchers and software developers in the aerospace industry.

This paper presents the results of a recent project studying the qualification of formal methods tools. We have identified potential obstacles to their qualification and proposed mitigation strategies. We have conducted two case studies based on different qualification approaches for an open source formal verification tool, the Kind 2 model checker. The first case study produced a qualification package for Kind 2. The second demonstrates the feasibility of independently verifying the output of Kind 2 through the generation of proof certificates and verifying these certificates with a qualified proof checker, in lieu of qualifying the model checker itself.


Qualification Certification Model checking Software verification 



This work was funded by NASA contract NNL14AA06C.


  1. 1.
    RTCA DO-178C: Software considerations in airborne systems and equipment certification, Washington, DC (2011)Google Scholar
  2. 2.
    Woodcock, J., Larsen, P.G., Bicarregui, J., Fitzgerald, J.S.: Formal methods: practice and experience. ACM Comput. Surv. 41, 19 (2009)CrossRefGoogle Scholar
  3. 3.
    RTCA DO-333: Formal methods supplement to DO-178C and DO-278A, Washington, DC (2011)Google Scholar
  4. 4.
    RTCA DO-330: Software tool qualification considerations, Washington, DC (2011)Google Scholar
  5. 5.
    Cofer, D., Miller, S.: DO-333 certification case studies. In: Badger, J.M., Rozier, K.Y. (eds.) NFM 2014. LNCS, vol. 8430, pp. 1–15. Springer, Cham (2014). doi: 10.1007/978-3-319-06200-6_1 CrossRefGoogle Scholar
  6. 6.
    Cofer, D., Klein, G., Slind, K., Wiels, V.: Qualification of formal methods tools (Dagstuhl seminar 15182). Dagstuhl Rep. 5, 142–159 (2015)Google Scholar
  7. 7.
    OCamlPro: Alt-ergo (2013).
  8. 8.
    AdaCore: SPARK Pro (2014).
  9. 9.
    Leroy, X.: A formally verified compiler back-end. J. Autom. Reason. 43, 363–446 (2009)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Camus, J.L., DeWalt, M.P., Pothon, F., Ladier, G., Boulanger, J.L., Blanquart, J.P., Quere, P., Ricque, B., Gassino, J.: Tool qualification in multiple domains: status and perspectives. In: Embedded Real Time Software and Systems, Toulouse, France, 5–7 February, vol. 7991. Springer (2014)Google Scholar
  11. 11.
    Miller, S.P., Whalen, M.W., Cofer, D.D.: Software model checking takes off. Commun. ACM 53, 58–64 (2010)CrossRefGoogle Scholar
  12. 12.
    Champion, A., Mebsout, A., Sticksel, C., Tinelli, C.: The Kind 2 model checker. In: Chaudhuri, S., Farzan, A. (eds.) CAV 2016. LNCS, vol. 9780, pp. 510–517. Springer, Cham (2016). doi: 10.1007/978-3-319-41540-6_29 Google Scholar
  13. 13.
    NASA: Qualification of Formal Methods Tools Under DO-330 (2017).
  14. 14.
    Mebsout, A., Tinelli, C.: Proof certificates for SMT-based model checkers for infinite-state systems. In: FMCAD, Mountain View, California, USA, October 2016.
  15. 15.
    Halbwachs, N., Caspi, P., Raymond, P., Pilaud, D.: The synchronous dataflow programming language LUSTRE. In: Proceedings of the IEEE, pp. 1305–1320 (1991)Google Scholar
  16. 16.
    de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-78800-3_24 CrossRefGoogle Scholar
  17. 17.
    Barrett, C., Conway, C.L., Deters, M., Hadarean, L., Jovanović, D., King, T., Reynolds, A., Tinelli, C.: CVC4. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 171–177. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-22110-1_14 CrossRefGoogle Scholar
  18. 18.
    Stump, A., Oe, D., Reynolds, A., Hadarean, L., Tinelli, C.: SMT proof checking using a logical framework. Form. Methods Syst. Des. 41, 91–118 (2013)CrossRefMATHGoogle Scholar
  19. 19.
    Gacek, A.: JKind - a Java implementation of the KIND model checker (2014).

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Lucas Wagner
    • 1
  • Alain Mebsout
    • 2
  • Cesare Tinelli
    • 2
  • Darren Cofer
    • 1
  • Konrad Slind
    • 1
  1. 1.Advanced Technology CenterRockwell CollinsCedar RapidsUSA
  2. 2.The University of IowaIowa CityUSA

Personalised recommendations