Skip to main content
Book cover

International Conference on Tests and Proofs

TAP 2015: Tests and Proofs pp 92–108Cite as

Software Validation via Model Animation

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 9154)

Abstract

This paper explores a new approach to validating software implementations that have been produced from formally-verified algorithms. Although visual inspection gives some confidence that the implementations faithfully reflect the formal models, it does not provide complete assurance that the software is correct. The proposed approach, which is based on animation of formal specifications, compares the outputs computed by the software implementations on a given suite of input values to the outputs computed by the formal models on the same inputs, and determines if they are equal up to a given tolerance. The approach is illustrated on a prototype air traffic management system that computes simple kinematic trajectories for aircraft. Proofs for the mathematical models of the system’s algorithms are carried out in the Prototype Verification System (PVS). The animation tool PVSio is used to evaluate the formal models on a set of randomly generated test cases. Output values computed by PVSio are compared against output values computed by the actual software. This comparison improves the assurance that the translation from formal models to code is faithful and that, for example, floating point errors do not greatly affect correctness and safety properties.

Keywords

  • Test Point
  • Software Implementation
  • Symbolic Execution
  • Generate Test Case
  • Ground Speed

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-21215-9_6
  • Chapter length: 17 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   44.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-21215-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   59.99
Price excludes VAT (USA)
Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Federal Aviation Administration. Airworthiness approval of automatic dependent surveillance-broadcast (ADS-B) out systems. Advisory Circular AC 20–165A, FAA (November 2012)

    Google Scholar 

  2. Aichernig, B.K., Gerstinger, A., Aster, R.: Formal specification techniques as a catalyst in validation. In: Fifth IEEE International Symposim on High Assurance Systems Engineering, HASE 2000, pp. 203–206. IEEE (2000)

    Google Scholar 

  3. Berghofer, S., Nipkow, T.: Random testing in Isabelle/HOL. In: Cuellar, J., Liu, Z. (eds.) Software Engineering and Formal Methods, SEFM 2004, pp. 230–239. IEEE Computer Society (2004)

    Google Scholar 

  4. Boldo, S.: Deductive formal verification: how to make your floating-point programs behave. Thèse d’habilitation, Université Paris-Sud (October 2014)

    Google Scholar 

  5. Boldo, S., Marché, C.: Formal Verification of Numerical Programs: from C Annotated Programs to Mechanical Proofs. Mathematics in Computer Science 5, 377–393 (2011)

    CrossRef  Google Scholar 

  6. Butler, R.: Formalization of the integral calculus in the PVS theorem prover. Journal of Formalized Reasoning 2(1) (2009)

    Google Scholar 

  7. Cadar, C., Godefroid, P., Khurshid, S., Păsăreanu, C.S., Sen, K., Tillmann, N., Visser, W.: Symbolic execution for software testing in practice: preliminary assessment. In: Proceedings of the 33rd International Conference on Software Engineering, ICSE 2011, pp. 1066–1071. ACM, New York (2011)

    Google Scholar 

  8. Claessen, K., Hughes, J.: QuickCheck: a lightweight tool for random testing of Haskell programs. In: Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming, ICFP 2000, pp. 268–279. ACM, New York (2000)

    Google Scholar 

  9. Crow, J., Owre, S., Rushby, J., Shankar, N., Stringer-Calvert, D.: Evaluating, testing, and animating PVS specifications. Technical report, Computer Science Laboratory, SRI International, Menlo Park, CA (March 2001)

    Google Scholar 

  10. Hagen, G., Butler, R., Maddalon, J.: Stratway: a modular approach to strategic conflict resolution. In: Preceedings of 11th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference, Virgina Beach, VA (September 2011)

    Google Scholar 

  11. Hagen, G.E., Butler, R.W.: Towards a formal semantics of flight plans and trajectories. Technical Memorandum NASA/TM-2014-218862, NASA, Langley Research Center, Hampton VA 23681–2199, USA (December 2014)

    Google Scholar 

  12. Hayhurst, K.J., Veerhusen, D.S., Chilenski, J.J., Rierson, L.K.: A practical tutorial on modified condition/decision coverage. Technical Memorandum NASA/TM-2001-210876, NASA, Langley Research Center, Hampton VA 23681–2199, USA (May 2001)

    Google Scholar 

  13. Lensink, L., Smetsers, S., van Eekelen, M.: Generating verifiable java code from verified PVS specifications. In: Goodloe, A.E., Person, S. (eds.) NFM 2012. LNCS, vol. 7226, pp. 310–325. Springer, Heidelberg (2012)

    CrossRef  Google Scholar 

  14. Leuschel, M., Butler, M.: ProB: a model checker for B. In: Araki, K., Gnesi, S., Mandrioli, D. (eds.) FME 2003. LNCS, vol. 2805, pp. 855–874. Springer, Heidelberg (2003)

    CrossRef  Google Scholar 

  15. Marché, C.: Verification of the functional behavior of a floating-point program: an industrial case study. Science of Computer Programming 96(3), 279–296 (2014)

    MathSciNet  CrossRef  Google Scholar 

  16. Masci, P., Oladimeji, P., Curzon, P., Thimbleby, H.: Tool demo: using PVSio-web to demonstrate software issues in medical user interfaces. In: 4th International Symposium on Foundations of Healthcare Information Engineering and Systems (FHIES2014) (2014)

    Google Scholar 

  17. Meyer, B.: Applying “Design by Contract”. Computer 25(10), 40–51 (1992)

    CrossRef  Google Scholar 

  18. Meyer, B., Fiva, A., Ciupa, I., Leitner, A., Wei, Y., Stapf, E.: Programs that test themselves. Computer 42(9), 46–55 (2009)

    CrossRef  Google Scholar 

  19. Muñoz, C.: Rapid prototyping in PVS. Contractor Report NASA/CR-2003-212418, NASA, Langley Research Center, Hampton VA 23681–2199, USA (May 2003)

    Google Scholar 

  20. Narkawicz, A., Muñoz, C.: State-based implicit coordination and applications. Technical Publication NASA/TP-2011-217067, NASA, Langley Research Center, Hampton VA 23681–2199, USA (March 2011)

    Google Scholar 

  21. Owre, S., Rushby, J., Shankar, N.: PVS: a prototype verification. In: Kapur, D. (ed.) CADE 1992. LNCS, vol. 607, pp. 748–752. Springer, Heidelberg (1992)

    Google Scholar 

  22. Palanque, P., Ladry, J.-F., Navarre, D., Barboni, E.: High-Fidelity prototyping of interactive systems can be formal too. In: Jacko, J.A. (ed.) HCI International 2009, Part I. LNCS, vol. 5610, pp. 667–676. Springer, Heidelberg (2009)

    Google Scholar 

  23. Sen, K., Marinov, D., Agha, G.: CUTE: A concolic unit testing engine for C. In: Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-13, pp. 263–272. ACM, New York (2005)

    Google Scholar 

  24. Shankar, N.: Efficiently executing PVS. Technical report, Project report, ComputerScience Laboratory, SRI International, Menlo Park (1999)

    Google Scholar 

  25. Yang, F., Jacquot, J.-P., Souquières, J.: Jeb: safe simulation of event-b models in javascript. In: 2013 20th Asia-Pacific Software Engineering Conference (APSEC), vol. 1, pp. 571–576 (December 2013)

    Google Scholar 

  26. Yusuke, W., Shigeru, K.: Performance evaluation of a testing framework using quickcheck and hadoop. IPSJ Journal 53(2), 7 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NASA Langley Research Center, Hampton, Virginia, 23681-2199, USA

    Aaron M. Dutle, César A. Muñoz, Anthony J. Narkawicz & Ricky W. Butler

Authors
  1. Aaron M. Dutle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. César A. Muñoz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anthony J. Narkawicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ricky W. Butler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aaron M. Dutle .

Editor information

Editors and Affiliations

  1. Inria Nancy & LORIA, Villers-lès-Nancy, France

    Jasmin Christian Blanchette

  2. CEA LIST Nano-Innov, Saclay, France

    Nikolai Kosmatov

Rights and permissions

Reprints and Permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Dutle, A.M., Muñoz, C.A., Narkawicz, A.J., Butler, R.W. (2015). Software Validation via Model Animation. In: Blanchette, J., Kosmatov, N. (eds) Tests and Proofs. TAP 2015. Lecture Notes in Computer Science(), vol 9154. Springer, Cham. https://doi.org/10.1007/978-3-319-21215-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-21215-9_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-21214-2

  • Online ISBN: 978-3-319-21215-9

  • eBook Packages: Computer ScienceComputer Science (R0)