A First Step in the Design of a Formally Verified Constraint-Based Testing Tool: FocalTest

  • Matthieu Carlier
  • Catherine Dubois
  • Arnaud Gotlieb
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7305)

Abstract

Constraint-based test data generators rely on SMT or constraint solvers to automatically generate test data (e.g., Pex, Sage, Gatel, PathCrawler, Euclide). However, for some test data generation requests corresponding to particular test objectives, these tools may fail to deliver the expected test data because they focus on efficiency rather than soundness and completeness. We adopt an opposite view in the development of FocalTest, a test data generation tool for Focalize programs. The goal of the tool is to generate an MC/DC-compliant set of test data over the precondition of user-defined program properties. The development of such a correct-by-construction test data generator requires 1) to provide a formally verified translation of Focalize programs and properties into constraint systems; 2) to introduce a formally verified constraint solver able to solve those constraint systems. This paper is concerned with the first step only where we formally demonstrate with Coq the soundness of the translation of an intermediate functional language into a constraint system. This objective requires to formally define the operational semantics of the source language that features the manipulation of concrete data types via pattern-matching and function calls, constructions that are mirrored in the constraint language. Although such a semantics-oriented formalization is only a first step of a larger goal which is to provide a formally verified constraint-based testing tool, we argue that it is an important contribution to the building of more robust software testing tools.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ayrault, P., Carlier, M., Delahaye, D., Dubois, C., Doligez, D., Habib, L., Hardin, T., Jaume, M., Morisset, C., Pessaux, F., Rioboo, R., Weis, P.: Trusted software within focal. In: C&ESAR 2008, Computer Electronics Security Applications Rendez-vous, pp. 162–179 (2008)Google Scholar
  2. 2.
    Brucker, A.D., Wolff, B.: Interactive Testing with HOL-TestGen. In: Grieskamp, W., Weise, C. (eds.) FATES 2005. LNCS, vol. 3997, pp. 87–102. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Carlier, M.: Test automatique de propriétés dans un atelier de développement de logiciels sûrs. PhD thesis, CEDRIC Laboratory, Paris, France (2009)Google Scholar
  4. 4.
    Carlier, M., Dubois, C.: Functional Testing in the Focal Environment. In: Beckert, B., Hähnle, R. (eds.) TAP 2008. LNCS, vol. 4966, pp. 84–98. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Carlier, M., Dubois, C., Gotlieb, A.: Constraint reasoning in focaltest. In: Int. Conf. on Soft. and Data Tech. (ICSOFT 2010), Athens (July 2010); Also, CNAM Tech. Report CEDRIC-09-1703, 36 pages (2009)Google Scholar
  6. 6.
    Charreteur, F., Gotlieb, A.: Constraint-based test input generation for java bytecode. In: 21st IEEE Int. Symp. on Softw. Reliability Eng. (ISSRE 2010), San Jose, CA, USA (November 2010)Google Scholar
  7. 7.
    Denmat, T., Gotlieb, A., Ducasse, M.: Improving constraint-based testing with dynamic linear relaxations. In: 18th IEEE Int. Symp. on Soft. Reliability Eng. (ISSRE 2007), Trollhttan, Sweden (November 2007)Google Scholar
  8. 8.
    Coq development team. The Coq proof assistant reference manual, Ver. 8.3 (2009)Google Scholar
  9. 9.
    Fink, G., Bishop, M.: Property-based testing: A new approach to testing for assurance. ACM SIGSOFT Software Engineering Notes 22(4), 74–80 (1997)CrossRefGoogle Scholar
  10. 10.
    Flanagan, C.: Automatic software model checking via constraint logic. Sci. Comput. Program. 50(1-3), 253–270 (2004)MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Gómez-Zamalloa, M., Albert, E., Puebla, G.: Test case generation for object-oriented imperative languages in clp. TPLP 10(4-6), 659–674 (2010)MATHGoogle Scholar
  12. 12.
    Gotlieb, A.: Euclide: A constraint-based testing platform for critical c programs. In: Int. Conf. on Soft. Testing, Valid. and Verif. (ICST 2009), Denver (April 2009)Google Scholar
  13. 13.
    Gotlieb, A., Botella, B., Rueher, M.: A CLP Framework for Computing Structural Test Data. In: Palamidessi, C., Moniz Pereira, L., Lloyd, J.W., Dahl, V., Furbach, U., Kerber, M., Lau, K.-K., Sagiv, Y., Stuckey, P.J. (eds.) CL 2000. LNCS (LNAI), vol. 1861, pp. 399–413. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  14. 14.
    Hayhurst, K., Veerhusen, S., Chilenski, J., Rierson, L.K.: A practical tutorial on modified condition/decision coverage, nasa langley. Technical report (2001)Google Scholar
  15. 15.
    Leroy, X.: Formal verification of a realistic compiler. Communications of the ACM 52(7), 107–115 (2009)CrossRefGoogle Scholar
  16. 16.
    Maranget, L.: Compiling Lazy Pattern Matching. In: Conference on Lisp and Functional Programming. ACM Press (1992)Google Scholar
  17. 17.
    Marre, B., Arnould, A.: Test sequences generation from lustre descriptions: Gatel. In: Proc. of the 15th IEEE Conference on Automated Software Engineering (ASE 2000). IEEE CS Press (September 2000)Google Scholar
  18. 18.
    Podelski, A.: Model Checking as Constraint Solving. In: Palsberg, J. (ed.) SAS 2000. LNCS, vol. 1824, pp. 22–37. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  19. 19.
    Wotawa, F., Nica, M.: On the compilation of programs into their equivalent constraint representation. Informatica 32(4), 359–371 (2008)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Matthieu Carlier
    • 1
  • Catherine Dubois
    • 1
    • 2
  • Arnaud Gotlieb
    • 3
  1. 1.CEDRIC-ENSIIEÉvryFrance
  2. 2.INRIAParisFrance
  3. 3.Certus V&V CenterSIMULA RESEARCH LAB.LysakerNorway

Personalised recommendations