FocalTest: A Constraint Programming Approach for Property-Based Testing

  • Matthieu Carlier
  • Catherine Dubois
  • Arnaud Gotlieb
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 170)

Abstract

Property-based testing is the process of selecting test data from user-specified properties fro testing a program. Current automatic property-based testing techniques adopt direct generate-and-test approaches for this task, consisting in generating first test data and then checking whether a property is satisfied or not. are generated at random and rejected when they do not satisfy selected coverage criteria. In this paper, we propose a technique and tool called FocalTest, which adopt a test-and-generate approach through the usage of constraint reasoning. Our technique utilizes the property to prune the search space during the test data generation process. A particular difficulty is the generation of test data satisfying MC/DC on the precondition of a property, when it contains function calls with pattern matching and high-order functions. Our experimental results show that a non-naive implementation of constraint reasoning on these constructions outperform traditional generation techniques when used to find test data for testing properties.

Keywords

Software testing Automated test data generation MC/DC Constraint reasoning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Fink, G., Bishop, M.: Property-based testing: a new approach to testing for assurance. SIGSOFT Softw. Eng. Notes 22(4), 74–80 (1997)CrossRefGoogle Scholar
  2. 2.
    Claessen, K., Hughes, J.: QuickCheck: a lightweight tool for random testing of Haskell programs. ACM SIGPLAN Notices 35(9), 268–279 (2000)CrossRefGoogle Scholar
  3. 3.
    Koopman, P., Alimarine, A., Tretmans, J., Plasmeijer, R.: Gast: Generic Automated Software Testing. In: Peña, R., Arts, T. (eds.) IFL 2002. LNCS, vol. 2670, pp. 84–100. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  4. 4.
    Fischer, S., Kuchen, H.: Systematic generation of glass-box test cases for functional logic programs. In: Conf. on Princ. and Practice of Declarative Programming (PPDP 2007), pp. 63–74 (2007)Google Scholar
  5. 5.
    Fischer, S., Kuchen, H.: Data-flow testing of declarative programs. In: Proc. of ICFP 2008, pp. 201–212 (2008)Google Scholar
  6. 6.
    Christiansen, J., Fischer, S.: EasyCheck — Test Data for Free. In: Garrigue, J., Hermenegildo, M.V. (eds.) FLOPS 2008. LNCS, vol. 4989, pp. 322–336. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Carlier, M., Dubois, C.: Functional testing in the focal environment. In: Test And Proof, TAP (April 2008)Google Scholar
  8. 8.
    Dubois, C., Hardin, T., Viguié Donzeau-Gouge, V.: Building certified components within focal. In: Fifth Symp. on Trends in Functional Prog., TFP 2004, vol. 5, pp. 33–48 (2006)Google Scholar
  9. 9.
    Gotlieb, A., Botella, B., Rueher, M.: Automatic test data generation using constraint solving techniques. In: Int. Symp. on Soft. Testing and Analysis, ISSTA, pp. 53–62 (1998)Google Scholar
  10. 10.
    Johnsson, T.: Lambda Lifting: Transforming Programs to Recursive Equations. In: Jouannaud, J.-P. (ed.) FPCA 1985. LNCS, vol. 201, pp. 190–203. Springer, Heidelberg (1985)CrossRefGoogle Scholar
  11. 11.
    Carlier, M.: Constraint Reasoning in FocalTest (2009) CEDRIC Technical report, http://cedric.cnam.fr
  12. 12.
    Ayrault, P., Hardin, T., Pessaux, F.: Development life cycle of critical software under focal. In: Int. Workshop on Harnessing Theories for Tool Support in Software, TTSS (2008)Google Scholar
  13. 13.
    Dick, J., Faivre, A.: Automating the Generation and Sequencing of Test Cases from Model-Based Specifications. In: Larsen, P.G., Wing, J.M. (eds.) FME 1993. LNCS, vol. 670, pp. 268–284. Springer, Heidelberg (1993)CrossRefGoogle Scholar
  14. 14.
    Marre, B.: Toward Automatic Test Data Set Selection using Algebraic Specifications and Logic Programming. In: Furukawa, K. (ed.) Int. Conf. on Logic Programming, ICLP, pp. 202–219 (1991)Google Scholar
  15. 15.
    Gotlieb, A.: Euclide: A constraint-based testing platform for critical c programs. In: Int. Conf. on Software Testing, Validation and Verification, ICST (April 2009)Google Scholar
  16. 16.
    Legeard, B., Peureux, F.: Generation of functional test sequences from B formal specifications - presentation and industrial case-study. In: Int. Conf. on Automated Soft. Eng., ASE 2001, pp. 377–381 (2001)Google Scholar
  17. 17.
    Pretschner, A.: Classical search strategies for test case generation with constraint logic programming. In: Formal Approaches to Testing of Soft., FATES, pp. 47–60 (2001)Google Scholar
  18. 18.
    Williams, N., Marre, B., Mouy, P., Roger, M.: PathCrawler: Automatic Generation of Path Tests by Combining Static and Dynamic Analysis. In: Dal Cin, M., Kaâniche, M., Pataricza, A. (eds.) EDCC 2005. LNCS, vol. 3463, pp. 281–292. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  19. 19.
    Godefroid, P., Klarlund, N., Sen, K.: Dart: directed automated random testing. In: ACM Conf. on Prog. Lang. Design and Impl., PLDI, pp. 213–223 (2005)Google Scholar
  20. 20.
    Sen, K., Marinov, D., Agha, G.: Cute: a concolic unit testing engine for c. In: ESEC/FSE-13, pp. 263–272. ACM Press (2005)Google Scholar
  21. 21.
    Uzuncaova, E., Khurshid, S.: Constraint Prioritization for Efficient Analysis of Declarative Models. In: Cuellar, J., Sere, K. (eds.) FM 2008. LNCS, vol. 5014, pp. 310–325. Springer, Heidelberg (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Matthieu Carlier
    • 1
  • Catherine Dubois
    • 2
  • Arnaud Gotlieb
    • 1
  1. 1.INRIA Rennes Bretagne AtlantiqueRennesFrance
  2. 2.CEDRIC-ENSIIEÉvryFrance

Personalised recommendations