Skip to main content
SpringerLink
Log in

Property-Based Test Case Generators for Free

  • Conference paper
  • First Online:
Tests and Proofs (TAP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11823))

Included in the following conference series:

Abstract

Property-Based Testing requires the programmer to write suitable generators, i.e., programs that generate (possibly in a random way) input values for which the program under test should be run. However, the process of writing generators is quite a costly, error-prone activity. In the context of Property-Based Testing of Erlang programs, we propose an approach to relieve the programmer from the task of writing generators. Our approach allows the automatic, efficient generation of input test values that satisfy a given specification. In particular, we have considered the case when the input values are data structures satisfying complex constraints. That generation is performed via the symbolic execution of the specification using constraint logic programming.

This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Ciencia, Innovación y Universidades/AEI, grant TIN2016-76843-C4-1-R and by the Generalitat Valenciana, grant PROMETEO-II/2015/013 (SmartLogic).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    More detailed information about types and contract specifications can be found at http://erlang.org/doc/reference_manual/typespec.html

  2. 2.

    https://fmlab.unich.it/testing/

References

  1. Amaral, C., Florido, M., Santos Costa, V.: PrologCheck – property-based testing in Prolog. In: Codish, M., Sumii, E. (eds.) FLOPS 2014. LNCS, vol. 8475, pp. 1–17. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07151-0_1

    Chapter  Google Scholar 

  2. Arts, Th., Hughes, J., Johansson, J., Wiger, U.T.: Testing telecoms software with Quviq QuickCheck. In: Feeley, M., Trinder, Ph.W., (eds.) Proceedings of the 2006 ACM SIGPLAN Workshop on Erlang, Portland, Oregon, USA, 16 September 2006, pp. 2–10. ACM (2006)

    Google Scholar 

  3. Bouquet, F., Dadeau, F., Legeard, B.: Automated boundary test generation from JML specifications. In: Misra, J., Nipkow, T., Sekerinski, E. (eds.) FM 2006. LNCS, vol. 4085, pp. 428–443. Springer, Heidelberg (2006). https://doi.org/10.1007/11813040_29

    Chapter  Google Scholar 

  4. Boyapati, C., Khurshid, S., Marinov, D.: Korat: automated testing based on Java predicates. In: Proceedings of the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 123–133. ACM, New York (2002)

    Article  Google Scholar 

  5. Carlier, M., Dubois, C., Gotlieb, A.: A first step in the design of a formally verified constraint-based testing tool: FocalTest. In: Brucker, A.D., Julliand, J. (eds.) TAP 2012. LNCS, vol. 7305, pp. 35–50. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30473-6_5

    Chapter  Google Scholar 

  6. Carlier, M., Dubois, C., Gotlieb, A.: FocalTest: a constraint programming approach for property-based testing. In: Cordeiro, J., Virvou, M., Shishkov, B. (eds.) ICSOFT 2010. CCIS, vol. 170, pp. 140–155. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-29578-2_9

    Chapter  Google Scholar 

  7. Cheney, J., Momigliano, A.: \(\alpha \)Check: a mechanized metatheory model checker. Theory Pract. Log. Program. 17(3), 311–352 (2017)

    Article  MathSciNet  Google Scholar 

  8. Claessen, K., Hughes, J.: QuickCheck: a lightweight tool for random testing of Haskell programs. In: Odersky, M., Wadler, Ph. (eds.) Proceedings of the 5th ACM SIGPLAN International Conference on Functional Programming (ICFP 2000), Montreal, Canada, 18–21 September 2000, pp. 268–279. ACM (2000)

    Google Scholar 

  9. Coppit, D., Le, W., Sullivan, K.J., Khurshid, S., Yang, J.: Software assurance by bounded exhaustive testing. IEEE Trans. Softw. Eng. 31(4), 328–339 (2005)

    Article  Google Scholar 

  10. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 3rd edn. MIT Press, Cambridge (2009)

    MATH  Google Scholar 

  11. Dick, J., Faivre, A.: Automating the generation and sequencing of test cases from model-based specifications. In: Woodcock, J.C.P., Larsen, P.G. (eds.) FME 1993. LNCS, vol. 670, pp. 268–284. Springer, Heidelberg (1993). https://doi.org/10.1007/BFb0024651

    Chapter  Google Scholar 

  12. Carlsson, R., et al.: Core Erlang 1.0.3. language specification. Technical Report, Department of Information Technology, Uppsala University, Uppsala, Sweden (2004). https://www.it.uu.se/research/group/hipe/cerl/doc/core_erlang-1.0.3.pdf

  13. Fioravanti, F., Proietti, M., Senni, V.: Efficient generation of test data structures using constraint logic programming and program transformation. J. Log. Comput. 25(6), 1263–1283 (2015)

    Article  MathSciNet  Google Scholar 

  14. Gligoric, M., Gvero, T., Jagannath, V., Khurshid, S., Kuncak, V., Marinov, D.: Test generation through programming in UDITA. In: Kramer, J., Bishop, J., Devanbu, P.T., Uchitel, S. (eds.) Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering, Cape Town, South Africa, 2–8 May 2010, pp. 225–234. ACM (2010)

    Google Scholar 

  15. Godefroid, P., Klarlund, N., Sen, K.: DART: directed automated random testing. In: Sarkar, V., Hall, M.W. (eds.) Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation, Chicago, IL, USA, 12–15 June 2005, pp. 213–223. ACM (2005)

    Google Scholar 

  16. Gómez-Zamalloa, M., Albert, E., Puebla, G.: Test case generation for object-oriented imperative languages in CLP. Theory Pract. Log. Program. 10(4–6), 659–674 (2010)

    Article  MathSciNet  Google Scholar 

  17. Gotlieb, A.: Euclide: a constraint-based testing framework for critical C programs. In: 2nd International Conference on Software Testing, Verification and Validation, ICST 2009, Denver, Colorado, USA, 1–4 April 2009, pp. 151–160. IEEE Computer Society (2009)

    Google Scholar 

  18. Gotlieb, A., Botella, B., Rueher, M.: A CLP framework for computing structural test data. In: Lloyd, J. (ed.) CL 2000. LNCS (LNAI), vol. 1861, pp. 399–413. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-44957-4_27

    Chapter  Google Scholar 

  19. Hewitt, C., Bishop, P., Steiger, R.: A universal modular ACTOR formalism for artificial intelligence. In: Proceedings of the 3rd International Joint Conference on Artificial Intelligence, IJCAI 1973, San Francisco, CA, USA, pp. 235–245. Morgan Kaufmann Publishers Inc. (1973)

    Google Scholar 

  20. Jaffar, J., Maher, M.: Constraint logic programming: a survey. J. Log. Program. 19(20), 503–581 (1994)

    Article  MathSciNet  Google Scholar 

  21. Jones, N.D., Gomard, C.K., Sestoft, P.: Partial Evaluation and Automatic Program Generation. Prentice Hall, Upper Saddle River (1993)

    MATH  Google Scholar 

  22. junit-quickcheck: Property-based testing, JUnit-style. https://github.com/pholser/junit-quickcheck

  23. Kowalski, R.A.: Logic for Problem Solving. North Holland, Amsterdam (1979)

    MATH  Google Scholar 

  24. Lampropoulos, L., Paraskevopoulou, Z., Pierce, B.C.: Generating good generators for inductive relations. In: Proceedings of the ACM on Programming Languages, vol. 2, pp. 45:1–45:30 (2017)

    Article  Google Scholar 

  25. Marre, B.: Toward automatic test data set selection using algebraic specifications and logic programming. In: Furukawa, K. (eds.) Logic Programming, Proceedings of the 8th International Conference, Paris, France, 24–28 June 1991, pp. 202–219. MIT Press (1991)

    Google Scholar 

  26. Marre, B., Arnould, A.: Test sequences generation from LUSTRE descriptions: GATeL. In: Proceedings of the 15th IEEE International Conference on Automated Software Engineering, ASE 2000, Grenoble, France, 11–15 September 2000, p. 229. IEEE Computer Society (2000)

    Google Scholar 

  27. Papadakis, M., Sagonas, K.: A PropEr integration of types and function specifications with property-based testing. In: Rikitake, K., Stenman, E. (eds.) Proceedings of the 10th ACM SIGPLAN Workshop on Erlang, Tokyo, Japan, 23 September 2011, pp. 39–50. ACM (2011)

    Google Scholar 

  28. Paraskevopoulou, Z., Hriţcu, C., Dénès, M., Lampropoulos, L., Pierce, B.C.: Foundational property-based testing. In: Urban, C., Zhang, X. (eds.) ITP 2015. LNCS, vol. 9236, pp. 325–343. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22102-1_22

    Chapter  Google Scholar 

  29. Pretschner, A., Lötzbeyer, H.: Model based testing with constraint logic programming: first results and challenges. In: Proceedings of the 2nd ICSE Workshop on Automated Program Analysis, Testing and Verification (WAPATV), pp. 1–9 (2001)

    Google Scholar 

  30. PropEr: Property-Based Testing for Erlang. http://proper.softlab.ntua.gr/

  31. Runciman, C., Naylor, M., Lindblad, F.: SmallCheck and lazy SmallCheck: automatic exhaustive testing for small values. In: Gill, A. (ed.) Proceedings of the 1st ACM SIGPLAN Symposium on Haskell, Haskell 2008, Victoria, BC, Canada, 25 September 2008, pp. 37–48. ACM (2008)

    Google Scholar 

  32. Rushby, J.: Automated test generation and verified software. In: Meyer, B., Woodcock, J. (eds.) VSTTE 2005. LNCS, vol. 4171, pp. 161–172. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69149-5_18

    Chapter  Google Scholar 

  33. ScalaCheck: Property-Based Testing for Scala. http://www.scalacheck.org/

  34. Sen, K., Marinov, D., Agha, G.: CUTE: a concolic unit testing engine for C. In: Wermelinger, M., Gall, H.C. (eds.) Proceedings of the 10th European Software Engineering Conference held jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2005, Lisbon, Portugal, 5–9 September 2005, pp. 263–272. ACM (2005)

    Google Scholar 

  35. The SWI Prolog Logic Programming System. http://www.swi-prolog.org/

  36. Virding, R., Wikström, C., Williams, M.: Concurrent programming in Erlang, 2nd edn. In: Armstrong, J. (ed.) Prentice Hall International Ltd, Hertfordshire (1996)

    Google Scholar 

  37. Williams, N., Marre, B., Mouy, P., Roger, M.: PathCrawler: automatic generation of path tests by combining static and dynamic analysis. In: Dal Cin, M., Kaâniche, M., Pataricza, A. (eds.) EDCC 2005. LNCS, vol. 3463, pp. 281–292. Springer, Heidelberg (2005). https://doi.org/10.1007/11408901_21

    Chapter  Google Scholar 

  38. Xu, G., Yang, Z.: JMLAutoTest: a novel automated testing framework based on JML and JUnit. In: Petrenko, A., Ulrich, A. (eds.) FATES 2003. LNCS, vol. 2931, pp. 70–85. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24617-6_6

    Chapter  Google Scholar 

  39. Yatoh, K., Sakamoto, K., Ishikawa, F., Honiden, S.: ArbitCheck: a highly automated property-based testing tool for Java. In: Proceedings of the 7th IEEE International Conference on Software Testing, Verification, and Validation Workshops, ICSTW 2014, Cleveland, Ohio, USA, 31 March–April 4 2014, pp. 405–412. IEEE Computer Society (2014)

    Google Scholar 

Download references

Acknowledgements

We would like to thank the anonymous reviewers for their very helpful and constructive comments.

E. De Angelis, F. Fioravanti, A. Pettorossi, and M. Proietti are members of the INdAM Research group GNCS. E. De Angelis, F. Fioravanti, and A. Pettorossi are research associates at CNR-IASI, Rome, Italy.

A. Palacios was partially supported by the EU (FEDER) and the Spanish Ayudas para contratos predoctorales para la formación de doctores and Ayudas a la movilidad predoctoral para la realización de estancias breves en centros de I+D (MICINN) under FPI grants BES-2014-069749 and EEBB-I-17-12101.

Author information

Authors and Affiliations

  1. DEC, University “G. d’Annunzio” of Chieti-Pescara, Pescara, Italy

    Emanuele De Angelis & Fabio Fioravanti

  2. MiST, DSIC, Polytechnic University of Valencia, Valencia, Spain

    Adrián Palacios

  3. University of Rome Tor Vergata, Rome, Italy

    Alberto Pettorossi

  4. CNR-IASI, Rome, Italy

    Maurizio Proietti

Authors
  1. Emanuele De Angelis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabio Fioravanti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adrián Palacios
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alberto Pettorossi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maurizio Proietti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Emanuele De Angelis , Fabio Fioravanti , Adrián Palacios , Alberto Pettorossi or Maurizio Proietti .

Editor information

Editors and Affiliations

  1. Ludwig-Maximilians-Universität München, Munich, Germany

    Dirk Beyer

  2. University of Paris-Sud, Orsay, France

    Chantal Keller

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

De Angelis, E., Fioravanti, F., Palacios, A., Pettorossi, A., Proietti, M. (2019). Property-Based Test Case Generators for Free. In: Beyer, D., Keller, C. (eds) Tests and Proofs. TAP 2019. Lecture Notes in Computer Science(), vol 11823. Springer, Cham. https://doi.org/10.1007/978-3-030-31157-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-31157-5_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31156-8

  • Online ISBN: 978-3-030-31157-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation