International School on Formal Methods for the Design of Computer, Communication and Software Systems

SFM 2014: Formal Methods for Executable Software Models pp 263-309

Part of the Lecture Notes in Computer Science book series (LNCS, volume 8483) | Cite as

Test Case Generation by Symbolic Execution: Basic Concepts, a CLP-Based Instance, and Actor-Based Concurrency

  • Elvira Albert
  • Puri Arenas
  • Miguel Gómez-Zamalloa
  • Jose Miguel Rojas

Abstract

The focus of this tutorial is white-box test case generation (TCG) based on symbolic execution. Symbolic execution consists in executing a program with the contents of its input arguments being symbolic variables rather than concrete values. A symbolic execution tree characterizes the set of execution paths explored during the symbolic execution of a program. Test cases can be then obtained from the successful branches of the tree. The tutorial is split into three parts: (1) The first part overviews the basic techniques used in TCG to ensure termination, handling heap-manipulating programs, achieving compositionality in the process and guiding TCG towards interesting test cases. (2) In the second part, we focus on a particular implementation of the TCG framework in constraint logic programming (CLP). In essense, the imperative object-oriented program under test is automatically transformed into an equivalent executable CLP-translated program. The main advantage of CLP-based TCG is that the standard mechanism of CLP performs symbolic execution for free. The PET system is an open-source software that implements this approach. (3) Finally, in the last part, we study the extension of TCG to actor-based concurrent programs.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agha, G.A.: Actors: A Model of Concurrent Computation in Distributed Systems. MIT Press, Cambridge (1986)Google Scholar
  2. 2.
    Albert, E., Arenas, P., Genaim, S., Puebla, G., Zanardini, D.: Cost Analysis of Java Bytecode. In: De Nicola, R. (ed.) ESOP 2007. LNCS, vol. 4421, pp. 157–172. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Albert, E., Cabañas, I., Flores-Montoya, A., Gómez-Zamalloa, M., Gutiérrez, S.: jPET: an Automatic Test-Case Generator for Java. In: WCRE 2011, pp. 441–442. IEEE Computer Society (2011)Google Scholar
  4. 4.
    Albert, E., de la Banda, M.G., Gómez-Zamalloa, M., Rojas, J.M., Stuckey, P.: A CLP Heap Solver for Test Case Generation. Theory and Practice of Logic Programming 13(4-5), 721–735 (2013)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Albert, E., Gómez-Zamalloa, M., Rojas, J.M., Puebla, G.: Compositional CLP-Based Test Data Generation for Imperative Languages. In: Alpuente, M. (ed.) LOPSTR 2010. LNCS, vol. 6564, pp. 99–116. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Anand, S., Burke, E.K., Chen, T.Y., Clark, J.A., Cohen, M.B., Grieskamp, W., Harman, M., Harrold, M.J., McMinn, P.: An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software 86(8), 1978–2001 (2013)CrossRefGoogle Scholar
  7. 7.
    Anand, S., Godefroid, P., Tillmann, N.: Demand-Driven Compositional Symbolic Execution. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 367–381. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Andrews, G.R.: Concurrent Programming: Principles and Practice. Benjamin/Cummings (1991)Google Scholar
  9. 9.
    Cadar, C., Sen, K.: Symbolic Execution for Software Testing: Three Decades Later. Commun. ACM 56(2), 82–90 (2013)CrossRefGoogle Scholar
  10. 10.
    Cadar, C., Godefroid, P., Khurshid, S., Pǎsǎreanu, C.S., Sen, K., Tillmann, N., Visser, W.: Symbolic Execution for Software Testing in Practice: Preliminary Assessment. In: ICSE 2011, pp. 1066–1071. ACM (2011)Google Scholar
  11. 11.
    Clarke, L.A.: A System to Generate Test Data and Symbolically Execute Programs. IEEE Transactions on Software Engineering 2(3), 215–222 (1976)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Cytron, R., Ferrante, J., Rosen, B.K., Wegman, M.N., Kenneth Zadeck, F.: Efficiently Computing Static Single Assignment Form and the Control Dependence Graph. ACM Trans. Program. Lang. Syst. 13(4), 451–490 (1991)CrossRefGoogle Scholar
  13. 13.
    Degrave, F., Schrijvers, T., Vanhoof, W.: Towards a Framework for Constraint-Based Test Case Generation. In: De Schreye, D. (ed.) LOPSTR 2009. LNCS, vol. 6037, pp. 128–142. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Dovier, A., Formisano, A., Pontelli, E.: A Comparison of CLP(FD) and ASP Solutions to NP-Complete Problems. In: Gabbrielli, M., Gupta, G. (eds.) ICLP 2005. LNCS, vol. 3668, pp. 67–82. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  15. 15.
    Engel, C., Hähnle, R.: Generating Unit Tests from Formal Proofs. In: Gurevich, Y., Meyer, B. (eds.) TAP 2007. LNCS, vol. 4454, pp. 169–188. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  16. 16.
    Esparza, J.: Model checking using net unfoldings. Sci. Comput. Program. 23(2-3), 151–195 (1994)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Ferguson, R., Korel, B.: The Chaining Approach for Software Test Data Generation. ACM Trans. Softw. Eng. Methodol. 5(1), 63–86 (1996)CrossRefGoogle Scholar
  18. 18.
    Flanagan, C., Godefroid, P.: Dynamic partial-order reduction for model checking software. In: POPL, pp. 110–121. ACM (2005)Google Scholar
  19. 19.
    Godefroid, P.: Compositional Dynamic Test Generation. In: POPL 2007, pp. 47–54. ACM (2007)Google Scholar
  20. 20.
    Godefroid, P.: Using partial orders to improve automatic verification methods. In: Larsen, K.G., Skou, A. (eds.) CAV 1991. LNCS, vol. 575, pp. 176–185. Springer, Heidelberg (1992)CrossRefGoogle Scholar
  21. 21.
    Gómez-Zamalloa, M., Albert, E., Puebla, G.: Decompilation of Java Bytecode to Prolog by Partial Evaluation. Information and Software Technology 51(10), 1409–1427 (2009)CrossRefMATHGoogle Scholar
  22. 22.
    Gómez-Zamalloa, M., Albert, E., Puebla, G.: Test Case Generation for Object-Oriented Imperative Languages in CLP. Theory and Practice of Logic Programming, ICLP 2010 Special Issue 10(4-6), 659–674 (2010)MathSciNetMATHGoogle Scholar
  23. 23.
    Gotlieb, A., Botella, B., Rueher, M.: A CLP Framework for Computing Structural Test Data. In: Palamidessi, C., et al. (eds.) CL 2000. LNCS (LNAI), vol. 1861, pp. 399–413. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  24. 24.
    Gupta, N., Mathur, A.P., Soffa, M.L.: Generating Test Data for Branch Coverage. In: ASE 2000, pp. 219–228. IEEE Computer Society (2000)Google Scholar
  25. 25.
    Haller, P., Odersky, M.: Scala actors: Unifying thread-based and event-based programming. Theor. Comput. Sci. 410(2-3), 202–220 (2009)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Howden, W.E.: Symbolic Testing and the DISSECT Symbolic Evaluation System. IEEE Transactions on Software Engineering 3(4), 266–278 (1977)CrossRefMATHGoogle Scholar
  27. 27.
    Jaffar, J., Maher, M.J.: Constraint Logic Programming: A Survey. Journal of Logic Programming 19/20, 503–581 (1994)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    The Java Modelling Language homepage (2013), http://www.eecs.ucf.edu/~leavens/JML//index.shtml
  29. 29.
    Kähkönen, K., Saarikivi, O., Heljanko, K.: Using unfoldings in automated testing of multithreaded programs. In: Goedicke, M., Menzies, T., Saeki, M. (eds.) ASE, pp. 150–159. ACM (2012)Google Scholar
  30. 30.
    Khurshid, S., Păsăreanu, C.S., Visser, W.: Generalized Symbolic Execution for Model Checking and Testing. In: Garavel, H., Hatcliff, J. (eds.) TACAS 2003. LNCS, vol. 2619, pp. 553–568. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  31. 31.
    King, J.C.: Symbolic Execution and Program Testing. Commun. ACM 19(7), 385–394 (1976)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Lauterburg, S., Karmani, R.K., Marinov, D., Agha, G.: Evaluating Ordering Heuristics for Dynamic Partial-Order Reduction Techniques. In: Rosenblum, D.S., Taentzer, G. (eds.) FASE 2010. LNCS, vol. 6013, pp. 308–322. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  33. 33.
    Lloyd, J.W.: Foundations of Logic Programming, 2nd ext. edn. Springer (1987)Google Scholar
  34. 34.
    Marriott, K., Stuckey, P.J.: Programming with Constraints: an Introduction. MIT Press (1998)Google Scholar
  35. 35.
    Meudec, C.: ATGen: Automatic Test Data Generation using Constraint Logic Programming and Symbolic Execution. Softw. Test., Verif. Reliab. 11(2), 81–96 (2001)CrossRefGoogle Scholar
  36. 36.
    Müller, R.A., Lembeck, C., Kuchen, H.: A Symbolic Java Virtual Machine for Test Case Generation. In: IASTEDSE 2004, pp. 365–371. ACTA Press (2004)Google Scholar
  37. 37.
    Pasareanu, C.S., Visser, W., Bushnell, D.H., Geldenhuys, J., Mehlitz, P.C., Rungta, N.: Symbolic pathfinder: integrating symbolic execution with model checking for java bytecode analysis. Autom. Softw. Eng. 20(3), 391–425 (2013)CrossRefGoogle Scholar
  38. 38.
    Pǎsǎreanu, C.S., Visser, W.: A Survey of New Trends in Symbolic Execution for Software Testing and Analysis. Int. J. Softw. Tools Technol. Transf. 11(4), 339–353 (2009)CrossRefGoogle Scholar
  39. 39.
    Rojas, J.M., Gómez-Zamalloa, M.: A Framework for Guided Test Case Generation in Constraint Logic Programming. In: Albert, E. (ed.) LOPSTR 2012. LNCS, vol. 7844, pp. 176–193. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  40. 40.
    Rojas, J.M., Păsăreanu, C.S.: Compositional Symbolic Execution through Program Specialization. In: 8th Workshop on Bytecode Semantics, Verification, Analysis and Transformation, BYTECODE 2013 (March 2013)Google Scholar
  41. 41.
    Sen, K., Agha, G.: Automated Systematic Testing of Open Distributed Programs. In: Baresi, L., Heckel, R. (eds.) FASE 2006. LNCS, vol. 3922, pp. 339–356. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  42. 42.
    Sen, K., Agha, G.: A race-detection and flipping algorithm for automated testing of multi-threaded programs. In: Bin, E., Ziv, A., Ur, S. (eds.) HVC 2006. LNCS, vol. 4383, pp. 166–182. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  43. 43.
    Sen, K., Roşu, G., Agha, G.: Online efficient predictive safety analysis of multithreaded programs. In: Jensen, K., Podelski, A. (eds.) TACAS 2004. LNCS, vol. 2988, pp. 123–138. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  44. 44.
    Sen, K., Roşu, G., Agha, G.: Detecting errors in multithreaded programs by generalized predictive analysis of executions. In: Steffen, M., Zavattaro, G. (eds.) FMOODS 2005. LNCS, vol. 3535, pp. 211–226. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  45. 45.
    Tasharofi, S., Karmani, R.K., Lauterburg, S., Legay, A., Marinov, D., Agha, G.: TransDPOR: A Novel Dynamic Partial-Order Reduction Technique for Testing Actor Programs. In: Giese, H., Rosu, G. (eds.) FORTE 2012 and FMOODS 2012. LNCS, vol. 7273, pp. 219–234. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  46. 46.
    Tillmann, N., de Halleux, J.: Pex–White Box Test Generation for.NET. In: Beckert, B., Hähnle, R. (eds.) TAP 2008. LNCS, vol. 4966, pp. 134–153. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  47. 47.
    Triska, M.: The Finite Domain Constraint Solver of SWI-Prolog. In: Schrijvers, T., Thiemann, P. (eds.) FLOPS 2012. LNCS, vol. 7294, pp. 307–316. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  48. 48.
    Wielemaker, J., Schrijvers, T., Triska, M., Lager, T.: SWI-prolog. Theory and Practice of Logic Programming 12(1-2), 67–96 (2012)MathSciNetCrossRefMATHGoogle Scholar
  49. 49.
    Zhu, H., Hall, P.A.V., May, J.H.R.: Software Unit Test Coverage and Adequacy. ACM Comput. Surv. 29(4), 366–427 (1997)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Elvira Albert
    • 1
  • Puri Arenas
    • 1
  • Miguel Gómez-Zamalloa
    • 1
  • Jose Miguel Rojas
    • 2
  1. 1.DSICComplutense University of Madrid (UCM)Spain
  2. 2.Department of Computer ScienceUniversity of SheffieldUK

Personalised recommendations