Skip to main content
SpringerLink
Log in

Symbolic PathFinder: integrating symbolic execution with model checking for Java bytecode analysis

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Symbolic PathFinder (SPF) is a software analysis tool that combines symbolic execution with model checking for automated test case generation and error detection in Java bytecode programs. In SPF, programs are executed on symbolic inputs representing multiple concrete inputs and the values of program variables are represented by expressions over those symbolic inputs. Constraints over these expressions are generated from the analysis of different paths through the program. The constraints are solved with off-the-shelf solvers to determine path feasibility and to generate test inputs. Model checking is used to explore different symbolic program executions, to systematically handle aliasing in the input data structures, and to analyze the multithreading present in the code. SPF incorporates techniques for handling input data structures, strings, and native calls to external libraries, as well as for solving complex mathematical constraints. We describe the tool and its application at NASA, in academia, and in industry.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. Assuming here our solver cannot deal with non-linear integer arithmetic

References

  • Anand, S., Păsăreanu, C.S., Visser, W.: JPF–SE: a symbolic execution extension to Java PathFinder. In: Proc. 13th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). LNCS, vol. 4424, pp. 134–138. Springer, Berlin (2007)

    Chapter  Google Scholar 

  • Anand, S., Păsăreanu, C.S., Visser, W.: Symbolic execution with abstraction. Int. J. Softw. Tools Technol. Transf. 11(1), 53–67 (2009)

    Article  Google Scholar 

  • Balasubramanian, D., Păsăreanu, C.S., Whalen, M.W., Karsai, G., Lowry, M.R.: Polyglot: modeling and analysis for multiple statechart formalisms. In: Proc. 2011 International Symposium on Software Testing and Analysis (ISSTA), pp. 45–55 (2011)

    Chapter  Google Scholar 

  • Balasubramanian, D., Păsăreanu, C.S., Biatek, J., Pressburger, T., Karsai, G., Lowry, M.R., Whalen, M.W.: Integrating statechart components in Polyglot. In: Proc. NASA Formal Methods 2012, pp. 267–272 (2012)

    Chapter  Google Scholar 

  • Barrett, C., Tinelli, C.: CVC3. In: Proc. 19th Intl. Conf. on Computer Aided Verification (CAV). LNCS, vol. 4590, pp. 298–302. Springer, Berlin (2007)

    Chapter  Google Scholar 

  • Bjørner, N., Tillmann, N., Voronkov, A.: Path feasibility analysis for string-manipulating programs. In: Proc. 15th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). LNCS, vol. 5505, pp. 307–321. Springer, Berlin (2009)

    Chapter  Google Scholar 

  • Borges, M., D’Amorim, M., Anand, S., Bushnell, D., Păsăreanu, C.S.: Symbolic execution with interval solving and meta-heuristic search. In: Proc. 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation (ICST), pp. 111–120 (2012)

    Chapter  Google Scholar 

  • Bush, W., Pincus, J., Sielaff, D.: A static analyzer for finding dynamic programming errors. Softw. Pract. Exp. 30(7), 775–802 (2000)

    Article  MATH  Google Scholar 

  • Bushnell, D.: Continuity analysis for floating point software. In: Numerical Software Verification Workshop, NSV-2011 (2011)

    Google Scholar 

  • Cadar, C., Dunbar, D., Engler, D.: KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI’08, pp. 209–224. USENIX Association, Berkeley (2008a)

    Google Scholar 

  • Cadar, C., Ganesh, V., Pawlowski, P., Dill, D., Engler, D.: EXE: automatically generating inputs of death. ACM Trans. Inf. Syst. Secur. 12(2), 1–38 (2008b)

    Article  Google Scholar 

  • Cadar, C., Godefroid, P., Khurshid, S., Păsăreanu, C.S., Sen, K., Tillmann, N., Visser, W.: Symbolic execution for software testing in practice: preliminary assessment. In: Proc. 33rd International Conference on Software Engineering (ICSE), pp. 1066–1071 (2011)

    Google Scholar 

  • CERT/CC: Cert Advisory: multiple vulnerabilities in WU-FTPD. Tech. Rep. CA–2001–33 (2001)

  • Choco: Java constraint solver (2012). http://choco.emn.fr

  • Cristensen, A.S., Møller, A., Schwartzbach, M.I.: Precise analysis of string expressions. In: Proc. 10th Intl. Symposium on Static Analysis (SAS). LNCS, vol. 2694, pp. 1–18. Springer, Berlin (2003)

    Google Scholar 

  • Clarke, L.A.: A system to generate test data and symbolically execute programs. IEEE Trans. Softw. Eng. 2, 215–222 (1976). doi:10.1109/TSE.1976.233817, http://dl.acm.org/citation.cfm?id=1313320.1313532

    Article  Google Scholar 

  • Coen-Porisini, A., Denaro, G., Ghezzi, C., Pezzé, M.: Using symbolic execution for verifying safety-critical systems. In: Proc. ESEC/SIGSOFT FSE, p. 151. ACM Press, New York (2001)

    Google Scholar 

  • Collingbourne, P., Cadar, C., Kelly, P.H.: Symbolic crosschecking of floating-point and simd code. In: Proc. of the 6th Conference on Computer Systems, EuroSys’11, pp. 315–328. ACM Press, New York (2011). doi:10.1145/1966445.1966475

    Google Scholar 

  • d’Amorim, M., Pacheco, C., Xie, T., Marinov, D., Ernst, M.D.: An empirical comparison of automated generation and classification techniques for object-oriented unit testing. In: Proc. 21st IEEE/ACM Intl. Conf. on Automated Software Engineering (ASE), pp. 59–68. IEEE Computer Society, Washington (2006)

    Chapter  Google Scholar 

  • de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Proc. 14th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). LNCS, vol. 4963, pp. 337–340. Springer, Berlin (2008)

    Chapter  Google Scholar 

  • Deng, X., Lee, J., Robby: Bogor/Kiasan: a k-bounded symbolic execution for checking strong heap properties of open systems. In: Proc. 21st IEEE/ACM Intl. Conf. on Automated Software Engineering (ASE), pp. 157–166. IEEE Computer Society, Washington (2006)

    Chapter  Google Scholar 

  • Deng, X., Hatcliff, J., Robby: Kiasan/KUnit: automatic test case generation and analysis feedback for open object-oriented systems. In: 3rd International Workshop TAIC PART—Mutation Analysis, pp. 3–12 (2007)

    Google Scholar 

  • Deng, X., Lee, J., Robby: Efficient and formal generalized symbolic execution. Autom. Softw. Eng. 19, 233–301 (2012). doi:10.1007/s10515-011-0089-9

    Article  Google Scholar 

  • Detlefs, D.L., Leino, K.R.M., Nelson, G., Saxe, J.B.: Extended static checking. SRC Research Report 159. COMPAQ Systems Research Center (1998)

  • Filieri, A., Păsăreanu, C.S., Visser, W.: Reliability analysis in symbolic PathFinder. In: Proc. 35th International Conference on Software Engineering (ICSE) (2013)

    Google Scholar 

  • Geldenhuys, J., Dwyer, M.B., Visser, W.: Probabilistic symbolic execution. In: Proc. International Symposium on Software Testing and Analysis (ISSTA), pp. 166–176 (2012)

    Google Scholar 

  • Godefroid, P., Klarlund, N., Sen, K.: Dart: directed automated random testing. In: Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pp. 213–223. ACM Press, New York (2005)

    Chapter  Google Scholar 

  • Godefroid, P., de Halleux, P., Nori, A., Rajamani, S., Schulte, W., Tillmann, N., Levin, M.: Automating software testing using program analysis. IEEE Softw. 25(5), 30–37 (2008)

    Article  Google Scholar 

  • Hooimeijer, P., Weimer, W.: A decision procedure for subset constraints over regular languages. In: Proc. 2009 ACM SIGPLAN Conf. on Programming Language Design and Implementation (PLDI), pp. 188–198. ACM Press, New York (2009)

    Chapter  Google Scholar 

  • Hooimeijer, P., Weimer, W.: Solving string constraints lazily. In: Pecheur, C., Andrews, J. (eds.) Proc. 25th IEEE/ACM Intl. Conf. on Automated Software Engineering (ASE), pp. 377–386. ACM Press, New York (2010)

    Chapter  Google Scholar 

  • Hooimeijer, P., Molnar, D., Saxena, P., Veanes, M.: Modeling imperative string operations with transducers. Tech. Rep. MSR–TR–2010–96, Microsoft (2010)

  • IASolver: IASolver page (2010). http://www.cs.brandeis.edu/~tim/Applets/IAsolver.html

  • Jayaraman, K., Harvison, D., Ganesh, V., Kiezun, A.: JFuzz: a concolic whitebox fuzzer for Java. In: NASA Formal Methods Symposium, NASA Technical Memorandum (2009)

    Google Scholar 

  • JPF: JPF project (2012). http://babelfish.arc.nasa.gov/trac/jpf

  • Khurshid, S., Păsăreanu, C.S., Visser, W.: Generalized symbolic execution for model checking and testing. In: Proc. 9th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), pp. 553–568 (2003)

    Chapter  Google Scholar 

  • Kieżun, A.: Effective software testing with a string-constraint solver. PhD thesis, Massachusetts Institute of Technology, USA (2009)

  • Kieżun, A., Ganesh, V., Guo, P.J., Hooimeijer, P., Ernst, M.D.: HAMPI: a solver for string constraints. In: Rothermel, G., Dillon, L.K. (eds.) Proc. 2009 International Symposium on Software Testing and Analysis (ISSTA), pp. 105–116. ACM Press, New York (2009)

    Google Scholar 

  • King, J.C.: Symbolic execution and program testing. Commun. ACM 19, 385–394 (1976). doi:10.1145/360248.360252

    Article  MATH  Google Scholar 

  • LattE: LattE Integrale. UC Davis, Mathematics (2012). http://www.math.ucdavis.edu/~latte

  • Li, G., Ghosh, I., Rajan, S.P.: Klover: a symbolic execution and automatic test generation tool for C++ programs. In: Proceedings of the 23rd International Conference on Computer Aided Verification, CAV’11, pp. 609–615. Springer, Berlin (2011). http://dl.acm.org/citation.cfm?id=2032305.2032354

    Chapter  Google Scholar 

  • Li, X., Shannon, D., Ghosh, I., Ogawa, M., Rajan, S., Khurshid, S.: Context-sensitive relevancy analysis for efficient symbolic execution. In: Asian Symposium on Programming Languages and Systems (APLAS) (2008)

    Google Scholar 

  • Loera, J.A.D., Dutra, B., Köppe, M., Moreinis, S., Pinto, G., Wu, J.: Software for exact integration of polynomials over polyhedra (2011). arXiv:1108.0117v2 [math.MG]

  • Mirzaei, N., Malek, S., Păsăreanu, C., Esfahani, N., Mahmood, R.: Testing Android apps through symbolic execution. In: JPF Workshop (2012)

    Google Scholar 

  • Păsăreanu, C.S., Visser, W.: Verification of Java programs using symbolic execution and invariant generation. In: Proc. of 11th International SPIN Workshop (SPIN). LNCS, vol. 2989, pp. 164–181. Springer, Berlin (2004)

    Google Scholar 

  • Păsăreanu, C.S., Mehlitz, P.C., Bushnell, D.H., Gundy-Burlet, K., Lowry, M., Person, S., Pape, M.: Combining unit-level symbolic execution and system-level concrete execution for testing NASA software. In: Proc. 2008 International Symposium on Software Testing and Analysis (ISSTA), pp. 15–26 (2008)

    Chapter  Google Scholar 

  • Păsăreanu, C.S., Rungta, N., Visser, W.: Symbolic execution with mixed concrete-symbolic solving. In: Proceedings of the 2011 International Symposium on Software Testing and Analysis (ISSTA), pp. 34–44. ACM Press, New York (2011). doi:10.1145/2001420.2001425

    Chapter  Google Scholar 

  • Person, S., Yang, G., Rungta, N., Khurshid, S.: Directed incremental symbolic execution. In: Proc. 2011 ACM SIGPLAN Conf. on Programming Language Design and Implementation (PLDI), pp. 504–515 (2011)

    Google Scholar 

  • Phan, Q.-S., Malacaria, P., Tkachuk, O., Păsăreanu, C.S.: Symbolic quantitative information flow. Softw. Eng. Notes 37, 1–5 (2012)

    Article  Google Scholar 

  • Rajan, S., Tkachuk, O., Prasad, M., Ghosh, I., Goel, N., Uehara, T.: WEAVE: WEb Applications Validation Environment. In: Proc. 31st International Conference on Software Engineering (ICSE Companion) (2009)

    Google Scholar 

  • Redelinghuys, G.: Symbolic string execution. Master’s thesis, Stellenbosch University (2012)

  • Redis: Redis NoSQL database (2012). http://redis.io

  • Rossi, F., van Beek, P., Walsh, T.: Handbook of Constraint Programming. Elsevier, Amsterdam (2006)

    MATH  Google Scholar 

  • Rungta, N., Mercer, E.G., Visser, W.: Efficient testing of concurrent programs with abstraction-guided symbolic execution. In: Proc. of 16th International SPIN Workshop (SPIN), pp. 174–191 (2009)

    Google Scholar 

  • Sanfilippo, S., Noordhuis, P.: Redis: the Definitive Guide. O’Reilly Media, Sebastopol (2012)

    Google Scholar 

  • Santelices, R., Harrold, M.J.: Exploiting program dependencies for scalable multiple-path symbolic execution. In: Proc. 2010 International Symposium on Software Testing and Analysis (ISSTA), pp. 195–206 (2010)

    Google Scholar 

  • Sasnauskas, R., Dustmann, O.S., Kaminski, B.L., Wehrle, K., Weise, C., Kowalewski, S.: Scalable symbolic execution of distributed systems. In: Proceedings of the 2011 31st International Conference on Distributed Computing Systems, ICDCS ’11, pp. 333–342. IEEE Computer Society, Washington (2011). doi:10.1109/ICDCS.2011.28

    Chapter  Google Scholar 

  • Saxena, P., Akhawe, D., Hanna, S., Mao, F., McCamant, S., Song, D.: A symbolic execution framework for JavaScript. In: Proc. 31st IEEE Symposium on Security and Privacy, pp. 513–528. IEEE Computer Society, Washington (2010)

    Google Scholar 

  • Sen, K., Agha, G.: CUTE and jCUTE: concolic unit testing and explicit path model-checking tools. In: Proc. 18th International Conference on Computer Aided Verification (CAV), pp. 419–423 (2006)

    Chapter  Google Scholar 

  • Sen, K., Agha, G.: A race-detection and flipping algorithm for automated testing of multithreaded programs. In: Proc. Haifa Verification Conference (HVC). LNCS, vol. 4383, pp. 166–182. Springer, Berlin (2007)

    Google Scholar 

  • Shannon, D., Hajra, S., Lee, A., Zhan, D., Khurshid, S.: Abstracting symbolic execution with string analysis. In: Proc. Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 13–22. IEEE Computer Society, Washington (2007)

    Google Scholar 

  • Shannon, D., Ghosh, I., Rajan, S.P., Khurshid, S.: Efficient symbolic execution of strings for validating web applications. In: Proc. 2nd Intl. Workshop on Defects in Large Software Systems, pp. 22–26. ACM Press, New York (2009)

    Google Scholar 

  • Siegel, S., Zirkel, T.: Tass: the toolkit for accurate scientific software. Math. Comput. Sci. 5, 395–426 (2011). doi:10.1007/s11786-011-0100-7

    Article  MATH  Google Scholar 

  • Siegel, S., Mironova, A., Avrunin, G., Clarke, L.: Using model checking with symbolic execution to verify parallel numerical programs. In: Proc. 2006 International Symposium on Software Testing and Analysis (ISSTA), pp. 157–168. ACM Press, New York (2006)

    Chapter  Google Scholar 

  • Souza, M., Borges, M., d’Amorim, M., Păsăreanu, C.S.: CORAL: solving complex constraints for Symbolic PathFinder. In: NASA Formal Methods, pp. 359–374 (2011)

    Chapter  Google Scholar 

  • SPF: Symbolic Pathfinder (jpf-symbc) (2012). http://babelfish.arc.nasa.gov/trac/jpf

  • Staats, M., Păsăreanu, C.: Parallel symbolic execution for structural test generation. In: Proc. 2010 International Symposium on Software Testing and Analysis (ISSTA), pp. 183–194. ACM Press, New York (2010). doi:10.1007/s10515-013-0122-2

    Google Scholar 

  • Tillmann, N., de Halleux, J.: Pex–white box test generation for .NET. In: Beckert, B., Hähnle, R. (eds.) Proc. 2nd Intl. Conf. on Tests and Proofs. LNCS, vol. 4966, pp. 134–153. Springer, Berlin (2008)

    Chapter  Google Scholar 

  • Tomb, A., Brat, G., Visser, W.: Variably interprocedural program analysis for run-time error detection. In: Proc. 2007 International Symposium on Software Testing and Analysis (ISSTA), pp. 97–107. ACM Press, New York (2007)

    Chapter  Google Scholar 

  • Veanes, M., de Halleux, P., Tillmann, N.: Rex: symbolic regular expression explorer. In: Proc. 3rd Intl. Conf. on Software Testing, Verification and Validation, pp. 498–507. IEEE Computer Society, Washington (2010)

    Google Scholar 

  • Visser, W., Havelund, K., Brat, G.P., Park, S., Lerda, F.: Model checking programs. Autom. Softw. Eng. 10(2), 203–232 (2003)

    Article  Google Scholar 

  • Visser, W., Păsăreanu, C.S., Pelánek, R.: Test input generation for Java containers using state matching. In: Proc. 2006 International Symposium on Software Testing and Analysis (ISSTA), pp. 37–48 (2006)

    Chapter  Google Scholar 

  • Visser, W., Geldenhuys, J., Dwyer, M.B.: Green: reducing, reusing and recycling constraints in program analysis. In: International Symposium on the Foundations of Software Engineering (FSE), Cary, North Carolina, USA (2012)

    Google Scholar 

  • Xie, T., Marinov, D., Schulte, W., Notkin, D.: Symstra: a framework for generating object-oriented unit tests using symbolic execution. In: Proceedings of the 11th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), pp. 365–381. Springer, Berlin (2005)

    Chapter  Google Scholar 

  • Yang, G., Păsăreanu, C.S., Khurshid, S.: Memoized symbolic execution. In: Proc. International Symposium on Software Testing and Analysis (ISSTA), pp. 144–154 (2012)

    Google Scholar 

  • Yices: Yices SMT Solver (2012). http://yices.csl.sri.com/

  • Yu, F., Bultan, T., Cova, M., Ibarra, O.H.: Symbolic string verification: an automata-based approach. In: Proc. 15th Intl. SPIN Workshop on Model Checking Software. LNCS, vol. 5156, pp. 306–324. Springer, Berlin (2008)

    Chapter  Google Scholar 

  • Yu, F., Alkhalaf, M., Bultan, T.: Stranger: an automata-based string analysis tool for PHP. In: Proc. 16th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). LNCS, vol. 6015, pp. 154–157. Springer, Berlin (2010)

    Chapter  Google Scholar 

  • Zhang, P., Elbaum, S.G., Dwyer, M.B.: Automatic generation of load tests. In: Alexander, P., Păsăreanu, C.S., Hosking, J.G. (eds.) Proc. 26th IEEE/ACM Intl. Conf. on Automated Software Engineering, pp. 43–52. IEEE Press, New York (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NASA Ames Research Center, Moffett Field, CA, 94035, USA

    Corina S. Păsăreanu, David Bushnell, Peter Mehlitz & Neha Rungta

  2. University of Stellenbosch, Stellenbosch, South Africa

    Willem Visser & Jaco Geldenhuys

Authors
  1. Corina S. Păsăreanu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Willem Visser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Bushnell
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jaco Geldenhuys
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter Mehlitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Neha Rungta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Corina S. Păsăreanu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Păsăreanu, C.S., Visser, W., Bushnell, D. et al. Symbolic PathFinder: integrating symbolic execution with model checking for Java bytecode analysis. Autom Softw Eng 20, 391–425 (2013). https://doi.org/10.1007/s10515-013-0122-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10515-013-0122-2

Keywords

Search

Navigation