Advertisement

Automated Software Engineering

, Volume 20, Issue 3, pp 391–425 | Cite as

Symbolic PathFinder: integrating symbolic execution with model checking for Java bytecode analysis

  • Corina S. PăsăreanuEmail author
  • Willem Visser
  • David Bushnell
  • Jaco Geldenhuys
  • Peter Mehlitz
  • Neha Rungta
Article

Abstract

Symbolic PathFinder (SPF) is a software analysis tool that combines symbolic execution with model checking for automated test case generation and error detection in Java bytecode programs. In SPF, programs are executed on symbolic inputs representing multiple concrete inputs and the values of program variables are represented by expressions over those symbolic inputs. Constraints over these expressions are generated from the analysis of different paths through the program. The constraints are solved with off-the-shelf solvers to determine path feasibility and to generate test inputs. Model checking is used to explore different symbolic program executions, to systematically handle aliasing in the input data structures, and to analyze the multithreading present in the code. SPF incorporates techniques for handling input data structures, strings, and native calls to external libraries, as well as for solving complex mathematical constraints. We describe the tool and its application at NASA, in academia, and in industry.

Keywords

Symbolic execution Model checking Testing Java 

References

  1. Anand, S., Păsăreanu, C.S., Visser, W.: JPF–SE: a symbolic execution extension to Java PathFinder. In: Proc. 13th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). LNCS, vol. 4424, pp. 134–138. Springer, Berlin (2007) CrossRefGoogle Scholar
  2. Anand, S., Păsăreanu, C.S., Visser, W.: Symbolic execution with abstraction. Int. J. Softw. Tools Technol. Transf. 11(1), 53–67 (2009) CrossRefGoogle Scholar
  3. Balasubramanian, D., Păsăreanu, C.S., Whalen, M.W., Karsai, G., Lowry, M.R.: Polyglot: modeling and analysis for multiple statechart formalisms. In: Proc. 2011 International Symposium on Software Testing and Analysis (ISSTA), pp. 45–55 (2011) CrossRefGoogle Scholar
  4. Balasubramanian, D., Păsăreanu, C.S., Biatek, J., Pressburger, T., Karsai, G., Lowry, M.R., Whalen, M.W.: Integrating statechart components in Polyglot. In: Proc. NASA Formal Methods 2012, pp. 267–272 (2012) CrossRefGoogle Scholar
  5. Barrett, C., Tinelli, C.: CVC3. In: Proc. 19th Intl. Conf. on Computer Aided Verification (CAV). LNCS, vol. 4590, pp. 298–302. Springer, Berlin (2007) CrossRefGoogle Scholar
  6. Bjørner, N., Tillmann, N., Voronkov, A.: Path feasibility analysis for string-manipulating programs. In: Proc. 15th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). LNCS, vol. 5505, pp. 307–321. Springer, Berlin (2009) CrossRefGoogle Scholar
  7. Borges, M., D’Amorim, M., Anand, S., Bushnell, D., Păsăreanu, C.S.: Symbolic execution with interval solving and meta-heuristic search. In: Proc. 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation (ICST), pp. 111–120 (2012) CrossRefGoogle Scholar
  8. Bush, W., Pincus, J., Sielaff, D.: A static analyzer for finding dynamic programming errors. Softw. Pract. Exp. 30(7), 775–802 (2000) CrossRefzbMATHGoogle Scholar
  9. Bushnell, D.: Continuity analysis for floating point software. In: Numerical Software Verification Workshop, NSV-2011 (2011) Google Scholar
  10. Cadar, C., Dunbar, D., Engler, D.: KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI’08, pp. 209–224. USENIX Association, Berkeley (2008a) Google Scholar
  11. Cadar, C., Ganesh, V., Pawlowski, P., Dill, D., Engler, D.: EXE: automatically generating inputs of death. ACM Trans. Inf. Syst. Secur. 12(2), 1–38 (2008b) CrossRefGoogle Scholar
  12. Cadar, C., Godefroid, P., Khurshid, S., Păsăreanu, C.S., Sen, K., Tillmann, N., Visser, W.: Symbolic execution for software testing in practice: preliminary assessment. In: Proc. 33rd International Conference on Software Engineering (ICSE), pp. 1066–1071 (2011) Google Scholar
  13. CERT/CC: Cert Advisory: multiple vulnerabilities in WU-FTPD. Tech. Rep. CA–2001–33 (2001) Google Scholar
  14. Choco: Java constraint solver (2012). http://choco.emn.fr
  15. Cristensen, A.S., Møller, A., Schwartzbach, M.I.: Precise analysis of string expressions. In: Proc. 10th Intl. Symposium on Static Analysis (SAS). LNCS, vol. 2694, pp. 1–18. Springer, Berlin (2003) Google Scholar
  16. Clarke, L.A.: A system to generate test data and symbolically execute programs. IEEE Trans. Softw. Eng. 2, 215–222 (1976). doi: 10.1109/TSE.1976.233817, http://dl.acm.org/citation.cfm?id=1313320.1313532 CrossRefGoogle Scholar
  17. Coen-Porisini, A., Denaro, G., Ghezzi, C., Pezzé, M.: Using symbolic execution for verifying safety-critical systems. In: Proc. ESEC/SIGSOFT FSE, p. 151. ACM Press, New York (2001) Google Scholar
  18. Collingbourne, P., Cadar, C., Kelly, P.H.: Symbolic crosschecking of floating-point and simd code. In: Proc. of the 6th Conference on Computer Systems, EuroSys’11, pp. 315–328. ACM Press, New York (2011). doi: 10.1145/1966445.1966475 Google Scholar
  19. d’Amorim, M., Pacheco, C., Xie, T., Marinov, D., Ernst, M.D.: An empirical comparison of automated generation and classification techniques for object-oriented unit testing. In: Proc. 21st IEEE/ACM Intl. Conf. on Automated Software Engineering (ASE), pp. 59–68. IEEE Computer Society, Washington (2006) CrossRefGoogle Scholar
  20. de Moura, L., Bjørner, N.: Z3: an efficient SMT solver. In: Proc. 14th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). LNCS, vol. 4963, pp. 337–340. Springer, Berlin (2008) CrossRefGoogle Scholar
  21. Deng, X., Lee, J., Robby: Bogor/Kiasan: a k-bounded symbolic execution for checking strong heap properties of open systems. In: Proc. 21st IEEE/ACM Intl. Conf. on Automated Software Engineering (ASE), pp. 157–166. IEEE Computer Society, Washington (2006) CrossRefGoogle Scholar
  22. Deng, X., Hatcliff, J., Robby: Kiasan/KUnit: automatic test case generation and analysis feedback for open object-oriented systems. In: 3rd International Workshop TAIC PART—Mutation Analysis, pp. 3–12 (2007) Google Scholar
  23. Deng, X., Lee, J., Robby: Efficient and formal generalized symbolic execution. Autom. Softw. Eng. 19, 233–301 (2012). doi: 10.1007/s10515-011-0089-9 CrossRefGoogle Scholar
  24. Detlefs, D.L., Leino, K.R.M., Nelson, G., Saxe, J.B.: Extended static checking. SRC Research Report 159. COMPAQ Systems Research Center (1998) Google Scholar
  25. Filieri, A., Păsăreanu, C.S., Visser, W.: Reliability analysis in symbolic PathFinder. In: Proc. 35th International Conference on Software Engineering (ICSE) (2013) Google Scholar
  26. Geldenhuys, J., Dwyer, M.B., Visser, W.: Probabilistic symbolic execution. In: Proc. International Symposium on Software Testing and Analysis (ISSTA), pp. 166–176 (2012) Google Scholar
  27. Godefroid, P., Klarlund, N., Sen, K.: Dart: directed automated random testing. In: Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pp. 213–223. ACM Press, New York (2005) CrossRefGoogle Scholar
  28. Godefroid, P., de Halleux, P., Nori, A., Rajamani, S., Schulte, W., Tillmann, N., Levin, M.: Automating software testing using program analysis. IEEE Softw. 25(5), 30–37 (2008) CrossRefGoogle Scholar
  29. Hooimeijer, P., Weimer, W.: A decision procedure for subset constraints over regular languages. In: Proc. 2009 ACM SIGPLAN Conf. on Programming Language Design and Implementation (PLDI), pp. 188–198. ACM Press, New York (2009) CrossRefGoogle Scholar
  30. Hooimeijer, P., Weimer, W.: Solving string constraints lazily. In: Pecheur, C., Andrews, J. (eds.) Proc. 25th IEEE/ACM Intl. Conf. on Automated Software Engineering (ASE), pp. 377–386. ACM Press, New York (2010) CrossRefGoogle Scholar
  31. Hooimeijer, P., Molnar, D., Saxena, P., Veanes, M.: Modeling imperative string operations with transducers. Tech. Rep. MSR–TR–2010–96, Microsoft (2010) Google Scholar
  32. Jayaraman, K., Harvison, D., Ganesh, V., Kiezun, A.: JFuzz: a concolic whitebox fuzzer for Java. In: NASA Formal Methods Symposium, NASA Technical Memorandum (2009) Google Scholar
  33. JPF: JPF project (2012). http://babelfish.arc.nasa.gov/trac/jpf
  34. Khurshid, S., Păsăreanu, C.S., Visser, W.: Generalized symbolic execution for model checking and testing. In: Proc. 9th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), pp. 553–568 (2003) CrossRefGoogle Scholar
  35. Kieżun, A.: Effective software testing with a string-constraint solver. PhD thesis, Massachusetts Institute of Technology, USA (2009) Google Scholar
  36. Kieżun, A., Ganesh, V., Guo, P.J., Hooimeijer, P., Ernst, M.D.: HAMPI: a solver for string constraints. In: Rothermel, G., Dillon, L.K. (eds.) Proc. 2009 International Symposium on Software Testing and Analysis (ISSTA), pp. 105–116. ACM Press, New York (2009) Google Scholar
  37. King, J.C.: Symbolic execution and program testing. Commun. ACM 19, 385–394 (1976). doi: 10.1145/360248.360252 CrossRefzbMATHGoogle Scholar
  38. LattE: LattE Integrale. UC Davis, Mathematics (2012). http://www.math.ucdavis.edu/~latte
  39. Li, G., Ghosh, I., Rajan, S.P.: Klover: a symbolic execution and automatic test generation tool for C++ programs. In: Proceedings of the 23rd International Conference on Computer Aided Verification, CAV’11, pp. 609–615. Springer, Berlin (2011). http://dl.acm.org/citation.cfm?id=2032305.2032354 CrossRefGoogle Scholar
  40. Li, X., Shannon, D., Ghosh, I., Ogawa, M., Rajan, S., Khurshid, S.: Context-sensitive relevancy analysis for efficient symbolic execution. In: Asian Symposium on Programming Languages and Systems (APLAS) (2008) Google Scholar
  41. Loera, J.A.D., Dutra, B., Köppe, M., Moreinis, S., Pinto, G., Wu, J.: Software for exact integration of polynomials over polyhedra (2011). arXiv:1108.0117v2 [math.MG]
  42. Mirzaei, N., Malek, S., Păsăreanu, C., Esfahani, N., Mahmood, R.: Testing Android apps through symbolic execution. In: JPF Workshop (2012) Google Scholar
  43. Păsăreanu, C.S., Visser, W.: Verification of Java programs using symbolic execution and invariant generation. In: Proc. of 11th International SPIN Workshop (SPIN). LNCS, vol. 2989, pp. 164–181. Springer, Berlin (2004) Google Scholar
  44. Păsăreanu, C.S., Mehlitz, P.C., Bushnell, D.H., Gundy-Burlet, K., Lowry, M., Person, S., Pape, M.: Combining unit-level symbolic execution and system-level concrete execution for testing NASA software. In: Proc. 2008 International Symposium on Software Testing and Analysis (ISSTA), pp. 15–26 (2008) CrossRefGoogle Scholar
  45. Păsăreanu, C.S., Rungta, N., Visser, W.: Symbolic execution with mixed concrete-symbolic solving. In: Proceedings of the 2011 International Symposium on Software Testing and Analysis (ISSTA), pp. 34–44. ACM Press, New York (2011). doi: 10.1145/2001420.2001425 CrossRefGoogle Scholar
  46. Person, S., Yang, G., Rungta, N., Khurshid, S.: Directed incremental symbolic execution. In: Proc. 2011 ACM SIGPLAN Conf. on Programming Language Design and Implementation (PLDI), pp. 504–515 (2011) Google Scholar
  47. Phan, Q.-S., Malacaria, P., Tkachuk, O., Păsăreanu, C.S.: Symbolic quantitative information flow. Softw. Eng. Notes 37, 1–5 (2012) CrossRefGoogle Scholar
  48. Rajan, S., Tkachuk, O., Prasad, M., Ghosh, I., Goel, N., Uehara, T.: WEAVE: WEb Applications Validation Environment. In: Proc. 31st International Conference on Software Engineering (ICSE Companion) (2009) Google Scholar
  49. Redelinghuys, G.: Symbolic string execution. Master’s thesis, Stellenbosch University (2012) Google Scholar
  50. Redis: Redis NoSQL database (2012). http://redis.io
  51. Rossi, F., van Beek, P., Walsh, T.: Handbook of Constraint Programming. Elsevier, Amsterdam (2006) zbMATHGoogle Scholar
  52. Rungta, N., Mercer, E.G., Visser, W.: Efficient testing of concurrent programs with abstraction-guided symbolic execution. In: Proc. of 16th International SPIN Workshop (SPIN), pp. 174–191 (2009) Google Scholar
  53. Sanfilippo, S., Noordhuis, P.: Redis: the Definitive Guide. O’Reilly Media, Sebastopol (2012) Google Scholar
  54. Santelices, R., Harrold, M.J.: Exploiting program dependencies for scalable multiple-path symbolic execution. In: Proc. 2010 International Symposium on Software Testing and Analysis (ISSTA), pp. 195–206 (2010) Google Scholar
  55. Sasnauskas, R., Dustmann, O.S., Kaminski, B.L., Wehrle, K., Weise, C., Kowalewski, S.: Scalable symbolic execution of distributed systems. In: Proceedings of the 2011 31st International Conference on Distributed Computing Systems, ICDCS ’11, pp. 333–342. IEEE Computer Society, Washington (2011). doi: 10.1109/ICDCS.2011.28 CrossRefGoogle Scholar
  56. Saxena, P., Akhawe, D., Hanna, S., Mao, F., McCamant, S., Song, D.: A symbolic execution framework for JavaScript. In: Proc. 31st IEEE Symposium on Security and Privacy, pp. 513–528. IEEE Computer Society, Washington (2010) Google Scholar
  57. Sen, K., Agha, G.: CUTE and jCUTE: concolic unit testing and explicit path model-checking tools. In: Proc. 18th International Conference on Computer Aided Verification (CAV), pp. 419–423 (2006) CrossRefGoogle Scholar
  58. Sen, K., Agha, G.: A race-detection and flipping algorithm for automated testing of multithreaded programs. In: Proc. Haifa Verification Conference (HVC). LNCS, vol. 4383, pp. 166–182. Springer, Berlin (2007) Google Scholar
  59. Shannon, D., Hajra, S., Lee, A., Zhan, D., Khurshid, S.: Abstracting symbolic execution with string analysis. In: Proc. Testing: Academic and Industrial Conf. Practice and Research Techniques, pp. 13–22. IEEE Computer Society, Washington (2007) Google Scholar
  60. Shannon, D., Ghosh, I., Rajan, S.P., Khurshid, S.: Efficient symbolic execution of strings for validating web applications. In: Proc. 2nd Intl. Workshop on Defects in Large Software Systems, pp. 22–26. ACM Press, New York (2009) Google Scholar
  61. Siegel, S., Zirkel, T.: Tass: the toolkit for accurate scientific software. Math. Comput. Sci. 5, 395–426 (2011). doi: 10.1007/s11786-011-0100-7 CrossRefzbMATHGoogle Scholar
  62. Siegel, S., Mironova, A., Avrunin, G., Clarke, L.: Using model checking with symbolic execution to verify parallel numerical programs. In: Proc. 2006 International Symposium on Software Testing and Analysis (ISSTA), pp. 157–168. ACM Press, New York (2006) CrossRefGoogle Scholar
  63. Souza, M., Borges, M., d’Amorim, M., Păsăreanu, C.S.: CORAL: solving complex constraints for Symbolic PathFinder. In: NASA Formal Methods, pp. 359–374 (2011) CrossRefGoogle Scholar
  64. SPF: Symbolic Pathfinder (jpf-symbc) (2012). http://babelfish.arc.nasa.gov/trac/jpf
  65. Staats, M., Păsăreanu, C.: Parallel symbolic execution for structural test generation. In: Proc. 2010 International Symposium on Software Testing and Analysis (ISSTA), pp. 183–194. ACM Press, New York (2010). doi: 10.1007/s10515-013-0122-2 Google Scholar
  66. Tillmann, N., de Halleux, J.: Pex–white box test generation for .NET. In: Beckert, B., Hähnle, R. (eds.) Proc. 2nd Intl. Conf. on Tests and Proofs. LNCS, vol. 4966, pp. 134–153. Springer, Berlin (2008) CrossRefGoogle Scholar
  67. Tomb, A., Brat, G., Visser, W.: Variably interprocedural program analysis for run-time error detection. In: Proc. 2007 International Symposium on Software Testing and Analysis (ISSTA), pp. 97–107. ACM Press, New York (2007) CrossRefGoogle Scholar
  68. Veanes, M., de Halleux, P., Tillmann, N.: Rex: symbolic regular expression explorer. In: Proc. 3rd Intl. Conf. on Software Testing, Verification and Validation, pp. 498–507. IEEE Computer Society, Washington (2010) Google Scholar
  69. Visser, W., Havelund, K., Brat, G.P., Park, S., Lerda, F.: Model checking programs. Autom. Softw. Eng. 10(2), 203–232 (2003) CrossRefGoogle Scholar
  70. Visser, W., Păsăreanu, C.S., Pelánek, R.: Test input generation for Java containers using state matching. In: Proc. 2006 International Symposium on Software Testing and Analysis (ISSTA), pp. 37–48 (2006) CrossRefGoogle Scholar
  71. Visser, W., Geldenhuys, J., Dwyer, M.B.: Green: reducing, reusing and recycling constraints in program analysis. In: International Symposium on the Foundations of Software Engineering (FSE), Cary, North Carolina, USA (2012) Google Scholar
  72. Xie, T., Marinov, D., Schulte, W., Notkin, D.: Symstra: a framework for generating object-oriented unit tests using symbolic execution. In: Proceedings of the 11th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), pp. 365–381. Springer, Berlin (2005) CrossRefGoogle Scholar
  73. Yang, G., Păsăreanu, C.S., Khurshid, S.: Memoized symbolic execution. In: Proc. International Symposium on Software Testing and Analysis (ISSTA), pp. 144–154 (2012) Google Scholar
  74. Yices: Yices SMT Solver (2012). http://yices.csl.sri.com/
  75. Yu, F., Bultan, T., Cova, M., Ibarra, O.H.: Symbolic string verification: an automata-based approach. In: Proc. 15th Intl. SPIN Workshop on Model Checking Software. LNCS, vol. 5156, pp. 306–324. Springer, Berlin (2008) CrossRefGoogle Scholar
  76. Yu, F., Alkhalaf, M., Bultan, T.: Stranger: an automata-based string analysis tool for PHP. In: Proc. 16th Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). LNCS, vol. 6015, pp. 154–157. Springer, Berlin (2010) CrossRefGoogle Scholar
  77. Zhang, P., Elbaum, S.G., Dwyer, M.B.: Automatic generation of load tests. In: Alexander, P., Păsăreanu, C.S., Hosking, J.G. (eds.) Proc. 26th IEEE/ACM Intl. Conf. on Automated Software Engineering, pp. 43–52. IEEE Press, New York (2011) Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Corina S. Păsăreanu
    • 1
    Email author
  • Willem Visser
    • 2
  • David Bushnell
    • 1
  • Jaco Geldenhuys
    • 2
  • Peter Mehlitz
    • 1
  • Neha Rungta
    • 1
  1. 1.NASA Ames Research CenterMoffett FieldUSA
  2. 2.University of StellenboschStellenboschSouth Africa

Personalised recommendations