Statically Validating Must Summaries for Incremental Compositional Dynamic Test Generation

  • Patrice Godefroid
  • Shuvendu K. Lahiri
  • Cindy Rubio-González
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6887)


Compositional dynamic test generation can achieve significant scalability by memoizing symbolic execution sub-paths as test summaries. In this paper, we formulate the problem of statically validating symbolic test summaries against code changes. Summaries that can be proved still valid using a static analysis of a new program version do not need to be retested or recomputed dynamically. In the presence of small code changes, incrementality can considerably speed up regression testing since static checking is much cheaper than dynamic checking and testing. We provide several checks ranging from simple syntactic ones to ones that use a theorem prover. We present preliminary experimental results comparing these approaches on three large Windows applications.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aho, A., Sethi, R., Ullman, J.: Compilers: Principles, Techniques and Tools. Addison-Wesley, Reading (1986)zbMATHGoogle Scholar
  2. 2.
    Anand, S., Godefroid, P., Tillmann, N.: Demand-Driven Compositional Symbolic Execution. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 367–381. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Barnett, M., Chang, B.E., DeLine, R., Jacobs, B., Leino, K.R.M.: Boogie: A modular reusable verifier for object-oriented programs. In: de Boer, F.S., Bonsangue, M.M., Graf, S., de Roever, W.-P. (eds.) FMCO 2005. LNCS, vol. 4111, pp. 364–387. Springer, Heidelberg (2005)Google Scholar
  4. 4.
    Barnett, M., Leino, K.R.M.: Weakest-precondition of unstructured programs. In: PASTE 2005, pp. 82–87 (2005)Google Scholar
  5. 5.
    Cadar, C., Ganesh, V., Pawlowski, P.M., Dill, D.L., Engler, D.R.: EXE: Automatically Generating Inputs of Death. In: ACM CCS (2006)Google Scholar
  6. 6.
    Chandra, S., Fink, S.J., Sridharan, M.: Snugglebug: A Powerful Approach to Weakest Preconditions. In: PLDI 2009 (2009)Google Scholar
  7. 7.
    Conway, C.L., Namjoshi, K.S., Dams, D., Edwards, S.A.: Incremental algorithms for inter-procedural analysis of safety properties. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 449–461. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  8. 8.
    de Moura, L., Bjorner, N.: Z3: An Efficient SMT Solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Dijkstra, E.W.: Guarded commands, nondeterminacy and formal derivation of programs. Communications of the ACM 18, 453–457 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Edwards, A., Srivastava, A., Vo, H.: Vulcan: Binary transformation in a distributed environment. Technical report, MSR-TR-2001-50, Microsoft Research (2001)Google Scholar
  11. 11.
    Godefroid, P.: Compositional Dynamic Test Generation. In: POPL 2007, pp. 47–54 (2007)Google Scholar
  12. 12.
    Godefroid, P.: Software Model Checking Improving Security of a Billion Computers. In: Păsăreanu, C.S. (ed.) Model Checking Software. LNCS, vol. 5578, pp. 1–1. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  13. 13.
    Godefroid, P., Huth, M., Jagadeesan, R.: Abstraction-Based Model Checking Using Modal Transition Systems. In: Larsen, K.G., Nielsen, M. (eds.) CONCUR 2001. LNCS, vol. 2154, pp. 426–440. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  14. 14.
    Godefroid, P., Klarlund, N., Sen, K.: DART: Directed Automated Random Testing. In: PLDI 2005, pp. 213–223 (2005)Google Scholar
  15. 15.
    Godefroid, P., Levin, M., Molnar, D.: Automated Whitebox Fuzz Testing. In: NDSS 2008, pp. 151–166 (2008)Google Scholar
  16. 16.
    Godefroid, P., Nori, A., Rajamani, S., Tetali, S.: Compositional Must Program Analysis: Unleashing The Power of Alternation. In: POPL 2010 (2010)Google Scholar
  17. 17.
    Graves, T.L., Harrold, M.J., Kim, J.-M., Porter, A., Rothermel, G.: An Empirical Study of Regression Test Selection Techniques. ACM Transactions on Software Engineering and Methodology (TOSEM) 10(2), 184–208 (2001)CrossRefzbMATHGoogle Scholar
  18. 18.
    Gurfinkel, A., Wei, O., Chechik, M.: Yasm: A Software Model-Checker for Verification and Refutation. In: Ball, T., Jones, R.B. (eds.) CAV 2006. LNCS, vol. 4144, pp. 170–174. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  19. 19.
    Hallem, S., Chelf, B., Xie, Y., Engler, D.: A System and Language for Building System-Specific Static Analyses. In: PLDI 2002, pp. 69–82 (2002)Google Scholar
  20. 20.
    Hoenicke, J., Leino, K.R.M., Podelski, A., Schäf, M., Wies, T.: It’s doomed; we can prove it. In: Cavalcanti, A., Dams, D.R. (eds.) FM 2009. LNCS, vol. 5850, pp. 338–353. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  21. 21.
    Molnar, D., Li, X.C., Wagner, D.: Dynamic test generation to find integer bugs in x86 binary linux programs. In: Proc. of the 18th Usenix Security Symposium (2009)Google Scholar
  22. 22.
    Person, S., Dwyer, M.B., Elbaum, S.G., Pasareanu, C.S.: Differential symbolic execution. In: SIGSOFT FSE, pp. 226–237 (2008)Google Scholar
  23. 23.
    Person, S., Yang, G., Rungta, N., Khurshid, S.: Directed Incremental Symbolic Execution. In: PLDI 2011, pp. 504–515 (2011)Google Scholar
  24. 24.
    Ramalingam, G., Reps, T.: A Categorized Bibliography on Incremental Algorithms. In: POPL 1993, pp. 502–510 (1993)Google Scholar
  25. 25.
    Reps, T., Horwitz, S., Sagiv, M.: Precise Interprocedural Dataflow Analysis via Graph Reachability. In: POPL 1995, pp. 49–61 (1995)Google Scholar
  26. 26.
    Santelices, R.A., Harrold, M.J., Orso, A.: Precisely detecting runtime change interactions for evolving software. In: ICST, pp. 429–438 (2010)Google Scholar
  27. 27.
    Satisfiability Modulo Theories Library (SMT-LIB),

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Patrice Godefroid
    • 1
  • Shuvendu K. Lahiri
    • 1
  • Cindy Rubio-González
    • 2
  1. 1.Microsoft ResearchRedmondUSA
  2. 2.University of WisconsinMadisonUSA

Personalised recommendations