Automatically Proving Thousands of Verification Conditions Using an SMT Solver: An Empirical Study

  • Aditi Tagore
  • Diego Zaccai
  • Bruce W. Weide
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7226)


Recently it has become possible to verify full functional correctness of certain kinds of software using automated theorem-proving technology. Empirical studies of the difficulty of automatically proving diverse verification conditions (VCs) would be helpful. For example, they could help direct those developing formal specifications toward techniques that tend to simplify VCs. They could also help focus the efforts of those improving automated theorem-proving tools that are targeted to handle VCs. This study explores two specific empirical questions of this sort: How does an SMT solver perform on VCs that involve user-defined mathematical functions and predicates? When it does not perform well, what can be done to improve the prospects for automated proof? Experience using Z3 to prove VCs for a solution to a fully generic sorting benchmark, along with thousands of other VCs generated for both clients and implementations of dozens of RESOLVE software components, suggests that providing the prover with universal algebraic lemmas about user-defined mathematical functions and predicates results in better outcomes than expanding (unfolding) definitions. The importance of such lemmas might not be surprising to those who have tried to carry out such proofs manually or with the help of an interactive prover, but the damage sometimes caused by expanding definitions might be unexpected. A large empirical study of these phenomena in the context of automated software verification has not been previously reported.


Mathematical Function Predicate Symbol Automate Proof Presburger Arithmetic Reference Semantic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sitaraman, M., Weide, B.: Component-based software using RESOLVE. SIGSOFT Softw. Eng. Notes 19, 21–63 (1994)CrossRefGoogle Scholar
  2. 2.
    Kirschenbaum, J., Adcock, B., Bronish, D., Smith, H., Harton, H., Sitaraman, M., Weide, B.W.: Verifying Component-Based Software: Deep Mathematics or Simple Bookkeeping? In: Edwards, S.H., Kulczycki, G. (eds.) ICSR 2009. LNCS, vol. 5791, pp. 31–40. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Sitaraman, M., et al.: Building a push-button RESOLVE verifier: Progress and challenges. Formal Aspects of Computing 23, 607–626 (2011)CrossRefGoogle Scholar
  4. 4.
    Leino, K.R.M.: Dafny: An Automatic Program Verifier for Functional Correctness. In: Clarke, E.M., Voronkov, A. (eds.) LPAR-16 2010. LNCS, vol. 6355, pp. 348–370. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  5. 5.
    de Moura, L., Bjørner, N.: Z3: An Efficient SMT Solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Weide, B.W., Sitaraman, M., Harton, H.K., Adcock, B., Bucci, P., Bronish, D., Heym, W.D., Kirschenbaum, J., Frazier, D.: Incremental Benchmarks for Software Verification Tools and Techniques. In: Shankar, N., Woodcock, J. (eds.) VSTTE 2008. LNCS, vol. 5295, pp. 84–98. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Hoare, T.: The verifying compiler: A grand challenge for computing research. J. ACM 50, 63–69 (2003)CrossRefGoogle Scholar
  8. 8.
    Heym, W.D.: Computer program verification: improvements for human reasoning. PhD thesis, The Ohio State University, Columbus, OH, USA (1995)Google Scholar
  9. 9.
    Sitaraman, M., Atkinson, S., Kulczycki, G., Weide, B.W., Long, T.J., Bucci, P., Heym, W.D., Pike, S.M., Hollingsworth, J.E.: Reasoning about Software-Component Behavior. In: Frakes, W.B. (ed.) ICSR 2000. LNCS, vol. 1844, pp. 266–283. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  10. 10.
    Leino, K.R.M.: Specification and verification of object-oriented software. Marktoberdorf International Summer School 2008, lecture notes (2008)Google Scholar
  11. 11.
    Leino, K.R.M.: This is Boogie 2. Manuscript KRML 178 (2008),
  12. 12.
    Leino, K.R.M., Monahan, R.: Dafny Meets the Verification Benchmarks Challenge. In: Leavens, G.T., O’Hearn, P., Rajamani, S.K. (eds.) VSTTE 2010. LNCS, vol. 6217, pp. 112–126. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Nelson, C.G.: Techniques for program verification. PhD thesis, Stanford University, Stanford, CA, USA (1980)Google Scholar
  14. 14.
    Kaufmann, M., Manolios, P., Moore, J.S. (eds.) Computer-Aided Reasoning: ACL2 Case Studies. Kluwer Academic Publishers (2000)Google Scholar
  15. 15.
    Detlefs, D., Nelson, G., Saxe, J.B.: Simplify: a theorem prover for program checking. J. ACM 52, 365–473 (2005)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Aditi Tagore
    • 1
  • Diego Zaccai
    • 1
  • Bruce W. Weide
    • 1
  1. 1.Dept. of Computer Science and EngineeringThe Ohio State UniversityColumbusUSA

Personalised recommendations