Comparing Verification Condition Generation with Symbolic Execution: An Experience Report

  • Ioannis T. Kassios
  • Peter Müller
  • Malte Schwerhoff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7152)

Abstract

There are two dominant approaches for the construction of automatic program verifiers, Verification Condition Generation (VCG) and Symbolic Execution (SE). Both techniques have been used to develop powerful program verifiers. However, to the best of our knowledge, no systematic experiment has been conducted to compare them.

This paper reports on such an experiment. We have used the specification and programming language Chalice and compared the performance of its standard VCG verifier with a newer SE engine called Syxc, using the Chalice test suite as a benchmark. We have focused on comparing the efficiency of the two approaches, choosing suitable metrics for that purpose. Our metrics also suggest conclusions about the predictability of the performance. Our results show that verification via SE is roughly twice as fast as via VCG. It requires only a small fraction of the quantifier instantiations that are performed in the VCG-based verification.

Keywords

Theorem Prover Path Condition Proof Obligation Symbolic Execution Separation Logic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Almeida, J.B., Frade, M.J., Pinto, J.S., Melo de Sousa, S.: Verifying C programs. In: Rigorous Software Development. Undergraduate Topics in Computer Science, pp. 241–256. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    Banerjee, A., Naumann, D., Rosenberg, S.: Regional Logic for Local Reasoning about Global Invariants. In: Ryan, M. (ed.) ECOOP 2008. LNCS, vol. 5142, pp. 387–411. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Barnett, M., Fähndrich, M., Leino, K.R.M., Müller, P., Schulte, W., Venter, H.: Specification and verification: The Spec# experience. Communications of the ACM 54(6), 81–91 (2011)CrossRefGoogle Scholar
  4. 4.
    Beckert, B., Hähnle, R., Schmitt, P.H. (eds.): Verification of Object-Oriented Software. LNCS (LNAI), vol. 4334, pp. 69–177. Springer, Heidelberg (2007)Google Scholar
  5. 5.
    Berdine, J., Calcagno, C., O’Hearn, P.: Smallfoot: Modular Automatic Assertion Checking with Separation Logic. In: de Boer, F.S., Bonsangue, M.M., Graf, S., de Roever, W.-P. (eds.) FMCO 2005. LNCS, vol. 4111, pp. 115–137. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Boyland, J.: Checking Interference with Fractional Permissions. In: Cousot, R. (ed.) SAS 2003. LNCS, vol. 2694, pp. 55–72. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  7. 7.
    Chalin, P., Kiniry, J.R., Leavens, G.T., Poll, E.: Beyond Assertions: Advanced Specification and Verification with JML and ESC/Java2. In: de Boer, F.S., Bonsangue, M.M., Graf, S., de Roever, W.-P. (eds.) FMCO 2005. LNCS, vol. 4111, pp. 342–363. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Cohen, E., Dahlweid, M., Hillebrand, M., Leinenbach, D., Moskał, M., Santen, T., Schulte, W., Tobies, S.: VCC: A Practical System for Verifying Concurrent C. In: Berghofer, S., Nipkow, T., Urban, C., Wenzel, M. (eds.) TPHOLs 2009. LNCS, vol. 5674, pp. 23–42. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  9. 9.
    Dijkstra, E.W.: A Discipline of Programming. Prentice Hall Series in Automatic Computation. Prentice Hall (1976)Google Scholar
  10. 10.
    Filliâtre, J.C.: Why: a multi-language multi-prover verification tool. Technical Report 1366, LRI, Université Paris Sud (2003)Google Scholar
  11. 11.
    Heule, S., Leino, K.R.M., Müller, P., Summers, A.J.: Fractional permissions without the fractions. In: Formal Techniques for Java-like Programs, FTfJP (2011)Google Scholar
  12. 12.
    Jacobs, B., Smans, J., Piessens, F.: VeriFast: Imperative programs as proofs. In: VS-Tools Workshop at VSTTE 2010 (2010)Google Scholar
  13. 13.
    Kassios, I.T.: Dynamic Frames: Support for Framing, Dependencies and Sharing without Restrictions. In: Misra, J., Nipkow, T., Sekerinski, E. (eds.) FM 2006. LNCS, vol. 4085, pp. 268–283. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    King, J.C.: Symbolic execution and program testing. Communications of the ACM 19(7), 385–394 (1976)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Klebanov, V., Müller, P., Shankar, N., Leavens, G.T., Wüstholz, V., Alkassar, E., Arthan, R., Bronish, D., Chapman, R., Cohen, E., Hillebrand, M., Jacobs, B., Leino, K.R.M., Monahan, R., Piessens, F., Polikarpova, N., Ridge, T., Smans, J., Tobies, S., Tuerk, T., Ulbrich, M., Weiß, B.: The 1st Verified Software Competition: Experience report. In: Butler, M., Schulte, W. (eds.) FM 2011. LNCS, vol. 6664, pp. 154–168. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  16. 16.
    Leavens, G., Baker, A.L., Ruby, C.: JML: a notation for detailed design. In: Kilov, I., Rumpe, B., Simmonds, I. (eds.) Behavioral Specifications of Businesses and Systems, pp. 175–188. Kluwer (1999)Google Scholar
  17. 17.
    Leino, K.R.M.: Specification and verification of object-oriented software. In: Marktoberdorf International Summer School 2008. Lecture Notes (2008)Google Scholar
  18. 18.
    Leino, K.R.M.: This is Boogie 2. Working Draft (2008), http://-research.microsoft.com/en-us/um/people/leino/papers.html
  19. 19.
    Leino, K.R.M., Müller, P.: Object Invariants in Dynamic Contexts. In: Vetta, A. (ed.) ECOOP 2004. LNCS, vol. 3086, pp. 491–515. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  20. 20.
    Leino, K.R.M., Müller, P.: A Basis for Verifying Multi-Threaded Programs. In: Castagna, G. (ed.) ESOP 2009. LNCS, vol. 5502, pp. 378–393. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  21. 21.
    Leino, K.R.M., Müller, P., Smans, J.: Verification of Concurrent Programs with Chalice. In: Aldini, A., Barthe, G., Gorrieri, R. (eds.) Foundations of Security Analysis and Design V. Lecture Notes In Computer Science, vol. 5705, pp. 195–222. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  22. 22.
    Moura, L., Bjørner, N.: Z3: An Efficient SMT Solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  23. 23.
    Müller, P., Ruskiewicz, J.N.: Using Debuggers to Understand Failed Verification Attempts. In: Butler, M., Schulte, W. (eds.) FM 2011. LNCS, vol. 6664, pp. 73–87. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  24. 24.
    Parkinson, M., Distefano, D.: jStar: Towards practical verification for Java. In: Harris, G.E. (ed.) OOPSLA 2008, pp. 213–226. ACM (2008)Google Scholar
  25. 25.
    Parkinson, M., Summers, A.: The Relationship Between Separation Logic and Implicit Dynamic Frames. In: Barthe, G. (ed.) ESOP 2011. LNCS, vol. 6602, pp. 439–458. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  26. 26.
    Reynolds, J.: Separation logic: A logic for shared mutable data structures. In: LICS 2002, pp. 55–74. IEEE Computer Society (2002)Google Scholar
  27. 27.
    Schmitt, P., Ulbrich, M., Weiß, B.: Dynamic Frames in Java Dynamic Logic. In: Beckert, B., Marché, C. (eds.) FoVeOOS 2010. LNCS, vol. 6528, pp. 138–152. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  28. 28.
    Schwerhoff, M.: Symbolic execution for Chalice. Master’s thesis, ETH Zurich (2011)Google Scholar
  29. 29.
    Smans, J., Jacobs, B., Piessens, F.: Implicit Dynamic Frames: Combining Dynamic Frames and Separation Logic. In: Drossopoulou, S. (ed.) ECOOP 2009. LNCS, vol. 5653, pp. 148–172. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  30. 30.
    Smans, J., Jacobs, B., Piessens, F.: Symbolic execution for implicit dynamic frames (2009), http://people.cs.kuleuven.be/~jan.smans/vericool3/

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ioannis T. Kassios
    • 1
  • Peter Müller
    • 1
  • Malte Schwerhoff
    • 1
  1. 1.ETH ZurichSwitzerland

Personalised recommendations