Abstract
The comprehensive functionality and nontrivial design of realistic general-purpose container libraries pose challenges to formal verification that go beyond those of individual benchmark problems mainly targeted by the state of the art. We present our experience verifying the full functional correctness of EiffelBase2: a container library offering all the features customary in modern language frameworks, such as external iterators, and hash tables with generic mutable keys and load balancing. Verification uses the automated deductive verifier AutoProof, which we extended as part of the present work. Our results indicate that verification of a realistic container library (135 public methods, 8400 LOC) is possible with moderate annotation overhead (1.4 lines of specification per LOC) and good performance (0.2 s per method on average).
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Ameri M, Furia CA (June 2016) Why just Boogie? Translating between intermediate verification languages. In: Proceedings of the 12th international conference on integrated formal methods (iFM), volume 9681 of lecture notes in computer science. Springer, pp 1–17
Barnett M, Chang B-YE, DeLine R, Jacobs B, Leino KRM (2005) Boogie: a modular reusable verifier for object-oriented programs. In: FMCO, pp 364–387
Barnett, M.; DeLine, R.; Fähndrich, M.; Leino, K.R.M.; Schulte, W.: Verification of object-oriented programs with invariants. J Object Technol 3(6), 27–56 (2004)
Blanc, N.; Groce, A.; Kroening, D.: Verifying \({\rm C}^{++}\) with STL containers via predicate abstraction. 22nd IEEE/ACM international conference on automated software engineering (ASE 2007), Nov 5–9, 2007, pp. 521–524. Atlanta, Georgia, USA (2007)
Beyer D, Henzinger TA, Théoduloz G (2006) Lazy shape analysis. In: 18th international conference on computer aided verification, CAV 2006, Seattle, WA, USA, Aug 17–20, 2006, Proceedings, volume 4144 of lecture notes in computer science. Springer, pp 532–546
Beyer D, Henzinger TA, Théoduloz G, Zufferey D (2010) Shape refinement through explicit heap analysis. In: Fundamental approaches to software engineering, volume 6013 of lecture notes in computer science. Springer, pp 263–277
Barnett M, Naumann DA (2004) Friends need a bit more: maintaining invariants over shared state. In: 7th international conference on mathematics of program construction, MPC 2004, Stirling, Scotland, UK, July 12–14, 2004, Proceedings, pp 54–84
Bruns D (2011) Specification of red-black trees: showcasing dynamic frames, model fields and sequences. In: 10th keY symposium, Nijmegen, The Netherlands, Extended Abstract.
Cousot P, Cousot R, Logozzo F (2011) A parametric segmentation functor for fully automatic and scalable array content analysis. In: Proceedings of the 38th ACM SIGPLAN-SIGACT symposium on principles of programming languages, POPL 2011, Austin, TX, USA, Jan 26–28, 2011. ACM, pp 105–118
Cohen E, Dahlweid M, Hillebrand MA, Leinenbach D, Moskal M, Santen T, Schulte W, Tobies S (2009) VCC: a practical system for verifying concurrent C. In: 22nd international conference on theorem proving in higher order logics, TPHOLs 2009, Munich, Germany, Aug 17–20, 2009. Proceedings, volume 5674 of lecture notes in computer science. Springer, pp 23–42
Chin, W.; David, C.; Nguyen, H.H.; Qin, S.: Automated verification of shape, size and bag properties via user-defined predicates in separation logic. Sci Comput Program 77(9), 1006–1036 (2012)
Calcagno, C.; Distefano, D.; O’Hearn, P.W.; Yang, H.: Compositional shape analysis by means of bi-abduction. J ACM 58(6), 26 (2011)
Charles J (2006) Adding native specifications to JML. In: Workshop on formal techniques for java-like programs (FTfJP)
Cormen, T.H.; Leiserson, C.E.; Rivest, R.L.; Stein, C.: Introduction to algorithms, 3rd edn. The MIT Press, Cambridge (2009)
Christakis M, Leino KRM, Schulte W (2014) Formalizing and verifying a modern build language. In: FM 2014: formal methods—19th international symposium, Singapore, May 12–16, 2014. Proceedings, volume 8442 of lecture notes in computer science. Springer, pp 643–657
Cheon, Y.; Leavens, G.; Sitaraman, M.; Edwards, S.: Model variables: cleanly supporting abstraction in design by contract. Softw Pract Exper 35(6), 583–599 (2005)
Chlipala A, Malecha JG, Morrisett G, Shinnar A, Wisnesky R (2009) Effective interactive proofs for higher-order imperative programs. In: Proceeding of the 14th ACM SIGPLAN international conference on functional programming, ICFP 2009, Edinburgh, Scotland, UK, Aug 31–Sept 2, 2009. ACM, pp 79–90
Cok DR (2006) Specifying Java iterators with JML and ESC/Java2. In: Proceedings of the 2006 conference on specification and verification of component-based systems, SAVCBS ’06. ACM, pp 71–74
Dafny example gallery. http://dafny.codeplex.com/SourceControl/latest. Last access Feb 2016.
Dunlop, D.D.; Basili, V.R.: A comparative analysis of functional correctness. ACM Comput Surv 14(2), 229–244 (1982)
Dillig I, Dillig T, Aiken A (2011) Precise reasoning for programs using containers. In: Proceedings of the 38th annual ACM SIGPLAN-SIGACT symposium on principles of programming languages, POPL’11, New York, NY, USA. ACM, pp 187–200
Dross C, Filliâtre J-C, Moy Y (2011) Correct code containing containers. In: 5th international conference on tests and proofs (TAP’11), volume 6706 of lecture notes in computer science, Zurich. Springer, pp 102–118
Moura dL Bjørner N (2008) Z3: an efficient SMT solver. In:14th international conference tools and algorithms for the construction and analysis of systems, TACAS 2008, held as part of the Joint European conferences on theory and practice of software, ETAPS 2008, Budapest, Hungary, March 29–April 6, 2008. Proceedings, volume 4963 of lecture notes in computer science. Springer, pp 337–340
Filliâtre J, Gondelman L, Paskevich A (2014) The spirit of ghost code. In: Proceedings of the 26th international conference on computer aided verification (CAV), volume 8559 of lecture notes in computer science. Springer, pp 1–16
Furia CA, Nordio M, Polikarpova N, Tschannen J (2016) AutoProof: auto-active functional verification of object-oriented programs. Int J Softw Tools Technol Transf, Online since April 2016. http://link.springer.com/article/10.1007/s10009-016-0419-0.
Filliâtre J-C, Paskevich A, Stump A (2012) The 2nd verified software competition: experience report. In: COMPARE, volume 873 of CEUR workshop proceedings. CEUR-WS.org, https://sites.google.com/site/vstte2012/compet.
Furia CA, Poskitt CM, Tschannen J (June 2015) The AutoProof verifier: Usability by non-experts and on standard code. In: Proceedings of the 2nd workshop on formal integrated development environment (F-IDE), volume 187 of electronic proceedings in theoretical computer science. EPTCS, Workshop co-located with FM 2015, pp 42–55
Gamboa, R.A.: A formalization of powerlist algebra in ACL2. J Autom Reason 43(2), 139–172 (2009)
Gulwani S, McCloskey B, Tiwari A (2008) Lifting abstract interpreters to quantified logical domains. In: Proceedings of the 35th ACM SIGPLAN-SIGACT symposium on principles of programming languages, POPL 2008, San Francisco, California, USA, Jan 7–12, 2008. ACM, pp 235–246
Gregor, D.; Schupp, S.: STLlint: lifting static checking from languages to libraries. Softw Pract Exper 36(3), 225–254 (2006)
Gladisch C Tyszberowicz S (2013) Specifying a linked data structure in JML for formal verification and runtime checking. In: Brazilian symposium on formal methods (SBMF), volume 8195 of lecture notes in computer science. Springer, pp 99–114
Hawkins P, Aiken A, Fisher K, Rinard M, Sagiv M (2011) Data representation synthesis. In: Proceedings of the 32Nd ACM SIGPLAN conference on programming language design and implementation, PLDI’11, New York, NY, USA. ACM, pp 38–49
Hatcliff, J.; Leavens, G.T.; Leino, K.R.M.; Müller, P.; Parkinson, M.J.: Behavioral interface specification languages. ACM Comput Surv 44(3), 16 (2012)
Itzhaky S, Bjørner N, Reps TW, Sagiv M, Thakur AV (2014) Property-directed shape analysis. In: 26th international conference computer aided verification, CAV 2014, Held as part of the Vienna Summer of Logic, VSL 2014, Vienna, Austria, July 18–22, 2014. Proceedings, volume 8559 of lecture notes in computer science. Springer, pp 35–51
Documentation of java.util.LinkedList. http://docs.oracle.com/javase/8/docs/api/java/util/LinkedList.html. Last access Feb 2016.
Documentation of java.util.Map. http://docs.oracle.com/javase/8/docs/api/java/util/Map.html. Last access Feb 2016.
Jensen, J.B.; Birkedal, L.; Sestoft, P.: Modular verification of linked lists with views via separation logic. J Object Technol 10(2), 1–20 (2011)
Jacobs S Kuncak V (2011) Towards complete reasoning about axiomatic specifications. In: 12th international conference on verification, model checking, and abstract interpretation, VMCAI 2011, Austin, TX, USA, Jan 23–25, 2011. Proceedings, volume 6538 of lecture notes in computer science. Springer, pp 278–293
Jacobs B, Piessens F, Schulte W (2006) VC generation for functional behavior and non-interference of iterators. In: Proceedings of the 2006 conference on specification and verification of component-based systems, SAVCBS’06. ACM, pp 71–74
Jacobs B, Smans J, Philippaerts P, Vogels F, Penninckx W, Piessens F (2011) VeriFast: A powerful, sound, predictable, fast verifier for C and Java. NASA Form Methods, pp 41–55
Kassios IT (2006) Dynamic frames: support for framing, dependencies and sharing without restrictions. In: FM 2006: formal methods, 14th international symposium on formal methods, Hamilton, Canada, Aug 21–27, 2006. Proceedings, pp 268–283
Klein G, Elphinstone K, Heiser G, Andronick J, Cock D, Derrin P, Elkaduwe D, Engelhardt K, Kolanski R, Norrish M, Sewell T, Tuch H, Winwood S (2009) seL4: formal verification of an OS kernel. In: Proceedings of the 22nd ACM symposium on operating systems principles 2009, SOSP 2009, Big Sky, Montana, USA, Oct 11–14, 2009. ACM, pp 207–220
Kuncak V, Piskac R, Suter P (2010) Ordered sets in the calculus of data structures. In: Computer science logic, 24th international workshop, CSL 2010, 19th annual conference of the EACSL, Brno, Czech Republic, Aug 23–27, 2010. Proceedings, volume 6247 of lecture notes in computer science. Springer, pp 34–48
Kuncak V, Piskac R, Suter P, Wies T (2010) Building a calculus of data structures. In: 11th international conference on verification, model checking, and abstract interpretation, VMCAI 2010, Madrid, Spain, Jan 17–19, 2010. Proceedings, volume 5944 of lecture notes in computer science. Springer, pp 26–44
Kawaguchi M, Rondon PM, Jhala R (2009) Type-based data structure verification. In: Proceedings of the 2009 ACM SIGPLAN conference on programming language design and implementation, PLDI 2009, Dublin, Ireland, June 15–21, 2009, pp 304–315
Leavens, G.T.; Baker, A.L.; Ruby, C.: Preliminary design of JML: a behavioral interface specification language for Java. SIGSOFT Softw Eng Notes 31(3), 1–38 (2006)
Leino KRM (1995) Toward reliable modular programs. Ph.D. thesis, Caltech
Leino KRM (2010) Dafny: An automatic program verifier for functional correctness. In: 16th international conference on logic for programming, artificial intelligence, and reasoning, LPAR-16, Dakar, Senegal, April 25–May 1, 2010, revised selected papers, volume 6355 of lecture notes in computer science. Springer, pp 348–370
Leroy, X.: Formal verification of a realistic compiler. Commun ACM 52(7), 107–115 (2009)
Laviron, V.; Logozzo, F.: Subpolyhedra: a family of numerical abstract domains for the (more) scalable inference of linear inequalities. Softw Tools Technol Transf 13(6), 585–601 (2011)
Leino KRM, Müller P (2004) Object invariants in dynamic contexts. In: ECOOP 2004—object-oriented programming, 18th European conference, Oslo, Norway, June 14–18, 2004, Proceedings, volume 3086 of lecture notes in computer science. Springer, pp 491–516
Leino KRM, Müller P (2006) A verification methodology for model fields. In: 15th European symposium on programming—programming languages and systems, ESOP 2006, Held as part of the joint European conferences on theory and practice of software, ETAPS 2006, Vienna, Austria, March 27–28, 2006, Proceedings, volume 3924 of lecture notes in computer science. Springer, pp 115–130
Leino KRM, Müller P (Sept 2009) Using the Spec# language, methodology, and tools to write bug-free programs. http://www.codeplex.com/Download?ProjectName=specsharp&DownloadId=84056,
Leino KRM, Moskal M (2010) Usable auto-active verification. In: Usable verification workshop. http://fm.csl.sri.com/UV10/
Leino KRM, Moskal M (2010) VACID-0: Verification of ample correctness of invariants of data-structures, 0 edn. VSTTE Workshops, http://goo.gl/0VnvyO
Lochbihler A (2013) Light-weight containers for Isabelle: efficient, extensible, nestable. In: 4th international conference on interactive theorem proving, ITP 2013, Rennes, France, July 22–26, 2013. Proceedings, volume 7998 of lecture notes in computer science. Springer, pp 116–132
Leino KRM, Polikarpova N (2013) Verified calculations. In: 5th international conference on verified software: theories, tools, experiments, VSTTE 2013, Menlo Park, CA, USA, May 17–19, 2013, revised selected papers, pp 170–190
Leino KRM, Poetzsch-Heffter A, Zhou Y (2002) Using data groups to specify and check side effects. In: Proceedings of the 2002 ACM SIGPLAN conference on programming language design and implementation (PLDI), Berlin, Germany, June 17–19, 2002, pp 246–257
Lahiri SK, Qadeer S (2008) Back to the future: revisiting precise program verification using SMT solvers. In: Proceedings of the 35th ACM SIGPLAN-SIGACT symposium on principles of programming languages, POPL 2008, San Francisco, California, USA, Jan 7–12, 2008. ACM, pp 171–182
Leino KRM, Wallenburg A (2008) Class-local object invariants. In: Proceeding of the 1st annual India software engineering conference, ISEC 2008, Hyderabad, India, Feb 19–22, 2008, pp 57–66
Meyer, B.: Object-oriented software construction, 2nd edn. Prentice Hall, Upper Saddle River (1997)
Müller, P.; Poetzsch-Heffter, A.; Leavens, G.T.: Modular invariants for layered object structures. Sci Comput Program 62(3), 253–286 (2006)
Mehnert H, Sieczkowski F, Birkedal L, Sestoft P (2012) Formalized verification of snapshotable trees: separation and sharing. In: 4th International conference on verified software: theories, tools, experiments, VSTTE 2012, Philadelphia, PA, USA, Jan 28–29, 2012. Proceedings, pp 179–195
Müller P (2002) Modular specification and verification of object-oriented programs, volume 2262 of lecture notes in computer science. Springer
Documentation of Systems.Collections.Generic.Dictionary. https://msdn.microsoft.com/en-us/library/xfhwa508.aspx. Last access Feb 2016
Documentation of Systems.Collections.Generic.List.Enumerator. https://msdn.microsoft.com/en-us/library/x854yt9s.aspx. Last access Feb 2016
Nanevski A, Morrisett G, Shinnar A, Govereau P, Birkedal L (2008) Ynot: dependent types for imperative programs. In: Proceeding of the 13th ACM SIGPLAN international conference on Functional programming, ICFP 2008, Victoria, BC, Canada, Sept 20–28, 2008. ACM, pp 229–240
Paulin-Mohring C (2011) Introduction to the Coq proof-assistant for practical software verification. In: Tools for practical software verification, LASER, international summer school 2011, Elba Island, Italy, revised tutorial lectures, volume 7682 of lecture notes in computer science. Springer, pp 45–95
Parkinson MJ and Bierman GM (2008) Separation logic, abstraction and inheritance. In: Proceedings of the 35th ACM SIGPLAN-SIGACT symposium on principles of programming languages, POPL 2008, San Francisco, California, USA, Jan 7–12, 2008. ACM, pp 75–86
Polikarpova N, Furia CA, Meyer B (2010) Specifying reusable components. In: Proceedings of the 3rd international conference on verified software: theories, tools, and experiments (VSTTE’10), volume 6217 of lecture notes in computer science. Springer, pp 127–141
Polikarpova N (2014) Specified and verified reusable components. Ph.D. thesis, ETH Zurich
Polikarpova N (2015) EiffelBase2 (repository of verified code). http://dx.doi.org/10.5281/zenodo.16520
Pek E, Qiu X, Madhusudan P (2014) Natural proofs for data structure manipulation in C using separation logic. In: ACM SIGPLAN conference on programming language design and implementation, PLDI ’14, Edinburgh, UK June 09–11, 2014, pp 46
Polikarpova N, Tschannen J, Furia CA (June 2015) A fully verified container library. In: Proceedings of the 20th international symposium on formal methods (FM), volume 9109 of lecture notes in computer science. Springer, pp 414–434
Polikarpova N, Tschannen J, Furia CA, Meyer B (2014) Flexible invariants through semantic collaboration. In: FM 2014: formal methods—19th international symposium, Singapore, May 12–16, 2014. Proceedings, pp 514–530
Piskac R, Wies T, Zufferey D (2014) Automating separation logic with trees and data. In: 26th international conference on computer aided verification, CAV 2014, Held as part of the Vienna Summer of Logic, VSL 2014, Vienna, Austria, July 18–22, 2014. Proceedings, volume 8559 of lecture notes in computer science. Springer, pp 711–728
Régis-Gianas Y, Pottier F (2008) A Hoare logic for call-by-value functional programs. In: 9th international conference on mathematics of program construction, MPC 2008, Marseille, France, July 15–18, 2008. Proceedings, volume 5133 of lecture notes in computer science. Springer, pp 305–335
Sagiv, S.; Reps, T.W.; Wilhelm, R.: Parametric shape analysis via 3-valued logic. ACM Trans Program Lang Syst 24(3), 217–298 (2002)
Suter P, Steiger R, Kuncak V (2011) Sets with cardinality constraints in satisfiability modulo theories. In: 12th international conference on verification, model checking, and abstract interpretation, VMCAI 2011, Austin, TX, USA, Jan 23–25, 2011. Proceedings, volume 6538 of lecture notes in computer science. Springer, pp 403–418
Tschannen J, Furia CA, Nordio M, Polikarpova N (2015) AutoProof: Auto-active functional verification of object-oriented programs. In: Proceedings of the 21st international conference on tools and algorithms for the construction and analysis of systems (TACAS), volume 9035 of lecture notes in computer science. Springer, pp 566–580
Verifast example gallery. http://people.cs.kuleuven.be/~bart.jacobs/verifast/examples/. Last access Feb 2016
Vazou N, Seidel EL, Jhala R (2014) LiquidHaskell: experience with refinement types in the real world. In: Proceedings of the 2014 ACM SIGPLAN symposium on haskell, Haskell’14, New York, NY, USA. ACM, pp 39–51
Weide B, Edwards S, Heym WD, Long T, and Ogden W (April 1996) Characterizing observability and controllability of software components. In: Proceedings fourth international conference on software reuse, 1996, pp 62–71
Why3 example gallery. http://toccata.lri.fr/gallery/why3.en.html. Last access Feb 2016.
Wies T, Muñiz M, Kuncak V (2011) An efficient decision procedure for imperative tree data structures. In: Automated deduction—CADE-23—23rd international conference on automated deduction, Wroclaw, Poland, July 31 Aug 5, 2011. Proceedings, volume 6803 of lecture notes in computer science. Springer, pp 476–491
Wies T, Muñiz M, Kuncak V (2012) Deciding functional lists with sublist sets. In: 4th international conference on verified software: theories, tools, experiments, VSTTE 2012, Philadelphia, PA, USA, Jan 28–29, 2012. Proceedings, volume 7152 of lecture notes in computer science. Springer, pp 66–81
Xi H, Pfenning F (1999) Dependent types in practical programming. In: Proceedings of the 26th ACM SIGPLAN-SIGACT symposium on principles of programming languages, POPL’99, New York, NY, USA. ACM, pp 214–227
Yang H, Lee O, Berdine J, Calcagno C, Cook B, Distefano D, O’Hearn PW (2008) Scalable shape analysis for systems code. In: 20th international conference Computer Aided Verification, CAV 2008, Princeton, NJ, USA, July 7–14, 2008. Proceedings, volume 5123 of lecture notes in computer science. Springer, pp 385–398
Zee K, Kuncak V, Rinard MC (2008) Full functional verification of linked data structures. In: Proceedings of the ACM SIGPLAN 2008 conference on programming language design and implementation, Tucson, AZ, USA, June 7–13, 2008, pp 349–361
Author information
Authors and Affiliations
Corresponding author
Additional information
Frank de Boer, Nikolaj Bjorner, and Andrew Butterfield
Work mainly done while the authors were affiliated with ETH Zurich, Switzerland. A preliminary version appeared in the 20th International Symposium on Formal Methods in 2015 [PTF15].
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Polikarpova, N., Tschannen, J. & Furia, C.A. A fully verified container library. Form Asp Comp 30, 495–523 (2018). https://doi.org/10.1007/s00165-017-0435-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00165-017-0435-1