Efficient Certified Resolution Proof Checking

  • Luís Cruz-FilipeEmail author
  • Joao Marques-Silva
  • Peter Schneider-Kamp
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10205)


We present a novel propositional proof tracing format that eliminates complex processing, thus enabling efficient (formal) proof checking. The benefits of this format are demonstrated by implementing a proof checker in C, which outperforms a state-of-the-art checker by two orders of magnitude. We then formalize the theory underlying propositional proof checking in Coq, and extract a correct-by-construction proof checker for our format from the formalization. An empirical evaluation using 280 unsatisfiable instances from the 2015 and 2016 SAT competitions shows that this certified checker usually performs comparably to a state-of-the-art non-certified proof checker. Using this format, we formally verify the recent 200 TB proof of the Boolean Pythagorean Triples conjecture.



The authors thank M. Heule for comments on an early draft of this paper. This work was partially supported by the Danish Council for Independent Research, Natural Sciences, grant DFF-1323-00247. Computational resources on the Abacus 2.0 supercomputer were provided by the DeIC National HPC Centre at the University of Southern Denmark.


  1. 1.
    Alkassar, E., Böhme, S., Mehlhorn, K., Rizkallah, C.: A framework for the verification of certifying computations. J. Autom. Reason. 52(3), 241–273 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Armand, M., Faure, G., Grégoire, B., Keller, C., Théry, L., Werner, B.: A modular integration of SAT/SMT solvers to Coq through proof witnesses. In: Jouannaud, J.-P., Shao, Z. (eds.) CPP 2011. LNCS, vol. 7086, pp. 135–150. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-25379-9_12 CrossRefGoogle Scholar
  3. 3.
    Beame, P., Kautz, H.A., Sabharwal, A.: Towards understanding and harnessing the potential of clause learning. J. Artif. Intell. Res. (JAIR) 22, 319–351 (2004)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Bertot, Y., Castéran, P.: Interactive Theorem Proving and Program Development. Texts in Theoretical Computer Science. Springer, Heidelberg (2004)CrossRefzbMATHGoogle Scholar
  5. 5.
    Biere, A.: PicoSAT essentials. JSAT 4(2–4), 75–97 (2008)zbMATHGoogle Scholar
  6. 6.
    Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press, Amsterdam (2009)zbMATHGoogle Scholar
  7. 7.
    Blanchette, J.C., Fleury, M., Weidenbach, C.: A verified SAT solver framework with learn, forget, restart, and incrementality. In: Olivetti, N., Tiwari, A. (eds.) IJCAR 2016. LNCS (LNAI), vol. 9706, pp. 25–44. Springer, Cham (2016). doi: 10.1007/978-3-319-40229-1_4 Google Scholar
  8. 8.
    Blum, M., Kannan, S.: Designing programs that check their work. In: STOC, pp. 86–97 (1989)Google Scholar
  9. 9.
    Bras, R.L., Gomes, C.P., Selman, B.: On the Erdős discrepancy problem. In: CP, pp. 440–448 (2014)Google Scholar
  10. 10.
    Coquand, T., Huet, G.P.: The calculus of constructions. Inf. Comput. 76(2/3), 95–120 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Cruz-Filipe, L., Heule, M., Hunt, W., Kaufmann, M., Schneider-Kamp, P.: Efficient certified RAT verification. CoRR, abs/1610.06984 (2016)Google Scholar
  12. 12.
    Cruz-Filipe, L., Schneider-Kamp, P.: Checking the Boolean Pythagorean Triples conjecture.
  13. 13.
    Cruz-Filipe, L., Schneider-Kamp, P.: Grit format, formalization, and checkers. Source codes also available from:
  14. 14.
    Cruz-Filipe, L., Schneider-Kamp, P.: Formalizing size-optimal sorting networks: extracting a certified proof checker. In: Urban, C., Zhang, X. (eds.) ITP 2015. LNCS, vol. 9236, pp. 154–169. Springer, Cham (2015). doi: 10.1007/978-3-319-22102-1_10 Google Scholar
  15. 15.
    Cruz-Filipe, L., Schneider-Kamp, P.: Optimizing a certified proof checker for a large-scale computer-generated proof. In: Kerber, M., Carette, J., Kaliszyk, C., Rabe, F., Sorge, V. (eds.) CICM 2015. LNCS (LNAI), vol. 9150, pp. 55–70. Springer, Cham (2015). doi: 10.1007/978-3-319-20615-8_4 CrossRefGoogle Scholar
  16. 16.
    Darbari, A., Fischer, B., Marques-Silva, J.: Formalizing a SAT proof checker in Coq. In: First Coq Workshop (2009)Google Scholar
  17. 17.
    Darbari, A., Fischer, B., Marques-Silva, J.: Industrial-strength certified SAT solving through verified SAT proof checking. In: Cavalcanti, A., Deharbe, D., Gaudel, M.-C., Woodcock, J. (eds.) ICTAC 2010. LNCS, vol. 6255, pp. 260–274. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-14808-8_18 CrossRefGoogle Scholar
  18. 18.
    Goldberg, E.I., Novikov, Y.: Verification of proofs of unsatisfiability for CNF formulas. In: DATE, pp. 10886–10891 (2003)Google Scholar
  19. 19.
    Heule, M.: The DRAT format and DRAT-trim checker. CoRR, abs/1610.06229 (2016).
  20. 20.
    Heule, M., Biere, A.: Proofs for satisfiability problems. In: All About Proofs, Proofs for All (APPA), July 2014.
  21. 21.
    Heule, M., Hunt Jr., W.A., Wetzler, N.: Trimming while checking clausal proofs. In: FMCAD, pp. 181–188 (2013)Google Scholar
  22. 22.
    Heule, M.J.H., Hunt, W.A., Wetzler, N.: Verifying refutations with extended resolution. In: Bonacina, M.P. (ed.) CADE 2013. LNCS (LNAI), vol. 7898, pp. 345–359. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38574-2_24 CrossRefGoogle Scholar
  23. 23.
    Heule, M., Hunt Jr., W.A., Wetzler, N.: Bridging the gap between easy generation and efficient verification of unsatisfiability proofs. Softw. Test. Verif. Reliab. 24(8), 593–607 (2014)CrossRefGoogle Scholar
  24. 24.
    Heule, M.J.H., Hunt, W.A., Wetzler, N.: Expressing symmetry breaking in DRAT proofs. In: Felty, A.P., Middeldorp, A. (eds.) CADE 2015. LNCS (LNAI), vol. 9195, pp. 591–606. Springer, Cham (2015). doi: 10.1007/978-3-319-21401-6_40 CrossRefGoogle Scholar
  25. 25.
    Heule, M.J.H., Kullmann, O., Marek, V.W.: Solving and verifying the boolean pythagorean triples problem via cube-and-conquer. In: Creignou, N., Le Berre, D. (eds.) SAT 2016. LNCS, vol. 9710, pp. 228–245. Springer, Cham (2016). doi: 10.1007/978-3-319-40970-2_15 Google Scholar
  26. 26.
    Heule, M., Seidl, M., Biere, A.: Efficient extraction of skolem functions from QRAT proofs. In: FMCAD, pp. 107–114 (2014)Google Scholar
  27. 27.
    Jussila, T., Biere, A., Sinz, C., Kröning, D., Wintersteiger, C.M.: A first step towards a unified proof checker for QBF. In: Marques-Silva, J., Sakallah, K.A. (eds.) SAT 2007. LNCS, vol. 4501, pp. 201–214. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-72788-0_21 CrossRefGoogle Scholar
  28. 28.
    Jussila, T., Sinz, C., Biere, A.: Extended resolution proofs for symbolic SAT solving with quantification. In: Biere, A., Gomes, C.P. (eds.) SAT 2006. LNCS, vol. 4121, pp. 54–60. Springer, Heidelberg (2006). doi: 10.1007/11814948_8 CrossRefGoogle Scholar
  29. 29.
    Konev, B., Lisitsa, A.: Computer-aided proof of Erdős discrepancy properties. CoRR, abs/1405.3097 (2014)Google Scholar
  30. 30.
    Konev, B., Lisitsa, A.: A SAT attack on the Erdős discrepancy conjecture. In: Sinz, C., Egly, U. (eds.) SAT 2014. LNCS, vol. 8561, pp. 219–226. Springer, Cham (2014). doi: 10.1007/978-3-319-09284-3_17 Google Scholar
  31. 31.
    Konev, B., Lisitsa, A.: Computer-aided proof of Erdős discrepancy properties. Artif. Intell. 224, 103–118 (2015)CrossRefzbMATHGoogle Scholar
  32. 32.
    Lescuyer, S., Conchon, S.: Improving Coq propositional reasoning using a lazy CNF conversion scheme. In: Ghilardi, S., Sebastiani, R. (eds.) FroCoS 2009. LNCS (LNAI), vol. 5749, pp. 287–303. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-04222-5_18 CrossRefGoogle Scholar
  33. 33.
    Letouzey, P.: Extraction in Coq: an overview. In: Beckmann, A., Dimitracopoulos, C., Löwe, B. (eds.) CiE 2008. LNCS, vol. 5028, pp. 359–369. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-69407-6_39 CrossRefGoogle Scholar
  34. 34.
    Maric, F.: Formal verification of a modern SAT solver by shallow embedding into Isabelle/HOL. Theor. Comput. Sci. 411(50), 4333–4356 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Maric, F., Janicic, P.: Formalization of abstract state transition systems for SAT. LMCS 7(3:19), 1–37 (2011)MathSciNetzbMATHGoogle Scholar
  36. 36.
    McConnell, R.M., Mehlhorn, K., Näher, S., Schweitzer, P.: Certifying algorithms. Comput. Sci. Rev. 5(2), 119–161 (2011)CrossRefzbMATHGoogle Scholar
  37. 37.
    Shankar, N.: Trust and automation in verification tools. In: Cha, S.S., Choi, J.-Y., Kim, M., Lee, I., Viswanathan, M. (eds.) ATVA 2008. LNCS, vol. 5311, pp. 4–17. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88387-6_3 CrossRefGoogle Scholar
  38. 38.
    Sinz, C., Biere, A.: Extended resolution proofs for conjoining BDDs. In: Grigoriev, D., Harrison, J., Hirsch, E.A. (eds.) CSR 2006. LNCS, vol. 3967, pp. 600–611. Springer, Heidelberg (2006). doi: 10.1007/11753728_60 CrossRefGoogle Scholar
  39. 39.
    Smith, D.R., Westfold, S.J.: Synthesis of satisfiability solvers. Technical report, Kestrel Institute (2008)Google Scholar
  40. 40.
    Van Gelder, A.: Verifying RUP proofs of propositional unsatisfiability. In: ISAIM (2008)Google Scholar
  41. 41.
    Gelder, A.: Improved conflict-clause minimization leads to improved propositional proof traces. In: Kullmann, O. (ed.) SAT 2009. LNCS, vol. 5584, pp. 141–146. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-02777-2_15 CrossRefGoogle Scholar
  42. 42.
    Van Gelder, A.: Producing and verifying extremely large propositional refutations - have your cake and eat it too. Ann. Math. Artif. Intell. 65(4), 329–372 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Weber, T., Amjad, H.: Efficiently checking propositional refutations in HOL theorem provers. J. Appl. Logic 7(1), 26–40 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Wetzler, N., Heule, M.J.H., Hunt, W.A.: Mechanical verification of SAT refutations with extended resolution. In: Blazy, S., Paulin-Mohring, C., Pichardie, D. (eds.) ITP 2013. LNCS, vol. 7998, pp. 229–244. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-39634-2_18 CrossRefGoogle Scholar
  45. 45.
    Wetzler, N., Heule, M.J.H., Hunt, W.A.: DRAT-trim: efficient checking and trimming using expressive clausal proofs. In: Sinz, C., Egly, U. (eds.) SAT 2014. LNCS, vol. 8561, pp. 422–429. Springer, Cham (2014). doi: 10.1007/978-3-319-09284-3_31 Google Scholar
  46. 46.
    Wetzler, N.D.: Efficient, mechanically-verified validation of satisfiability solvers. Ph.D. thesis, The University of Texas at Austin (2015)Google Scholar
  47. 47.
    Zhang, L., Malik, S.: Validating SAT solvers using an independent resolution-based checker: practical implementations and other applications. In: DATE, pp. 10880–10885 (2003)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Luís Cruz-Filipe
    • 1
    Email author
  • Joao Marques-Silva
    • 2
  • Peter Schneider-Kamp
    • 1
  1. 1.Department of Mathematics and Computer ScienceUniversity of Southern DenmarkOdenseDenmark
  2. 2.LaSIGE, Faculty of ScienceUniversity of LisbonLisbonPortugal

Personalised recommendations