Automated Benchmarking of Incremental SAT and QBF Solvers

  • Uwe Egly
  • Florian LonsingEmail author
  • Johannes Oetsch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9450)


Incremental SAT and QBF solving potentially yields improvements when sequences of related formulas are solved. An incremental application is usually tailored towards some specific solver and decomposes a problem into incremental solver calls. This hinders the independent comparison of different solvers, particularly when the application program is not available. As a remedy, we present an approach to automated benchmarking of incremental SAT and QBF solvers. Given a collection of formulas in (Q)DIMACS format generated incrementally by an application program, our approach automatically translates the formulas into instructions to import and solve a formula by an incremental SAT/QBF solver. The result of the translation is a program which replays the incremental solver calls and thus allows to evaluate incremental solvers independently from the application program. We illustrate our approach by different hardware verification problems for SAT and QBF solvers.


Model Checker Application Program Truth Assignment Selector Variable Nest Level 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Audemard, G., Lagniez, J.-M., Simon, L.: Improving glucose for incremental SAT solving with assumptions: application to MUS extraction. In: Järvisalo, M., Van Gelder, A. (eds.) SAT 2013. LNCS, vol. 7962, pp. 309–317. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  2. 2.
    Biere, A.: PicoSAT essentials. JSAT 4(2–4), 75–97 (2008)zbMATHGoogle Scholar
  3. 3.
    Biere, A.: Lingeling essentials, a tutorial on design and implementation aspects of the the SAT solver lingeling. In: Berre, D.L. (ed.) Pragmatics of SAT (POS) Workshop. EPiC Series, vol. 27, p. 88. EasyChair (2014)Google Scholar
  4. 4.
    Bloem, R., Egly, U., Klampfl, P., Könighofer, R., Lonsing, F.: SAT-Based methods for circuit synthesis. In: FMCAD, pp. 31–34. IEEE (2014)Google Scholar
  5. 5.
    Eén, N., Sörensson, N.: An extensible SAT-solver. In: Giunchiglia, E., Tacchella, A. (eds.) SAT 2003. LNCS, vol. 2919, pp. 502–518. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  6. 6.
    Eén, N., Sörensson, N.: Temporal induction by incremental SAT solving. Electr. Notes Theor. Comput. Sci. 89(4), 543–560 (2003)CrossRefGoogle Scholar
  7. 7.
    Egly, U., Kronegger, M., Lonsing, F., Pfandler, A.: Conformant planning as a case study of incremental QBF solving. In: Aranda-Corral, G.A., Calmet, J., Martín-Mateos, F.J. (eds.) AISC 2014. LNCS, vol. 8884, pp. 120–131. Springer, Heidelberg (2014) Google Scholar
  8. 8.
    Egly, U., Lonsing, F., Oetsch, J.: Automated benchmarking of incremental SAT and QBF solvers. CoRR abs/1506.08563 (2015)., LPAR 2015 proceedings version (short paper) with appendix
  9. 9.
    Giunchiglia, E., Marin, P., Narizzano, M.: QuBE7.0. JSAT 7(2–3), 83–88 (2010)Google Scholar
  10. 10.
    Lagniez, J.-M., Biere, A.: Factoring out assumptions to speed up MUS extraction. In: Järvisalo, M., Van Gelder, A. (eds.) SAT 2013. LNCS, vol. 7962, pp. 276–292. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  11. 11.
    Lonsing, F., Egly, U.: Incremental QBF solving. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 514–530. Springer, Heidelberg (2014) Google Scholar
  12. 12.
    Marin, P., Miller, C., Lewis, M.D.T., Becker, B.: Verification of partial designs using incremental QBF solving. In: Rosenstiel, W., Thiele, L. (ed.) DATE, pp. 623–628. IEEE (2012)Google Scholar
  13. 13.
    Miller, C., Marin, P., Becker, B.: Verification of partial designs using incremental QBF. AI Commun. 28(2), 283–307 (2015)MathSciNetGoogle Scholar
  14. 14.
    Nadel, A., Ryvchin, V., Strichman, O.: Ultimately incremental SAT. In: Sinz, C., Egly, U. (eds.) SAT 2014. LNCS, vol. 8561, pp. 206–218. Springer, Heidelberg (2014) Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Institute of Information Systems 184/3Vienna University of TechnologyViennaAustria

Personalised recommendations