Skip to main content

CNFgen: A Generator of Crafted Benchmarks

  • Conference paper
  • First Online:
Theory and Applications of Satisfiability Testing – SAT 2017 (SAT 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10491))

Abstract

We present CNFgen, a generator of combinatorial benchmarks in DIMACS and OPB format. The proof complexity literature is a rich source not only of hard instances but also of instances that are theoretically easy but “extremal” in different ways, and therefore of potential interest in the context of SAT solving. Since most of these formulas appear not to be very well known in the SAT community, however, we propose CNFgen as a resource to make them readily available for solver development and evaluation. Many formulas studied in proof complexity are based on graphs, and CNFgen is also able to generate, parse and do basic manipulation of such objects. Furthermore, it includes a library cnfformula giving access to the functionality of CNFgen to Python programs.

Webpage: https://massimolauria.github.io/cnfgen/.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alekhnovich, M., Ben-Sasson, E., Alexander, A., Razborov, A.A., Wigderson, A.: Space complexity in propositional calculus. SIAM J. Comput. 31(4), 1184–1211 (2002). Preliminary version in STOC ’00

    Article  MathSciNet  MATH  Google Scholar 

  2. Alekhnovich, M., Johannsen, J., Pitassi, T., Urquhart, A.: An exponential separation between regular and general resolution. Theor. Comput. 3(5), 81–102 (2007). Preliminary version in STOC ’02

    Article  MathSciNet  MATH  Google Scholar 

  3. Alekhnovich, M., Alexander, A. Razborov, A.A.: Lower bounds for polynomial calculus: Non-binomial case. In: Proceedings of the Steklov Institute of Mathematics, 242, 18–35 (2003). http://people.cs.uchicago.edu/~razborov/files/misha.pdf. Preliminary version in FOCS ’01

  4. Balyo, T., Marijn, J., Heule, H., Järvisalo, M.: Proceedings of SAT competition 2016: Solver and benchmark descriptions. Technical report B-2016-1, University of Helsinki (2016). http://hdl.handle.net/10138/164630

  5. Roberto, J., Bayardo, Jr., Schrag, R.: Using CSP look-back techniques to solve real-world SAT instances. In Proceedings of the 14th National Conference on Artificial Intelligence (AAAI 1997), pp. 203–208 (1997)

    Google Scholar 

  6. Beame, P., Beck, C., Impagliazzo, R.: Time-space tradeoffs in resolution: Superpolynomial lower bounds for superlinear space. In: Proceedings of the 44th Annual ACM Symposium on Theory of Computing (STOC 2012), pp. 213–232 (2012)

    Google Scholar 

  7. Beck, C., Nordström, J., Tang, B.: Some trade-off results for polynomial calculus. In: Proceedings of the 45th Annual ACM Symposium on Theory of Computing (STOC 2013), pp. 813–822 (2013)

    Google Scholar 

  8. Ben-Sasson, E., Nordström, J.: Short proofs may be spacious: An optimal separation of space and length in resolution. In: Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2008), pp. 709–718 (2008)

    Google Scholar 

  9. Ben-Sasson, E., Nordström, J.: Understanding space in proof complexity: Separations and trade-offs via substitutions. In: Proceedings of the 2nd Symposium on Innovations in Computer Science (ICS 2011), pp. 401–416 (2011)

    Google Scholar 

  10. Beyersdorff, O., Galesi, N., Lauria, M.: Parameterized complexity of DPLL search procedures. ACM Trans. Comput. Logic 14(3), 20:1–20:21 (2013). Preliminary version in SAT ’11

    Article  MathSciNet  MATH  Google Scholar 

  11. Beyersdorff, O., Galesi, N., Lauria, M., Razborov, A.A.: Parameterized bounded-depth Frege is not optimal. ACM Trans. Comput. Theor. 4, 7:1–7:16 (2012). Preliminary version in ICALP ’11

    Article  MATH  Google Scholar 

  12. Blake, A.: Canonical expressions in boolean algebra. PhD thesis, University of Chicago (1937)

    Google Scholar 

  13. Bonet, M.L., Galesi, N.: Optimality of size-width tradeoffs for resolution. Comput. Complex. 10(4), 261–276 (2001). Preliminary version in FOCS ’99

    Article  MathSciNet  MATH  Google Scholar 

  14. Brickenstein, M., Dreyer, A.: PolyBoRi: A framework for Gröbner-basis computations with Boolean polynomials. J. Symb. Comput. 44(9), 1326–1345 (2009)

    Article  MATH  Google Scholar 

  15. Brickenstein, M., Dreyer, A., Greuel, G.-M., Wedler, M., Wienand, O.: New developments in the theory of Gröbner bases and applications to formal verification. J. Pure Appl. Algebr. 213(8), 1612–1635 (2009)

    Article  MATH  Google Scholar 

  16. Buss, S.R., Grigoriev, D., Impagliazzo, R., Pitassi, T.: Linear gaps between degrees for the polynomial calculus modulo distinct primes. J. Comput. Syst. Sci. 62(2), 267–289 (2001). Preliminary version in CCC ’99

    Article  MathSciNet  MATH  Google Scholar 

  17. Buss, S.R., Kołodziejczyk, L.: Small stone in pool. Logic. Method. Comput. Sci. 10, 10:16–16:22 (2014)

    MathSciNet  MATH  Google Scholar 

  18. Chai, D., Kuehlmann, A.: A fast pseudo-Boolean constraint solver. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 24(3), 305–317 (2005). Preliminary version in DAC ’03

    Article  Google Scholar 

  19. Chvátal, V., Szemerédi, E.: Many hard examples for resolution. J. ACM 35(4), 759–768 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  20. Clegg, M., Edmonds, J., Impagliazzo, R.: Using the Groebner basis algorithm to find proofs of unsatisfiability. In: Proceedings of the 28th Annual ACM Symposium on Theory of Computing (STOC 1996), pp. 174–183 (1996)

    Google Scholar 

  21. Cook, S.A.: The complexity of theorem-proving procedures. In: Proceedings of the 3rd Annual ACM Symposium on Theory of Computing (STOC 1971), pp. 151–158 (1971)

    Google Scholar 

  22. Cook, S.A., Reckhow, R.: The relative efficiency of propositional proof systems. J. Symbol. Logic 44(1), 36–50 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  23. Cook, W.: Collette Rene Coullard, and György Turán. On the complexity of cutting-plane proofs. Discr. Appl. Math. 18(1), 25–38 (1987)

    Article  Google Scholar 

  24. Dixon, H.E., Ginsberg, M.L., Hofer, D.K., Luks, E.M., Parkes, A.J.: Generalizing Boolean satisfiability III: Implementation. J. Artif. Intell. Res. 23, 441–531 (2005)

    MATH  Google Scholar 

  25. Elffers, J., Giráldez-Crú, J., Nordström, J., Vinyals, M.: Using combinatorial benchmarks to probe the reasoning power of pseudo-Boolean solvers (2017, Submitted)

    Google Scholar 

  26. Elffers, J., Nordström, J., Simon, L., Sakallah, K.A.: Seeking practical CDCL insights from theoretical SAT benchmarks. In: Presentation at the Pragmatics of SAT 2016 workshop (2016). http://www.csc.kth.se/~jakobn/research/TalkPoS16.pdf

  27. Galesi, N., Lauria, M.: Optimality of size-degree trade-offs for polynomial calculus. ACM Trans. Comput. Logic 12, 4:1–4:22 (2010)

    Article  MATH  Google Scholar 

  28. Hagberg, A.A., Schult, D., Swart, P.S.: Exploring network structure, dynamics, and function using NetworkX. In: Proceedings of the 7th Python in Science Conference (SciPy2008), Pasadena, CA USA, pp. 11–15 (2008)

    Google Scholar 

  29. Haken, A.: The intractability of resolution. Theor. Comput. Sci. 39(2–3), 297–308 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  30. Heule, M., van Maaren, H.: Aligning CNF- and equivalence-reasoning. In: Hoos, H.H., Mitchell, D.G. (eds.) SAT 2004. LNCS, vol. 3542, pp. 145–156. Springer, Heidelberg (2005). doi:10.1007/11527695_12

    Chapter  Google Scholar 

  31. Himsolt, M.: GML: A portable graph file format. Technical report, Universität of Passau (1996)

    Google Scholar 

  32. Huynh, T., Nordström, J.: On the virtue of succinct proofs: Amplifying communication complexity hardness to time-space trade-offs in proof complexity (Extended abstract). In: Proceedings of the 44th Annual ACM Symposium on Theory of Computing (STOC 2012), pp. 233–248 (2012)

    Google Scholar 

  33. Impagliazzo, R., Paturi, R.: On the complexity of \(k\)-SAT. J. Comput. Syst. Sci. 62(2), 367–375 (2001). Preliminary version in CCC ’99

    Article  MathSciNet  MATH  Google Scholar 

  34. Järvisalo, M., Matsliah, A., Nordström, J., Živný, S.: Relating proof complexity measures and practical hardness of SAT. In: Milano, M. (ed.) CP 2012. LNCS, pp. 316–331. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33558-7_25

    Chapter  Google Scholar 

  35. Le Berre, D., Parrain, A.: The Sat4j library, release 2.2. J. Satisf. Boolean Model. Comput. 7, 59–64 (2010)

    Google Scholar 

  36. Markström, K.: Locality and hard SAT-instances. J. Satisf. Boolean Model. Comput. 2(1–4), 221–227 (2006)

    MATH  Google Scholar 

  37. Marques-Silva, J.P., Sakallah, K.A.: GRASP: A search algorithm for propositional satisfiability. IEEE Trans. Comput. 48(5), 506–521 (1999). Preliminary version in ICCAD ’96

    Article  MathSciNet  Google Scholar 

  38. Massey, B.: DIMACS graph format (2001). http://prolland.free.fr/works/research/dsat/dimacs.html. Accessed 11 Feb 2016

  39. Mikša, M., Nordström, J.: Long proofs of (seemingly) simple formulas. In: Sinz, C., Egly, U. (eds.) SAT 2014. LNCS, vol. 8561, pp. 121–137. Springer, Cham (2014). doi:10.1007/978-3-319-09284-3_10

    Google Scholar 

  40. Mikša, M., Nordström, J.: A generalized method for proving polynomial calculus degree lower bounds. In: Proceedings of the 30th Annual Computational Complexity Conference (CCC 2015). Leibniz International Proceedings in Informatics (LIPIcs), vol. 33, pp. 467–487 (2015)

    Google Scholar 

  41. Moskewicz, M.W., Madigan, C.F., Zhao, Y., Zhang, L., Chaff, M.S.: Engineering an efficient SAT solver. In: Proceedings of the 38th Design Automation Conference (DAC 2001), pp. 530–535 (2001)

    Google Scholar 

  42. Nordström, J.: On the interplay between proof complexity and SAT solving. ACM SIGLOG News 2(3), 19–44 (2015)

    Google Scholar 

  43. Pseudo-Boolean competition (2016). http://www.cril.univ-artois.fr/PB16/

  44. Razborov, A.A.: Resolution lower bounds for perfect matching principles. J. Comput. Syst. Sci. 69(1), 3–27 (2004). Preliminary version in CCC ’02

    Article  MathSciNet  MATH  Google Scholar 

  45. Razborov, A.A.: A new kind of tradeoffs in propositional proof complexity. J. ACM 63, 16:1–16:14 (2016)

    Article  MathSciNet  Google Scholar 

  46. AT and T Research: Dot Language. http://www.graphviz.org/content/dot-language. Accessed 11 Feb 2016

  47. Sheini, H.M., Sakallah, K.A.: Pueblo: a hybrid pseudo-Boolean SAT solver. J. Satisf. Boolean Model. Comput. 2(1–4), 165–189 (2006). Preliminary version in DATE ’05

    MATH  Google Scholar 

  48. Soos, M., Nohl, K., Castelluccia, C.: Extending SAT solvers to cryptographic problems. In: Kullmann, O. (ed.) SAT 2009. LNCS, vol. 5584, pp. 244–257. Springer, Heidelberg (2009). doi:10.1007/978-3-642-02777-2_24

    Chapter  Google Scholar 

  49. Stålmarck, G.: Short resolution proofs for a sequence of tricky formulas. Acta Inform. 33(3), 277–280 (1996)

    Article  MathSciNet  Google Scholar 

  50. Urquhart, A.: Hard examples for resolution. J. ACM 34(1), 209–219 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  51. Gelder, A., Spence, I.: Zero-one designs produce small hard SAT instances. In: Strichman, O., Szeider, S. (eds.) SAT 2010. LNCS, vol. 6175, pp. 388–397. Springer, Heidelberg (2010). doi:10.1007/978-3-642-14186-7_37

    Chapter  Google Scholar 

  52. Vinyals, M., Elffers, J., Giráldez-Crú, J., Gocht, S., Nordström, J.: In between resolution and cutting planes: A study of proof systems for pseudo-Boolean SAT solving (2017, Submitted)

    Google Scholar 

Download references

Acknowledgments

The first author performed most of this work while at KTH Royal Institute of Technology. The authors were funded by the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007–2013) / ERC grant agreement no. 279611 as well as by Swedish Research Council grant 621-2012-5645. The first author was also supported by the European Research Council under the European Union’s Horizon 2020 Research and Innovation Programme / ERC grant agreement no. 648276 AUTAR.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Massimo Lauria .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Lauria, M., Elffers, J., Nordström, J., Vinyals, M. (2017). CNFgen: A Generator of Crafted Benchmarks. In: Gaspers, S., Walsh, T. (eds) Theory and Applications of Satisfiability Testing – SAT 2017. SAT 2017. Lecture Notes in Computer Science(), vol 10491. Springer, Cham. https://doi.org/10.1007/978-3-319-66263-3_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66263-3_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66262-6

  • Online ISBN: 978-3-319-66263-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics