Skip to main content
SpringerLink
Log in

A Time Leap Challenge for SAT-Solving

  • Conference paper
  • First Online:
Principles and Practice of Constraint Programming (CP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 12333))

Abstract

We compare the impact of hardware advancement and algorithm advancement for SAT-solving over the last two decades. In particular, we compare 20-year-old SAT-solvers on new computer hardware with modern SAT-solvers on 20-year-old hardware. Our findings show that the progress on the algorithmic side has at least as much impact as the progress on the hardware side.

The work has been supported by the Austrian Science Fund (FWF), Grants Y698, 32441, and 32830, and the Vienna Science and Technology Fund, Grant WWTF ICT19-065. Markus Hecher is also affiliated with the University of Potsdam, Germany.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The Theorem Prover Museum is available online at https://theoremprover-museum.github.io/.

  2. 2.

    https://en.wikipedia.org/wiki/SourceForge.

  3. 3.

    See: http://archive.dimacs.rutgers.edu/pub/challenge/sat/benchmarks/.

  4. 4.

    See: https://www.cs.ubc.ca/~hoos/SATLIB/benchm.html.

  5. 5.

    The webpage http://www.satcompetition.org/ gives a broad overview on the results and details of the competitions since 2002.

  6. 6.

    See: https://www.cs.ubc.ca/~hoos/SATLIB/solvers.html.

  7. 7.

    See: https://github.com/roberto-bayardo/relsat.

  8. 8.

    See: http://minisat.se/MiniSat.html.

References

  1. Audemard, G., Paulevé, L., Simon, L.: SAT heritage: a community-driven effort for archiving, building and running more than thousand SAT solvers. In: Pulina, L., Seidl, M. (eds.) SAT 2020. LNCS, vol. 12178, pp. 107–113. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-51825-7_8

    Chapter  Google Scholar 

  2. Audemard, G., Simon, L.: Predicting learnt clauses quality in modern SAT solvers. In: Boutilier, C. (ed.) IJCAI 2009, Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA, 11–17 July 2009, pp. 399–404 (2009)

    Google Scholar 

  3. Audemard, G., Simon, L.: Glucose 2.1: aggressive - but reactive - clause database management, dynamic restarts. In: Berre, D.L., Gelder, A.V. (eds.) Proceedings of 3rd International Workshop of Pragmatics of SAT (PoS 2012), Trento, Italie (2012)

    Google Scholar 

  4. Audemard, G., Simon, L.: Glucose in the SAT Race 2019. In: Heule, M.J., Järvisalo, M., Suda, M. (eds.) Proceedings of SAT Race 2019: Solver and Benchmark Descriptions. Department of Computer Science Report Series, vol. B-2019-1, pp. 19–20. University of Helsinki (2019)

    Google Scholar 

  5. Balyo, T., Biere, A., Iser, M., Sinz, C.: SAT race 2015. Artif. Intell. 241, 45–65 (2016). https://doi.org/10.1016/j.artint.2016.08.007

    Article  MathSciNet  MATH  Google Scholar 

  6. Bayardo, R.J., Schrag, R.C.: Using CSP look-back techniques to solve real-world sat instances. In: Kuipers, B., Webber, B. (eds.) Proceedings of the Fourteenth National Conference on Artificial Intelligence and Ninth Conference on Innovative Applications of Artificial Intelligence (AAAI 1997), pp. 203–208. The AAAI Press, Providence (1997)

    Google Scholar 

  7. Biere, A.: CaDiCaL, Lingeling, Plingeling, Treengeling, YalSAT entering the SAT competition 2017. In: Balyo, T., Heule, M., Järvisalo, M. (eds.) Proceedings of SAT Competition 2017 - Solver and Benchmark Descriptions. Department of Computer Science Series of Publications B, vol. B-2017-1, pp. 14–15. University of Helsinki (2017)

    Google Scholar 

  8. Biere, A.: CaDiCaL simplified satisfiability solver (2019). http://fmv.jku.at/cadical/

  9. Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press, Amsterdam (2009)

    Google Scholar 

  10. Bornebusch, F., Wille, R., Drechsler, R.: Towards lightweight satisfiability solvers for self-verification. In: Proceedings of the 7th International Symposium on Embedded Computing and System Design (ISED 2017), pp. 1–5, December 2017. https://doi.org/10.1109/ISED.2017.8303924

  11. Büning, H.K., Buro, M.: Report on a SAT competition. Bull. Eur. Assoc. Theoret. Comput. Sci. 49(1), 143–151 (1993)

    MATH  Google Scholar 

  12. Carter, J.: Debian 10 buster released (2019). https://www.debian.org/News/2019/20190706

  13. Chu, G., Harwood, A., Stuckey, P.: Cache conscious data structures for Boolean satisfiability solvers. J. Satisf. Boolean Model. Comput. 6, 99–120 (2009). https://doi.org/10.3233/SAT190064

    Article  MathSciNet  MATH  Google Scholar 

  14. Crawford, J.M., Auton, L.D.: Experimental results on the crossover point in satisfiability problems. In: Fikes, R., Lehnert, W. (eds.) Proceedings of the 11th National Conference on Artificial Intelligence (AAAI 1993), pp. 21–27. The AAAI Press, Washington, D.C. (1993)

    Google Scholar 

  15. Davis, M., Logemann, G., Loveland, D.: A machine program for theorem-proving. Commun. ACM 5(7), 394–397 (1962). https://doi.org/10.1145/368273.368557

    Article  MathSciNet  MATH  Google Scholar 

  16. Davis, M., Putnam, H.: A computing procedure for quantification theory. J. ACM 7(3), 201–215 (1960). https://doi.org/10.1145/321033.321034

    Article  MathSciNet  MATH  Google Scholar 

  17. Eén, N., Biere, A.: Effective preprocessing in SAT through variable and clause elimination. In: Bacchus, F., Walsh, T. (eds.) SAT 2005. LNCS, vol. 3569, pp. 61–75. Springer, Heidelberg (2005). https://doi.org/10.1007/11499107_5

    Chapter  MATH  Google Scholar 

  18. Eén, N., Sörensson, N.: An extensible SAT-solver. In: Giunchiglia, E., Tacchella, A. (eds.) SAT 2003. LNCS, vol. 2919, pp. 502–518. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24605-3_37

    Chapter  Google Scholar 

  19. Eén, N., Sörensson, N.: MiniSAT (2008). http://minisat.se/

  20. Elffers, J., Nordström, J.: Divide and conquer: towards faster pseudo-Boolean solving. In: Lang, J. (ed.) Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 1291–1299. International Joint Conferences on Artificial Intelligence Organization, July 2018. https://doi.org/10.24963/ijcai.2018/180

  21. Fichte, J.K., Hecher, M., Szeider, S.: Analyzed benchmarks and raw data on experiments for time leap challenge for SAT-solving (dataset). Zenodo, July 2020. https://doi.org/10.5281/zenodo.3948480

  22. Fichte, J.K., Manthey, N., Stecklina, J., Schidler, A.: Towards faster reasoners by using transparent huge pages. CoRR abs/2004.14378 (2020)

    Google Scholar 

  23. Fichte, J.K., Manthey, N., Schidler, A., Stecklina, J.: Towards faster reasoners by using transparent huge pages. In: Simonis, H. (ed.) CP 2020. LNCS, vol. 12333, pp. 304–322. Springer, Heidelberg (2020)

    Google Scholar 

  24. Franco, J., Martin, J.: Chapter 1: A history of satisfiability. In: Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.) Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185, pp. 3–74. IOS Press, Amsterdam (2009). https://doi.org/10.3233/978-1-58603-929-5-3

    Chapter  Google Scholar 

  25. Freeman, J.W.: Improvements to propositional satisfiability search algorithms. Ph.D. thesis, Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, USA (1995)

    Google Scholar 

  26. Fu, Z., Mahajan, Y., Malik, S.: zchaff (2004). https://www.princeton.edu/~chaff/zchaff.html

  27. Gocht, S., Nordström, J., Yehudayoff, A.: On division versus saturation in pseudo-Boolean solving. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp. 1711–1718. International Joint Conferences on Artificial Intelligence Organization, July 2019. https://doi.org/10.24963/ijcai.2019/237

  28. Goldberg, E., Novikov, Y.: Berkmin (2003). http://eigold.tripod.com/BerkMin.html

  29. Gomes, C.P., Kautz, H., Sabharwal, A., Selman, B.: Chapter 2: Satisfiability solvers. In: Frank van Harmelen, V.L., Porter, B. (eds.) Handbook of Knowledge Representation. Foundations of Artificial Intelligence, vol. 3, pp. 89–134. Elsevier Science Publishers, North-Holland (2008). https://doi.org/10.1016/S1574-6526(07)03002-7

  30. Hennessy, J.L., Patterson, D.A.: Computer Architecture: A Quantitative Approach, 5th edn. Morgan Kaufmann, Burlington (2011)

    MATH  Google Scholar 

  31. Hoos, H.: Satlib - benchmark problems (2000). https://www.cs.ubc.ca/~hoos/SATLIB/benchm.html

  32. Hoos, H.: Satlib - solvers (2000). https://www.cs.ubc.ca/~hoos/SATLIB/solvers.html

  33. Hoos, H.H., Kaufmann, B., Schaub, T., Schneider, M.: Robust benchmark set selection for Boolean constraint solvers. In: Nicosia, G., Pardalos, P. (eds.) LION 2013. LNCS, vol. 7997, pp. 138–152. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-44973-4_16

    Chapter  Google Scholar 

  34. Iser, M., Balyo, T., Sinz, C.: Memory efficient parallel sat solving with inprocessing. In: Keefer, R. (ed.) Proceedings of the IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI 2019), pp. 64–70. IEEE Computer Society (2019)

    Google Scholar 

  35. Jackson, I., Schwarz, C., Morris, D.A.: Debian GNU/Linux installation guide: 2.1. supported hardware (2019). https://www.debian.org/releases/stable/i386/ch02s01.en.html#idm272

  36. Järvisalo, M., Berre, D.L., Roussel, O., Simon, L.: The international SAT solver competitions. AI Mag. 33(1) (2012). http://www.aaai.org/ojs/index.php/aimagazine/article/view/2395

  37. Järvisalo, M., Heule, M.J.H., Biere, A.: Inprocessing rules. In: Gramlich, B., Miller, D., Sattler, U. (eds.) IJCAR 2012. LNCS (LNAI), vol. 7364, pp. 355–370. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31365-3_28

    Chapter  Google Scholar 

  38. Johnson, D., Trick, M.: Satisfiability suggested format (1993). https://www.cs.ubc.ca/~hoos/SATLIB/Benchmarks/SAT/satformat.ps

  39. Bayardo Jr., R.J., Pehoushek, J.D.: Counting models using connected components. In: Kautz, H.A., Porter, B. (eds.) Proceedings of the 17th Conference on Artificial Intelligence (AAAI 2000). The AAAI Press, Austin (2000)

    Google Scholar 

  40. Kahle, B.: Internet archive (2020). https://archive.org/

  41. Katebi, H., Sakallah, K.A., Marques-Silva, J.P.: Empirical study of the anatomy of modern SAT solvers. In: Sakallah, K.A., Simon, L. (eds.) SAT 2011. LNCS, vol. 6695, pp. 343–356. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21581-0_27

    Chapter  MATH  Google Scholar 

  42. Knuth, D.E.: The Art of Computer Programming, Volume 4, Fascicle 6: Satisfiability. Addison-Wesley, Boston (2015)

    Google Scholar 

  43. Kochemazov, S., Zaikin, O., Kondratiev, V., Semenov, A.: MapleLCMDistChronoBT-DL, duplicate learnts heuristic-aided solvers at the SAT Race 2019. In: Heule, M.J., Järvisalo, M., Suda, M. (eds.) Proceedings of SAT Race 2019: Solver and Benchmark Descriptions. Department of Computer Science Report Series, vol. B-2019-1, pp. 24–24. University of Helsinki (2019)

    Google Scholar 

  44. Kohlhase, M.: The theorem prover museum - conserving the system heritage of automated reasoning. CoRR abs/1904.10414 (2019)

    Google Scholar 

  45. van der Kouwe, E., Andriesse, D., Bos, H., Giuffrida, C., Heiser, G.: Benchmarking crimes: an emerging threat in systems security. CoRR abs/1801.02381 (2018). http://arxiv.org/abs/1801.02381

  46. Lewis, M.D.T., Schubert, T., Becker, B.W.: Speedup techniques utilized in modern SAT solvers. In: Bacchus, F., Walsh, T. (eds.) SAT 2005. LNCS, vol. 3569, pp. 437–443. Springer, Heidelberg (2005). https://doi.org/10.1007/11499107_36

    Chapter  Google Scholar 

  47. Li, C.M., Anbulagan, A.: Heuristics based on unit propagation for satisfiability problems. In: Georgeff, M.P., Pollack, M.E. (eds.) Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI 1997), pp. 366–371. Morgan Kaufmann, Nagoya (1997)

    Google Scholar 

  48. Liang, J.H., Ganesh, V., Poupart, P., Czarnecki, K.: Learning rate based branching heuristic for SAT solvers. In: Creignou, N., Le Berre, D. (eds.) SAT 2016. LNCS, vol. 9710, pp. 123–140. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40970-2_9

    Chapter  MATH  Google Scholar 

  49. Luby, M., Sinclair, A., Zuckerman, D.: Optimal speedup of Las Vegas algorithms. Inf. Process. Lett. 47(4), 173–180 (1993)

    Article  MathSciNet  Google Scholar 

  50. Mahapatra, N.R., Venkatrao, B.: The processor-memory bottleneck: problems and solutions. XRDS 5(3es), 2–9 (1999)

    Article  Google Scholar 

  51. Marques-Silva, J.: Grasp (SAT solver). Pers. Commun. (2020)

    Google Scholar 

  52. Marques-Silva, J., Lynce, I., Malik, S.: Chapter 4: Conflict-driven clause learning SAT solvers. In: Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.) Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185, pp. 127–148. IOS Press, Amsterdam (2009)

    Google Scholar 

  53. Mitchell, D.G.: A SAT solver primer. In: Bulletin of the EATCS 85, The Logic in Computer Science Column, pp. 112–132 (2005)

    Google Scholar 

  54. Moore, G.E.: Cramming more components onto integrated circuits. Electronics 38(8), 114 ff. (1965)

    Google Scholar 

  55. Moskewicz, M.W., Madigan, C.F., Zhao, Y., Zhang, L., Malik, S.: Chaff: engineering an efficient SAT solver. In: Rabaey, J. (ed.) Proceedings of the 38th Annual Design Automation Conference (DAC 2001), pp. 530–535. Association for Computing Machinery, New York (2001)

    Google Scholar 

  56. Munroe, R.: xkcd2268 (2019). https://xkcd.com/2268/

  57. Nielsen, L.H.: Software citations now available in Zenodo (2019). https://blog.zenodo.org/2019/01/10/2019-01-10-asclepias/

  58. Nordström, J.: On the interplay between proof complexity and sat solving. ACM SIGLOG News 2(3), 19–44 (2015)

    Article  Google Scholar 

  59. Pipatsrisawat, K., Darwiche, A.: A lightweight component caching scheme for satisfiability solvers. In: Marques-Silva, J., Sakallah, K.A. (eds.) SAT 2007. LNCS, vol. 4501, pp. 294–299. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72788-0_28

    Chapter  Google Scholar 

  60. Pipatsrisawat, K., Darwiche, A.: On the power of clause-learning SAT solvers with restarts. In: Gent, I.P. (ed.) CP 2009. LNCS, vol. 5732, pp. 654–668. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04244-7_51

    Chapter  Google Scholar 

  61. Roussel, O.: Controlling a solver execution with the runsolver tool. J. Satisf. Boolean Model. Comput. 7, 139–144 (2011)

    MathSciNet  MATH  Google Scholar 

  62. Ryan, L.: The siege SAT solver (2003). https://www2.cs.sfu.ca/research/groups/CL/software/siege/

  63. Sakallah, K.A., Marques-Silva, J.: Anatomy and empirical evaluation of modern SAT solvers. Bull. EATCS 103, 96–121 (2011). http://eatcs.org/beatcs/index.php/beatcs/article/view/138

    MathSciNet  MATH  Google Scholar 

  64. Silva, J.P.M., Sakallah, K.A.: GRASP - a new search algorithm for satisfiability. In: Proceedings on the 7th International Conference on Computer-Aided Design (ICCAD 1996), pp. 220–227. Association for Computing Machinery, New York, November 1996

    Google Scholar 

  65. Soos, M.: Enhanced Gaussian elimination in DPLL-based SAT solvers. In: Berre, D.L., Gelder, A.V. (eds.) Proceedings of the Pragmatics of SAT (POS 2010) (2010)

    Google Scholar 

  66. Soos, M.: The CryptoMiniSat 5.5 set of solvers at the SAT competition 2018. In: Heule, M.J.H., Järvisalo, M., Suda, M. (eds.) Proceedings of SAT Competition 2018: Solver and Benchmark Descriptions. Department of Computer Science Series, vol. B-2018-1, pp. 17–18. University of Helsinki (2018)

    Google Scholar 

  67. Sörensson, N., Een, N.: Minisat v1.13 - a SAT solver with conflict clause minimization. Technical report, Chalmers University of Technology, Sweden (2005)

    Google Scholar 

  68. Stallman, R.: The GNU manifesto (1985). https://www.gnu.org/gnu/manifesto.en.html

  69. Stiller, S.: Planet der Algorithmen. Albrecht Knaus Verlag, Munich (2015)

    Google Scholar 

  70. The NetBSD www team: The NetBSD project (2020). https://www.netbsd.org/

  71. Trick, M., Chvatal, V., Cook, B., Johnson, D., McGeoch, C., Tarjan, B.: The 2nd DIMACS implementation challenge: 1992–1993 on NP hard problems: maximum clique, graph coloring, and satisfiability (1993). http://archive.dimacs.rutgers.edu/pub/challenge/sat/benchmarks/

  72. Zabih, R., McAllester, D.A.: A rearrangement search strategy for determining propositional satisfiability. In: Shrobe, H.E., Mitchell, T.M., Smith, R.G. (eds.) Proceedings of the 7th National Conference on Artificial Intelligence, St. Paul, MN, USA, 21–26 August 1988, pp. 155–160. AAAI Press/The MIT Press (1988)

    Google Scholar 

  73. Zhang, H.: Sato: an efficient propositional prover. In: McCune, W. (ed.) Proceedings of the 14th International Conference on Automated Deduction (CADE 1997), Townsville, QLD, Australia, pp. 272–275, July 1997

    Google Scholar 

Download references

Acknowledgements

We would like to thank several colleagues. Dave Mitchell and João Marques-Silva supported us with source codes from their old disks or mailboxes. Uwe Pretzsch and Siegmar Schöne helped to organize old hardware and Toni Pisjak maintained the modern benchmarking system, which we used.

Author information

Authors and Affiliations

  1. School of Engineering Sciences, TU Dresden, Dresden, Germany

    Johannes K. Fichte

  2. Database and Artificial Intelligence Group, TU Wien, Vienna, Austria

    Markus Hecher

  3. Algorithms and Complexity Group, TU Wien, Vienna, Austria

    Stefan Szeider

Authors
  1. Johannes K. Fichte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Markus Hecher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Szeider
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Johannes K. Fichte .

Editor information

Editors and Affiliations

  1. Insight Centre for Data Analytics, School for Computer Science and Information Technology, University College Cork, Cork, Ireland

    Helmut Simonis

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fichte, J.K., Hecher, M., Szeider, S. (2020). A Time Leap Challenge for SAT-Solving. In: Simonis, H. (eds) Principles and Practice of Constraint Programming. CP 2020. Lecture Notes in Computer Science(), vol 12333. Springer, Cham. https://doi.org/10.1007/978-3-030-58475-7_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58475-7_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58474-0

  • Online ISBN: 978-3-030-58475-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation