HeavyTailed Phenomena in Satisfiability and Constraint Satisfaction Problems
 Carla P. Gomes,
 Bart Selman,
 Nuno Crato,
 Henry Kautz
 … show all 4 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
We study the runtime distributions of backtrack procedures for propositional satisfiability and constraint satisfaction. Such procedures often exhibit a large variability in performance. Our study reveals some intriguing properties of such distributions: They are often characterized by very long tails or “heavy tails”. We will show that these distributions are best characterized by a general class of distributions that can have infinite moments (i.e., an infinite mean, variance, etc.). Such nonstandard distributions have recently been observed in areas as diverse as economics, statistical physics, and geophysics. They are closely related to fractal phenomena, whose study was introduced by Mandelbrot. We also show how random restarts can effectively eliminate heavytailed behavior. Furthermore, for harder problem instances, we observe long tails on the lefthand side of the distribution, which is indicative of a nonnegligible fraction of relatively short, successful runs. A rapid restart strategy eliminates heavytailed behavior and takes advantage of short runs, significantly reducing expected solution time. We demonstrate speedups of up to two orders of magnitude on SAT and CSP encodings of hard problems in planning, scheduling, and circuit synthesis.
 Adler, R., Feldman, R. and Taggu, M. (eds.): A Practical Guide to Heavy Tails, Birkhäuser, 1998
 Alt, H., Guibas, L., Mehlhorn, K., Karp, R. and Wigderson, A.: A method for obtaining randomized algorithms with small tail probabilities, Algorithmica 16(4–5) (1996), 543–547.
 Anderson, L.: Completing partial latin squares, Math. Fys. Meddelelser 41 (1985), 23–69.
 Bayardo, R.: Personal communications, 1999.
 Bayardo, R. and Miranker, D.: A complexity analysis of spacebounded learning algorithms for the constraint satisfaction problem, in Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI96), Portland, OR, 1996, pp. 558–562.
 Bayardo, R. and Schrag, R.: Using CSP lookback techniques to solve realworld SAT instances, in Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI97), New Providence, RI, 1997, pp. 203–208.
 Brelaz, D.: New methods to color the vertices of a graph, Comm. ACM 22(4) (1979), 251–256.
 Colbourn, C.: Embedding partial Steiner triple systems is NPcomplete, J. Combin. Theory A 35 (1983), 100–105.
 Cook, S. and Mitchell, D.: Finding hard instances of the satisfiability problem: A survey, in Du, Gu, and Pardalos (eds), Satisfiability Problem: Theory and Applications, Dimacs Series in Discrete Math. and Theoret. Comput. Sci. 35, 1998.
 Crato, N., Gomes, C. and Selman, B.: On estimating the index of stability with truncated data, Manuscript in preparation.
 Crawford, J. and Baker, A.: Experimental results on the application of satisfiability algorithms to scheduling problems, in Proceedings of The Tenth National Conference on Artificial Intelligence (AAAI94), Austin, TX, 1994, pp. 1092–1098.
 Davis, M., Logemann, G. and Loveland, D.: A machine program for theorem proving, Comm. ACM 5 (1979), 394–397.
 Davis, M. and Putman, H.: A computing procedure for quantification theory, J. ACM 7 (1960), 201–215.
 Denes, J. and Keedwell, A.: Latin Squares and Their Applications, Akademiai Kiado, Budapest, and English Universities Press, 1974.
 Ertel, W. and Luby, M.: Optimal parallelization of Las Vegas algorithms, in Symp. on Theoretical Aspects of Computer Science, Lecture Notes in Comput. Sci. 775, 1994, pp. 463–475.
 Feller,W.: An Introduction to Probability Theory and Its Applications, Vol. I, Wiley, New York, 1968.
 Feller, W.: An Introduction to Probability Theory and Its Applications, Vol. II, Wiley, New York, 1971.
 Frost, D., Rish, I. and Vila, L.: Summarizing CSP hardness with continuous probability distributions, in Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI97), New Providence, RI, 1997, pp. 327–334.
 Fujita, M., Slaney, J. and Bennett, F.: Automatic generation of some results in finite algebra, in Proceedings of the International Joint Conference on Artificial Intelligence, France, 1993, pp. 52–57.
 Gent, I. and Walsh, T.: Easy problems are sometimes hard, Artif. Intell. 70 (1993), 335–345.
 Gomes, C. P. and Selman, B.: Algorit portfolio design: Theory vs. practice, in Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI97), Providence, RI, 1997, pp. 190–197.
 Gomes, C. P. and Selman, B.: Problem structure in the presence of perturbations, in Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI97), New Providence, RI, 1997, pp. 221–227.
 Gomes, C. P., Selman, B. and Crato, N.: Heavytailed distributions in combinatorial search, in G. Smolka (ed.), Principles and Practice of Constraint Programming (CP97), Lecture Notes in Comput. Sci., Linz, Austria, 1997, pp. 121–135.
 Hall, P.: On some simple estimates of an exponent of regular variation, J. Roy. Statist. Soc. 44 (1982), 37–42.
 Hayes, B.: Computer science: Can't get no satisfaction, American Scientist 85(2) (1996), 108–112.
 Hill, B.: A simple general approach to inference about the tail of a distribution, Ann. Statist. 3 (1975), 1163–1174.
 Hogg, T., Huberman, B. and Williams, C. (eds.): Phase transitions and complexity (Special Issue), Artif. Intell. 81(1–2) (1996).
 Hogg, T. and Williams, C.: Expected gains from parallelizing constraint solving for hard problems, in Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI94), Seattle, WA, 1994, pp. 1310–1315.
 Hoos, H.: Stochastic local searchmethods, models, applications, Ph.D. Thesis, TU Darmstadt, 1998.
 Huberman, B., Lukose, R. and Hogg, T.: An economics approach to hard computational problems, Science 275 (1993), 51–54.
 Johnson, D.: A theoretician's guide to the experimental analysis of algorithms, Preliminary manuscript. Invited talk presented at AAAI96, Portland, OR, 1996.
 Johnson, D. and Trick,M.: Cliques, coloring, and satisfiability: Second dimacs implementation challenge, in Dimacs Series in Discrete Math. and Theoret. Comput. Sci. 36, 1996.
 Kamath, A., Karmarkar, N., Ramakrishnan, K. and Resende, M.: Computational experience with an interior point algorithm on the satisfiability problem, in Proceedings of Integer Programming and Combinatorial Optimization, Waterloo, Canada, 1990, pp. 333–349.
 Kautz, H. and Selman, B.: Pushing the envelope: Planning, propositional logic, and stochastic search, in Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI96), Portland, OR, 1996, pp. 1188–1194.
 Kirkpatrick, S. and Selman, B.: Critical behavior in the satisfiability of random Boolean expressions, Science 264 (1994), 1297–1301.
 Kwan, A.: Validity of normality assumption in CSP research, in Proceedings of the Pacific Rim International Conference on Artificial Intelligence, 1995, pp. 459–465.
 Lam, C., Thiel, L. and Swiercz, S.: The nonexistence of finite projective planes of order 10, Canad. J. Math. XLI(6) (1994), 1117–1123.
 Li, C. M.: Personal communications, 1999.
 Li, C. M. and Anbulagan: Heuristics based on unit propagation for satisfiability problems, in Proceedings of the International Joint Conference on Artificial Intelligence, 1997, pp. 366–371.
 Luby, M., Sinclair, A. and Zuckerman, D.: Optimal speedup of Las Vegas algorithms, Inform. Process. Lett., 1993, pp. 173–180.
 Macready, W., Siapas, A. and Kauffman, S.: Criticality and parallelism in combinatorial optimization, Science 271 (1996), 56–59.
 Mandelbrot, B.: The ParetoLévy law and the distribution of income, Internat. Econom. Rev. 1 (1960), 79–106.
 Mandelbrot, B.: The variation of certain speculative prices, J. Business 36 (1963), 394–419.
 Mandelbrot, B.: The Fractal Geometry of Nature, Freeman, New York, 1983
 McAloon, K., Tretkoff, C. and Wetzel, G.: Sports league scheduling, in Proceedings of Third Ilog International Users Meeting, 1997.
 Mitchell, D. and Levesque, H.: Some pitfalls for experimenters with random SAT, Artif. Intell. 81(1–2) (1996), 111–125.
 Monasson, R., Zecchina, R., Kirkpatrick, S., Selman, B. and Troyansky, L.: Determining computational complexity from characteristic 'phase transitions', Nature 400(8) (1999), 133–137.
 Nemhauser, G. and Trick, M.: Scheduling a major college basketball conference, Oper. Res. 46(1) (1998), 1–8.
 Niemela, I.: Logic programs with stable model semantics as a constraint programming paradigm. Ext. version of paper presented at the Workshop on Computational Aspects of Nonmonotonic Reasoning, 1998.
 Niemela, I.: Personal communications, 1999.
 Nolan, J.: Stable distributions, Manuscript, in preparation, 1999.
 Prosser, P.: Hybrid algorithms for the constraint satisfaction problem, Comput. Intell. 9(3) (1993), 268–299.
 Puget, J. F. and Leconte, M.: Beyond the black box: Constraints as objects, in Proceedings of ILPS'95, Lisbon, Portugal, 1995, pp. 513–527.
 Regin, J. C.: A filtering algorithm for constraints of difference in CSP, in Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI94), Seattle, WA, 1994, pp. 362–367.
 Rish, I. and Frost, D.: Statistical analysis of backtracking on inconsistent CSPs, in Proceedings of Third International Conference on Principles and Practices of Constraint Programming, 1997, pp. 150–162.
 Samorodnitsky, G. and Taqqu, M.: Stable NonGaussian Random Processes: Stochastic Models with Infinite Variance, Chapman and Hall, 1994.
 Selman, B., Kautz, H. and Cohen, B.: Local search strategies for satisfiability testing, in D. Johnson and M. Trick (eds), Dimacs Series in Discrete Math. and Theoret. Comput. Sci. 26, 1993, pp. 521–532.
 Selman, B. and Kirkpatrick, S.: Finitesize scaling of the computational cost of systematic search, Artif. Intell. 81(1–2) (1996), 273–295.
 Shaw, P., Stergiou, K. and Walsh, T.: Arc consistency and quasigroup completion, in Proceedings of ECAI98, Workshop on Binary Constraints, 1998.
 Shonkwiler, R., E. and van Vleck, E.: Parallel speedup of Monte Carlo methods for global optimization, J. Complexity 10 (1994), 64–95.
 Slaney, J., Fujita, M. and Stickel, M.: Automated reasoning and exhaustive search: Quasigroup existence problems, Comput. Math. Appl. 29 (1995), 115–132.
 Smith, B. and Grant, S.: Sparse constraint graphs and exceptionally hard problems, in Proceedings of the International Joint Conference on Artificial Intelligence, 1995, pp. 646–651.
 Stickel, M.: Personal communications, 1998.
 Vandengriend and Culberson: The Gnm phase transition is not hard for the Hamiltonian cycle problem, J. Artif. Intell. Res. 9 (1999), 219–245.
 Walsh, T.: Search in a small world, in Proceedings of the International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 1999
 Zolotarev, V.: OneDimensional Stable Distributions, Transl. Math. Monographs 65, Amer. Math. Soc., 1986. Translation from the original 1983 Russian ed.
 Title
 HeavyTailed Phenomena in Satisfiability and Constraint Satisfaction Problems
 Journal

Journal of Automated Reasoning
Volume 24, Issue 12 , pp 67100
 Cover Date
 20000201
 DOI
 10.1023/A:1006314320276
 Print ISSN
 01687433
 Online ISSN
 15730670
 Publisher
 Kluwer Academic Publishers
 Additional Links
 Topics
 Keywords

 satisfiability
 constraint satisfaction
 heavy tails
 backtracking
 Industry Sectors