How Many Needles are in a Haystack, or How to Solve #P-Complete Counting Problems Fast

  • Reuven Y. RubinsteinEmail author


We present two randomized entropy-based algorithms for approximating quite general #P-complete counting problems, like the number of Hamiltonian cycles in a graph, the permanent, the number of self-avoiding walks and the satisfiability problem. In our algorithms we first cast the underlying counting problem into an associate rare-event probability estimation, and then apply dynamic importance sampling (IS) to estimate efficiently the desired counting quantity. We construct the IS distribution by using two different approaches: one based on the cross-entropy (CE) method and the other one on the stochastic version of the well known minimum entropy (MinxEnt) method. We also establish convergence of our algorithms and confidence intervals for some special settings and present supportive numerical results, which strongly suggest that both ones (CE and MinxEnt) have polynomial running time in the size of the problem.


Cross-entropy Rare-event probability estimation Hamilton cycles Self-avoiding walks #P-complete problems Stochastic and simulation-based optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. T. M. Cover and J. A. Thomas, Elements of Information Theory, John Wiley & Sons, inc, 1991.Google Scholar
  2. P. T. de Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein, “A tutorial on the cross-entropy method,” Annals of Operations Research, 2005.Google Scholar
  3. M. Dyer, A. Frieze, and M. Jerrum, “Approximately counting Hamiltonian paths and cycles in dense graphs,” SIAM Journal of Computing vol. 27(5) pp. 1262–1272, 1998.CrossRefMathSciNetGoogle Scholar
  4. M. Jerrum, A. Sinclair, and E. Vigoda, “A polynomial-time approximation algorithm for the permanent of a matrix with the nonnegarive entries,” Journal of the ACM vol. 51(4) pp. 671–697, 2004.CrossRefMathSciNetGoogle Scholar
  5. Jun Gu et al., “Algorithms for the satisfiability (SAT) problem: A survey,” DIMACS Series in Discrete Mathematics, 1991.Google Scholar
  6. J. N. Kapur and Kesavan H. K., Entropy Optimization with Applications, Academic Press, Inc., 1992.Google Scholar
  7. D. Lieber, R. Y. Rubinstein, and D. Elmakis, “Quick estimation of rare events in stochastic networks,” IEEE Transactions on Reliability Systems vol. 46(2) pp. 254–265, 1997.CrossRefGoogle Scholar
  8. R. Motwani and R. Raghavan, Randomized Algorithms, Cambridge University Press, 1997.Google Scholar
  9. R. Y. Rubinstein, “The cross-entropy method for combinatorial and continuous optimization,” Methodology and Computing in Applied Probability vol. 2 pp. 127–190, 1999.CrossRefzbMATHGoogle Scholar
  10. R. Y. Rubinstein, “A stochastic minimum cross-entropy method for combinatorial optimization and rare-event estimation,” Methodology and Computing in Applied Probability (1) pp. 1–46, 2005.Google Scholar
  11. R. Y. Rubinstein and B. Melamed, Modern Simulation and Modeling, John Wiley & Sons, Inc., 1998.Google Scholar
  12. R. Y. Rubinstein and D. P. Kroese, The Cross-Entropy Method: a Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning, Springer, 2004.Google Scholar
  13. D. J. A. Welsh, Complexity: Knots, Colouring and Counting, Cambridge University Press, 1993.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  1. 1.Faculty of Industrial Engineering and Management, TechnionHaifaIsrael

Personalised recommendations