Randomized Algorithms with Splitting: Why the Classic Randomized Algorithms Do Not Work and How to Make them Work

  • Reuven Rubinstein


We show that the original classic randomized algorithms for approximate counting in NP-hard problems, like for counting the number of satisfiability assignments in a SAT problem, counting the number of feasible colorings in a graph and calculating the permanent, typically fail. They either do not converge at all or are heavily biased (converge to a local extremum). Exceptions are convex counting problems, like estimating the volume of a convex polytope. We also show how their performance could be dramatically improved by combining them with the classic splitting method, which is based on simulating simultaneously multiple Markov chains. We present several algorithms of the combined version, which we simple call the splitting algorithms. We show that the most advance splitting version coincides with the cloning algorithm suggested earlier by the author. As compared to the randomized algorithms, the proposed splitting algorithms require very little warm-up time while running the MCMC from iteration to iteration, since the underlying Markov chains are already in steady-state from the beginning. What required is only fine tuning, i.e. keeping the Markov chains in steady-state while moving from iteration to iteration. We present extensive simulation studies with both the splitting and randomized algorithms for different NP-hard counting problems.


Combinatorial optimization Counting Cross-entropy Gibbs sampler Importance sampling Rare-event Randomized algorithms Splitting 

AMS 2000 Subject Classifications

65C05 60C05 68W20 90C59 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Asmussen S, Glynn PW (2007) Stochastic simulation: algorithms and analyses. Springer, Berlin Heidelberg New YorkGoogle Scholar
  2. Baumert S, Ghate A, Kiatsupaibul S, Shen Y, Smith RL, Zabinsky ZB (2009) Discrete hit-and-run for sampling points from arbitrary distributions over subsets of integer hyper-rectangles. Operations Research, JohannesburgGoogle Scholar
  3. Botev ZI, Kroese DP (2008) An efficient algorithm for rare-event probability estimation, combinatorial optimization, and counting. Methodol Comput Appl Probab 10:471–505CrossRefMathSciNetGoogle Scholar
  4. Del Moral P (2004a) Feynman-Kac formulae, genealogical and interacting particle systems with applications. Springer, New YorkzbMATHGoogle Scholar
  5. Del Moral P (2004b) Feynman-Kac formulae genealogical and interacting particle systems. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  6. Garvels MJJ (2000) The splitting method in rare-event simulation. PhD thesis, University of TwenteGoogle Scholar
  7. Hayes T, Vera J, Vigoda E (2007) Randomly coloring planar graphs with fewer colors than the maximum degree. STOCGoogle Scholar
  8. Kahn H, Harris TE (1951) Estimation of particle transmission by random sampling. Natl Bur Stand Appl Math Ser 12:27–30Google Scholar
  9. L’Ecuyer P, Demers V, Tuffin B (2007) Rare-events, splitting, and quasi-Monte Carlo. ACM Trans Model Comput Simul 17:2CrossRefGoogle Scholar
  10. Mitzenmacher M, Upfal E (2005) Probability and computing: randomized algorithms and probabilistic analysis. Cambridge University Press, New YorkzbMATHGoogle Scholar
  11. Motwani R, Raghavan R (1997) Randomized algorithms. Cambridge University Press, CambridgeGoogle Scholar
  12. Ross SM (2006) Simulation, 4th edn. Academic, LondonzbMATHGoogle Scholar
  13. Rubinstein RY (2008) Entropy and cloning methods for combinatorial optimization, sampling and counting using the Gibbs sampler. To be published in Information Theory and Statistical Learning. Springer, Berlin Heidelberg New YorkGoogle Scholar
  14. Rubinstein RY (2009) The Gibbs cloner for combinatorial optimization, counting and sampling. Methodol Comput Appl Probab (in press)Google Scholar
  15. Rubinstein RY, Kroese DP (2004) The cross-entropy method: a unified approach to combinatorial optimization, Monte-Carlo simulation and machine learning. Springer, Berlin Heidelberg New YorkzbMATHGoogle Scholar
  16. Rubinstein RY, Kroese DP (2007) Simulation and the Monte Carlo method, 2nd edn. Wiley, New YorkGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Faculty of Industrial Engineering and TechnionIsrael Institute of TechnologyHaifaIsrael

Personalised recommendations