Journal of Global Optimization

, Volume 55, Issue 3, pp 633–639 | Cite as

Markov chains, Hamiltonian cycles and volumes of convex bodies

  • Vivek S. Borkar
  • Jerzy A. FilarEmail author


In this note the Hamiltonian cycle problem is mapped into an infinite horizon discounted cost constrained Markov decision problem. The occupation measure based linear polytope associated with this control problem defines a convex set which either strictly contains or is equal to another convex set, depending on whether the underlying graph has a Hamiltonian cycle or not. This allows us to distinguish Hamiltonian graphs from non-Hamiltonian graphs by comparing volumes of two convex sets.


Hamiltonian cycle problem Markov decision process Discounted cost Volumes of convex sets Uniform sampling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ahamed T.P.I., Borkar V.S., Juneja S.K.: Adaptive importance sampling technique for Markov chains using stochastic approximation. Oper. Res. 54, 489–504 (2006)CrossRefGoogle Scholar
  2. 2.
    Altman E.: Constrained Markov Decision Processes. Chapman and Hall, Boca Raton (1999)Google Scholar
  3. 3.
    Andramonov M., Filar J.A., Pardalos P.M., Rubinov A.: Hamiltonian cycle problem via Markov chains and min-type approaches. In: Pardalos, P.M. (eds) Approximation and Complexity in Numerical Optimization, pp. 31–47. Kluwer Academic Publishers, Dordrecht (2000)Google Scholar
  4. 4.
    Borkar V.S.: Convex analytic approach to Markov decision processes. In: Shwartz, A., Feinberg, E. (eds.) Handbook of Markov Decision Proceses, Kluwer Academic, Boston (2000)Google Scholar
  5. 5.
    Denardo, E.V., Feinberg, E.A., Rothblum, U.G.: On occupation measures for total-reward MDPs. In: Proceedings of the 47th IEEE Conference on Decision and Control, pp. 4460–4465. Cancun, Mexico Dec. 9–11 2008Google Scholar
  6. 6.
    Dyer M.E., Frieze A.M.: On the complexity of computing the volume of a polyhedron. SIAM J. Comput. 17, 967–974 (1988)CrossRefGoogle Scholar
  7. 7.
    Dyer M.E., Frieze A.M., Kannan R.: A random polynomial time algorithm for approximating the volume of convex bodies. J. ACM 38, 1–17 (1998)CrossRefGoogle Scholar
  8. 8.
    Ejov V., Filar J.A., Haythorpe M., Nguyen G.T.: Refined MDP-based branch-and-fix algorithm for the Hamiltonian cycle problem. Math. Oper. Res. 34, 758–768 (2009)CrossRefGoogle Scholar
  9. 9.
    Ejov V., Filar J.A., Gondzio J.: An interior point heuristic algorithm for the HCP. J. Global Optim. 29(3), 315–334 (2004)CrossRefGoogle Scholar
  10. 10.
    Eshragh, A., Filar, J.A., Haythorpe M.: A hybrid simulation-optimization algorithm for the Hamiltonian cycle problem. Ann. Oper. Res. 189, 103–125Google Scholar
  11. 11.
    Feinberg E.A.: Constrained discounted Markov decision processes and Hamiltonian cycles. Math. Oper. Res. 25, 130–140 (2000)CrossRefGoogle Scholar
  12. 12.
    Filar J.A.: Controlled Markov chains, graphs and Hamiltonicity. Found. Trends Stoch. Syst. 1(2), 77–162 (2000)CrossRefGoogle Scholar
  13. 13.
    Filar J.A., Krass D.: Hamiltonian cycles and Markov chains. Math. Oper. Res 19, 223–237 (1994)CrossRefGoogle Scholar
  14. 14.
    Filar, J.A., Oberije, M., Pardalos, P.M.: Hamiltonian cycle problem, controlled Markov chains and quadratic programming. In: Sutton, D.J. Pearce, C.E.M., Cousins, E.A. (eds.) The Proceedings of the 12th National Conference of the Australian Society for Operations Research, pp. 263–281. Adelaide, July 7–9 1993Google Scholar
  15. 15.
    Haythorpe, M.: Markov Chain Based Algorithms for the Hamiltonian Cycle Problem. PhD Thesis, University of South Australia, Adelaide (2010)Google Scholar
  16. 16.
    Jerrum M.: Counting, Sampling and Integrating: Algorithms and Complexity. Birkhäuser, Basel (2003)CrossRefGoogle Scholar
  17. 17.
    Lovász L., Vempala S.: Simulated annealing in convex bodies and an O(n 4) volume algorithm. J. Comput. Syst. Sci. 72, 392–417 (2006)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2011

Authors and Affiliations

  1. 1.School of Technology and Computer ScienceTata Institute of Fundamental ResearchMumbaiIndia
  2. 2.School of Computer Science, Engineering and MathematicsFlinders UniversityAdelaideAustralia

Personalised recommendations