Journal of Global Optimization

, Volume 29, Issue 3, pp 315–334 | Cite as

An Interior Point Heuristic for the Hamiltonian Cycle Problem via Markov Decision Processes

  • Vladimir Ejov
  • Jerzy Filar
  • Jacek Gondzio


We consider the Hamiltonian cycle problem embedded in a singularly perturbed Markov decision process (MDP). More specifically, we consider the HCP as an optimization problem over the space of long-run state-action frequencies induced by the MDP's stationary policies. We show that Hamiltonian cycles (if any) correspond to the global minima of a suitably constructed indefinite quadratic programming problem over the frequency space. We show that the above indefinite quadratic can be approximated by quadratic functions that are `nearly convex' and as such suitable for the application of logarithmic barrier methods. We develop an interior-point type algorithm that involves an arc elimination heuristic that appears to perform rather well in moderate size graphs. The approach has the potential for further improvements.

Hamiltonian cycles interior point methods Markov decision processes non-convex optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Alspach, B. (1983), The classi cation of Hamiltonian generalized Petersen graphs, Journal of Combinatorial Theory ser.B 34, 293-312.Google Scholar
  2. [2]
    Altman A. and Gondzio J. (1999), Regularized symmetric inde nite systems in interior point methods for linear and quadratic optimization, Optimization Methods and Software 11-12, 275-302.Google Scholar
  3. [3]
    Andersen, E.D., Gondzio, J., Mészáros, C. and Xu, X. (1996), Implementation of interior point methods for large scale linear programming.In:Terlaky, T.(ed.), Interior Point Methods in Mathematical Programming, Kluwer Academic Publishers, Dordrecht, pp.189-252.Google Scholar
  4. [4]
    Andramonov, M., Filar, J.A., Rubinov, A. and Pardalos, P. (2000), Dordrecht Hamiltonian cycle problem via Markov chains and min-type approaches.In:Pardalos, P.M.(ed.), Approx-imation and Complexity in Numerical Optimization:Continuous and Discrete Problems, Kluwer Academic Publishers, Dordrecht, pp.31-47.Google Scholar
  5. [5]
    Bunch, J.R. and Parlett, B.N. (1971), Direct methods for solving symemtric inde nite systems of linear equations, SIAM Journal on Numerical Analysis, 8, 639-655.Google Scholar
  6. [6]
    Biggs, N.L. (1974), Algebraic Graph Theory, Cambridge University Press, London, New York.Google Scholar
  7. [7]
    Chen, M. and Filar, J.A. (1992), Hamiltonian cycles, quadractic programming and ranking of extreme points.In:Floudas, C.A. and Pardalos, P.M.(eds.), Recent Advances in Global Optimization, Princeton University Press, Princeton, NJ, pp.32-49.Google Scholar
  8. [8]
    Duff, I.S., Erisman, A.M. and Reid, J.K. (1987), Direct Methods for Sparse Matrices, Oxford University Press, New York.Google Scholar
  9. [9]
    Ejov, V., Filar, J. and Thredgold, J., Geometric interpretation of Hamiltonian Cycles problem via singularly perturbed Markov decision processes, Optimization (to appear).Google Scholar
  10. [10]
    Feinberg, E. (2000), Constrained discounted Markovdecision processes with Hamiltonian cycles, Mathematics of Operations Research, 25, 130-140.Google Scholar
  11. [11]
    Filar, J. and Krass, D. (1994), Hamiltonian cycles and Markovchains, Mathematics of Oper-ations Research, 19, 223-237.Google Scholar
  12. [12]
    Filar, J.A. and Lasserre, J.B. (2000), A non-standard branch and bound method for the Hamiltonian cycle problem, ANZIAM J.42(E), 556-577.Google Scholar
  13. [13]
    Filar, J. and Schultz, T. (1988), Commmunicating MDPs:equivalence and LP properties, Operations Research Letters, 7, 303-307.Google Scholar
  14. [14]
    Filar, J. and Vrieze, K. (1996), Competitive Markov Decision Processes, Springer, Berlin.Google Scholar
  15. [15]
    Gondzio, J. (1996), Multiple centrality corrections in a primal-dual method for linear pro-gramming, Computational Optimization and Applications, 6, 137-156.Google Scholar
  16. [16]
    Kallenberg, L. (2002), Classi cation problems in MDPs.In:Hou, Z., Filar, J. and Chen, A.(eds.), Markov Processes and Controlled Markov Chains, Kluwer Academic Publishers, pp.151-166.Google Scholar
  17. [17]
    Parberry, I. (1996), Scalability of a neural network for the knight 's tour problem, Neurocom-puting, 12, 19-34.Google Scholar
  18. [18]
    Pardalos, P.M. & Resende, M.G.C. (1996), Interior point methods for global optimization problems.In:Terlaky, T.(ed.), Interior Point Methods of Mathematical Programming, Kluwer Academic Publishers, Dordrecht, pp.467-500.Google Scholar
  19. [19]
    Vanderbei, R.J. (1995), Symmetric quaside nite matrices, SIAM Journal on Optimization, 5, 100-113.Google Scholar
  20. [20]
    Wolsey, L.A. (1998), Integer Programming, John Wiley & Sons, New York.Google Scholar
  21. [21]
    Wright, S.J. (1997), Primal-Dual Interior-Point Methods, SIAM, Philadelphia.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Vladimir Ejov
    • 1
  • Jerzy Filar
    • 1
  • Jacek Gondzio
    • 2
  1. 1.School of MathematicsThe University of South AustraliaMawson LakesAustralia
  2. 2.School of MathematicsUniversity of EdinburghEdinburghUK

Personalised recommendations