Annals of Operations Research

, Volume 201, Issue 1, pp 159–167 | Cite as

Strong polynomiality of the Gass-Saaty shadow-vertex pivoting rule for controlled random walks

Article

Abstract

We consider the subclass of linear programs that formulate Markov Decision Processes (mdps). We show that the Simplex algorithm with the Gass-Saaty shadow-vertex pivoting rule is strongly polynomial for a subclass of mdps, called controlled random walks (CRWs); the running time is O(|S|3⋅|U|2), where |S| denotes the number of states and |U| denotes the number of actions per state. This result improves the running time of Zadorojniy et al. (Mathematics of Operations Research 34(4):992–1007, 2009) algorithm by a factor of |S|. In particular, the number of iterations needed by the Simplex algorithm for CRWs is linear in the number of states and does not depend on the discount factor.

Keywords

Markov decision process Controlled queues Controlled random walks Simplex algorithm Gass-Saaty shadow-vertex pivoting rule 

References

  1. Amenta, N., & Ziegler, G. M. (1996). Advances in discrete and computational geometry. In Contemporary mathematics: Vol. 223. Deformed products and maximal shadows of polytopes. Providence: Am. Math. Soc. Google Scholar
  2. Barasz, M., & Vempala, S. (2010). A new approach to strongly polynomial linear programming. In Innovations in computer science (pp. 42–48). Google Scholar
  3. Borgwardt, K. H. (1982a). The average number of pivot steps required by the simplex-method is polynomial. Mathematical Methods of Operations Research, 26(1), 157–177. CrossRefGoogle Scholar
  4. Borgwardt, K. H. (1982). Some distribution-independent results about the asymptotic order of the average number of pivot steps of the simplex method. Mathematics of Operations Research, 7(3), 441–462. CrossRefGoogle Scholar
  5. de Ghellinck, G. (1960). Les problemes de decisions sequentielles. Cahiers Du Centre D’études de Recherche Opérationnelle, 2, 161–179. Google Scholar
  6. d’Epenoux, F. (1963). A probabilistic production and inventory problem. Management Science, 10(1), 98–108. CrossRefGoogle Scholar
  7. Derman, C. (1962). On sequential decisions and markov chains. Management Science, 9(1), 16–24. CrossRefGoogle Scholar
  8. Gass, S., & Saaty, T. (1955). The computational algorithm for the parametric objective function. Naval Research Logistics Quarterly, 2(1–2), 39–45. CrossRefGoogle Scholar
  9. Kitaev, M. Y., & Rykov, V. V. (1995). Controlled queueing systems. Boca Raton: CRC Press. Google Scholar
  10. Kleinrock, L. (1975). Queueing systems, Vol. I: Theory. New York: Wiley. Google Scholar
  11. Manne, A. S. (1960). Linear programming and sequential decisions. Management Science, 6(3), 259–267. CrossRefGoogle Scholar
  12. Matoušek, J., & Gärtner, B. (2007). Understanding and using linear programming. Berlin: Springer. Google Scholar
  13. Megiddo, N. (1984). Linear programming in linear time when the dimension is fixed. Journal of the ACM, 31(1), 114–127. CrossRefGoogle Scholar
  14. Megiddo, N. (1987). On the complexity of linear programming. In T. Bewley (Ed.), Advances in economic theory: fifth world congress (pp. 225–268). Cambridge: Cambridge University Press. CrossRefGoogle Scholar
  15. Megiddo, N., & Chandrasekaran, R. (1989). On the ε-perturbation method for avoiding degeneracy. Operations Research Letters, 8(6), 305–308. CrossRefGoogle Scholar
  16. Meyn, S. P. (2008). Control techniques for complex networks. Cambridge: Cambridge University Press. Google Scholar
  17. Puterman, M. L. (1994). Markov decision processes: discrete stochastic dynamic programming. New York: Wiley. Google Scholar
  18. Schrijver, A. (1998). Theory of linear and integer programming. New York: Wiley. Google Scholar
  19. Serfozo, R. F. (1979). An equivalence between continuous and discrete time markov decision processes. Operations Research, 27(3), 616–620. CrossRefGoogle Scholar
  20. Spielman, D. A., & Teng, S. H. (2004). Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time. Journal of the ACM, 51(3), 385–463. CrossRefGoogle Scholar
  21. Tardos, E. (1985). A strongly polynomial minimum cost circulation algorithm. Combinatorica, 5(3), 247–255. CrossRefGoogle Scholar
  22. Tardos, E. (1986). A strongly polynomial algorithm to solve combinatorial linear programs. Operations Research, 34(2), 250–256. CrossRefGoogle Scholar
  23. Terlaky, T., & Zhang, S. (1993). Pivot rules for linear programming: a survey on recent theoretical developments. Annals of Operations Research, 46(1), 203–233. CrossRefGoogle Scholar
  24. Vershynin, R. (2006). Beyond Hirsch conjecture: walks on random polytopes and smoothed complexity of the simplex method. In FOCS’06. 47th annual IEEE symposium on foundations of computer science, 2006 (pp. 133–142). New York: IEEE Press. CrossRefGoogle Scholar
  25. Yadin, M., & Naor, P. (1967). On queueing systems with variable service capacities. Naval Research Logistics Quarterly, 14, 43–53. CrossRefGoogle Scholar
  26. Ye, Y. (2005). A new complexity result on solving the Markov decision problem. Mathematics of Operations Research, 30(3), 733–749. CrossRefGoogle Scholar
  27. Ye, Y. (2010). The simplex and policy-iteration methods are strongly polynomial for the Markov decision problem with a fixed discount rate. Seminar, talk. Google Scholar
  28. Ye, Y. (2011). The simplex and policy-iteration methods are strongly polynomial for the Markov decision problem with a fixed discount rate. Mathematics of Operations Research, 36(4), 593–603. CrossRefGoogle Scholar
  29. Zadorojniy, A., & Even, G. Hyperbolic behavior of occupation measures between neighboring policies in CMDPs. http://www.eng.tau.ac.il/~sasha/.
  30. Zadorojniy, A., Even, G., & Shwartz, A. (2009). A strongly polynomial algorithm for controlled queues. Mathematics of Operations Research, 34(4), 992–1007. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.School of Electrical EngineeringTel-Aviv Univ.Tel-AvivIsrael
  2. 2.IBM ResearchHaifaIsrael

Personalised recommendations