Annals of Operations Research

, Volume 201, Issue 1, pp 159–167

Strong polynomiality of the Gass-Saaty shadow-vertex pivoting rule for controlled random walks

Article

DOI: 10.1007/s10479-012-1199-x

Cite this article as:
Even, G. & Zadorojniy, A. Ann Oper Res (2012) 201: 159. doi:10.1007/s10479-012-1199-x

Abstract

We consider the subclass of linear programs that formulate Markov Decision Processes (mdps). We show that the Simplex algorithm with the Gass-Saaty shadow-vertex pivoting rule is strongly polynomial for a subclass of mdps, called controlled random walks (CRWs); the running time is O(|S|3⋅|U|2), where |S| denotes the number of states and |U| denotes the number of actions per state. This result improves the running time of Zadorojniy et al. (Mathematics of Operations Research 34(4):992–1007, 2009) algorithm by a factor of |S|. In particular, the number of iterations needed by the Simplex algorithm for CRWs is linear in the number of states and does not depend on the discount factor.

Keywords

Markov decision process Controlled queues Controlled random walks Simplex algorithm Gass-Saaty shadow-vertex pivoting rule 

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.School of Electrical EngineeringTel-Aviv Univ.Tel-AvivIsrael
  2. 2.IBM ResearchHaifaIsrael

Personalised recommendations