Abstract
It is shown how a discrete Markov programming problem can be transformed, using a linear program, into an equivalent problem from which the optimal decision rule can be trivially deduced. This transformation is applied to problems which have either transient probabilities or discounted costs.
Similar content being viewed by others
References
M.L. Balinski, “Notes on a constructive approach to linear programming”, in: G.B. Dantzig and A.F. Veinott, eds.,Mathematics of the decision sciences (Am. Math. Soc., Providence, RI, 1968) pp. 38–64.
D. Blackwell, “Discrete dynamic programming”,Annals of Mathematical Statistics 33 (1962) 719–726.
F. D'Épenoux, “Sur un problème de production et de stockage dans l'aléatoire”,Revue Francaise de Recherche Opérationelle 14 (1960) 3–16.
G.T. DeGhellinck and C.D. Eppen, “Linear programming solutions for separable Markov decision problems”,Management Science 13 (1967) 371–394.
E.V. Denardo, “Contraction mappings in the theory underlying dynamic programming”,SIAM Review 9 (1967) 165–177.
R.A. Howard,Dynamic programming and Markov processes (Wiley, New York, 1960).
H.W. Kuhn, “The Hungarian method for the assignment problem”,Naval Research Logistics Quarterly 2 (1955) 83–97.
A.S. Manne, “Linear programming and sequential decisions”,Management Science 6 (1960) 259–267.
A.F. Veinott, Jr., “Markov decision chains”, in: G.B. Dantzig and B.C. Eaves, eds.,Studies in optimization (Mathematical Association of America, Washington, DC, 1974) pp. 124–159.
P. Wolfe and G.B. Dantzig, “Linear programming in a Markov chain”,Operations Research 10 (1962) 702–710.
Author information
Authors and Affiliations
Additional information
This research was supported by the National Research Council of Canada, Grant A7751.
Rights and permissions
About this article
Cite this article
Sutherland, W.R.S. Optimality in transient markov chains and linear programming. Mathematical Programming 18, 1–6 (1980). https://doi.org/10.1007/BF01588291
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01588291