Abstract
We consider the recently solved problem of Optimal Stopping of Seasonal Observations and its more general version. Informally, there is a finite number of dice, each for a state of “underlying” finite MC. If this MC is in a state k, then k-th die is tossed. A Decision Maker (DM) observes both MC and the value of a die, and at each moment of discrete time can either continue observations or to stop and obtain a discounted reward. The goal of a DM is to maximize the total expected discounted reward. This problem belongs to an important class of stochastic optimization problems—the problem of optimal stopping of Markov chains (MCs). The solution was obtained via an algorithm which is based on the general, so called, State Elimination algorithm developed by the author earlier. An important role in the solution is played by the relationship between the fundamental matrix of a transient MC in the “large” state space and the fundamental matrix for the modified underlying transient MC. In this paper such relationship is presented in a transparent way using the general concept of a projection of a Markov model. The general relationship between two fundamental matrices is obtained and used to clarify the solution of the optimal stopping problem.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Grassmann, W.K., Taksar, M., Heyman, D.: Regenerative analysis and steady state distributions for Markov chains. Oper. Res. 33(5), 1107–1116 (1985)
Howard, R.: Dynamic Probabilistic Systems. Markov Models. Wiley, New York (1971)
Kemeny, J., Snell, L.: Finite Markov Chains. Springer, Berlin (1960, 1983)
Presman, E.: The solution of optimal stopping problem based on a modification of a payoff function (2010). This volume
Presman, E., Sonin, I.: On optimal stopping of random sequences modulated by Markov chain. Theory Probab. Appl. 54(3), 534–542 (2010)
Sheskin, T.: A Markov chain partitioning algorithm for computing steady state probabilities. Oper. Res. 33(1), 228–235 (1985)
Sonin, I.: The optimal stopping of “seasonal observations”. In: Proc. 11th INFORMS Appl. Prob. Conf., p. 18. New York (2001)
Sonin, I.: The elimination algorithm for the problem of optimal stopping. Math. Methods Oper. Res. 4(1), 111–123 (1999)
Sonin, I.: The state reduction and related algorithms and their applications to the study of Markov chains, graph theory and the optimal stopping problem. Adv. Math. 145(2), 159–188 (1999)
Sonin, I., Thornton, J.: Recursive algorithm for the fundamental group inverse matrix of a Markov chain from an explicit formula. SIAM J. Matrix Anal. Appl. 23(1), 209–224 (2001)
Acknowledgements
The author would like to thank Joe Quinn, Ernst Presman, and an anonymous referee for valuable comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Sonin, I.M. (2014). Optimal Stopping of Seasonal Observations and Projection of a Markov Chain. In: Kabanov, Y., Rutkowski, M., Zariphopoulou, T. (eds) Inspired by Finance. Springer, Cham. https://doi.org/10.1007/978-3-319-02069-3_25
Download citation
DOI: https://doi.org/10.1007/978-3-319-02069-3_25
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-02068-6
Online ISBN: 978-3-319-02069-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)