Abstract
In this note an m-state aperiodic irreducible Markov Chain is considered with an associated reward matrix. Expressions for the second moment of Sn, the accumulated reward in n transitions, have been obtained using elementary methods. These provide an alternative computational method compared to methods using the eigen value structure. Asmyptotic forms are also considered.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
Reference
Bhat, U.N., (1972) Elements of Applied Stochastic Processes. John Wiley and Sons, Inc., New York.
Howard, R.A., (1971) Dynamic Probabilistic Systems. Vol.11: Semi Markov and Decision Processes. John Wiley and Sons, Inc.,New York.
Phatarfod, R.M., (1965) Sequential Analysis of Dependent Observations-I, Biometrika, 52, 157–165.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1991 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Subba Rao, S. (1991). The Second Moment of the Markovian Reward Process. In: Beckmann, M.J., Gopalan, M.N., Subramanian, R. (eds) Stochastic Processes and their Applications. Lecture Notes in Economics and Mathematical Systems, vol 370. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-58201-1_23
Download citation
DOI: https://doi.org/10.1007/978-3-642-58201-1_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-54635-1
Online ISBN: 978-3-642-58201-1
eBook Packages: Springer Book Archive