Starting from the definitions and the properties of reinforced renewal processes and reinforced Markov renewal processes, we characterize, via exchangeability and de Finetti’s representation theorem, a prior that consists of a family of Dirichlet distributions on the space of Markov transition matrices and beta-Stacy processes on distribution functions. Then, we show that this family is conjugate and give some estimate results.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Markov renewal process
reinforced Markov renewal process
reinforced semi-Markov process
Brock DB (1973) Bayesian analysis for Markov renewal process. Metron 29:185–198
Çinlar E (1975) Markov renewal theory: a survey. Manag Sci J Instit Manage Sci 21:727–752
Duncan GT (1990) Bayes estimation in the Markov renewal process. Bayesian and Likelihood Methods in Statistics and Econometrics: Essays in Honor of George A. Barnard. North-Holland, Amsterdam, pp 265–275
Gill RD (1980) Nonparametric estimation based on censored observations of a Markov renewal process. Zeitschrift Wahrscheinlichkeitstheorie Verwandte Gebiete [Became: @J(ProbTher)] 53:97–116
Gill RD (1994) Lectures on survival analysis. In: Bernard P (eds) Lectures on probability theory (Ecole d’été de Probabilité de Saint-Flour XXII—1992). Springer, Berlin
Gill RD, Johansen S (1990) A survey of product-integration with a view toward application in survival analysis. Ann Stat 18:1501–1555
Greenwood PE, Wefelmeyer W (1996) Empirical estimators for semi-Markov processes. Math Methods Statist 5:299–315
Hjort NL (1990) Nonparametric bayes estimators based on beta processes in models for life history data. Ann Statis 18(3):1259–1294
Jain RK (1990) Statistical estimation for semi-Markov models. Metron 48:223–242
Lagakos SW, Sommer CJ, Zelen M (1978) Semi-Markov models for partially censored data. Biometrika 65:311–318
Lévy P (1954a) Processus semi-Markoviens. Proc Int Congr Math 3:416–426
Lévy P (1954b) Systèmes semi-Markoviens à au plus une infinité dénombrable d’états possibles’. Proc Int Congr Math 2:294
Moore EH, Pyke R (1968) Estimation of the transition distributions of a Markov renewal process. Ann Instit Statist, Math 20:411–424
Muliere P, Secchi P, Walker SG (2000) Urn schemes and reinforced random walks. Stochastic Processes Appl 88(1):59–78
Muliere P, Secchi P, Walker SG (2003) Reinforced random processes in continuous time. Stochastic Processes Appl 104:117–130
Phelan MJ (1990a) Bayes estimation from a Markov renewal process. Ann Statist 18:603–616
Phelan MJ (1990b) Estimating the transition probabilities from censored Markov renewal processes. Statist Probab Lett 10:43–47
Pyke R (1961a) Markov renewal processes: definitions and preliminary properties. Ann Math Statist 32:1231–1242
Pyke R (1961b) Markov renewal processes with finitely many states. Ann Math Statist 32:1243–1259
Pyke R, Schaufele R (1964) Limit theorems for Markov renewal processes. Ann Math Statist 35:1746–1764
Smith WL (1955) Regenerative stochastic processes. Proc R Soc (Lond) Ser. A 232:6–31
Voelkel JG, Crowley J (1984) Nonparametric inference for a class of semi-Markov processes with censored observations. Ann Statist 12:142–160
Walker S, Muliere P (1997) Beta-Stacy processes and a generalization of the Pólya-urn scheme. Ann Statist 25:1762–1780
About this article
Cite this article
Bulla, P., Muliere, P. Bayesian Nonparametric Estimation for Reinforced Markov Renewal Processes. Stat Infer Stoch Process 10, 283–303 (2007). https://doi.org/10.1007/s11203-006-9000-x
- Markov renewal processes
- Bayesian nonparametrics