Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Bayesian Nonparametric Estimation for Reinforced Markov Renewal Processes

  • 75 Accesses

  • 3 Citations


Starting from the definitions and the properties of reinforced renewal processes and reinforced Markov renewal processes, we characterize, via exchangeability and de Finetti’s representation theorem, a prior that consists of a family of Dirichlet distributions on the space of Markov transition matrices and beta-Stacy processes on distribution functions. Then, we show that this family is conjugate and give some estimate results.

This is a preview of subscription content, log in to check access.



renewal process


Markov renewal process


semi-Markov process


reinforced Markov renewal process


reinforced semi-Markov process


  1. Brock DB (1973) Bayesian analysis for Markov renewal process. Metron 29:185–198

  2. Çinlar E (1975) Markov renewal theory: a survey. Manag Sci J Instit Manage Sci 21:727–752

  3. Duncan GT (1990) Bayes estimation in the Markov renewal process. Bayesian and Likelihood Methods in Statistics and Econometrics: Essays in Honor of George A. Barnard. North-Holland, Amsterdam, pp 265–275

  4. Gill RD (1980) Nonparametric estimation based on censored observations of a Markov renewal process. Zeitschrift Wahrscheinlichkeitstheorie Verwandte Gebiete [Became: @J(ProbTher)] 53:97–116

  5. Gill RD (1994) Lectures on survival analysis. In: Bernard P (eds) Lectures on probability theory (Ecole d’été de Probabilité de Saint-Flour XXII—1992). Springer, Berlin

  6. Gill RD, Johansen S (1990) A survey of product-integration with a view toward application in survival analysis. Ann Stat 18:1501–1555

  7. Greenwood PE, Wefelmeyer W (1996) Empirical estimators for semi-Markov processes. Math Methods Statist 5:299–315

  8. Hjort NL (1990) Nonparametric bayes estimators based on beta processes in models for life history data. Ann Statis 18(3):1259–1294

  9. Jain RK (1990) Statistical estimation for semi-Markov models. Metron 48:223–242

  10. Lagakos SW, Sommer CJ, Zelen M (1978) Semi-Markov models for partially censored data. Biometrika 65:311–318

  11. Lévy P (1954a) Processus semi-Markoviens. Proc Int Congr Math 3:416–426

  12. Lévy P (1954b) Systèmes semi-Markoviens à au plus une infinité dénombrable d’états possibles’. Proc Int Congr Math 2:294

  13. Moore EH, Pyke R (1968) Estimation of the transition distributions of a Markov renewal process. Ann Instit Statist, Math 20:411–424

  14. Muliere P, Secchi P, Walker SG (2000) Urn schemes and reinforced random walks. Stochastic Processes Appl 88(1):59–78

  15. Muliere P, Secchi P, Walker SG (2003) Reinforced random processes in continuous time. Stochastic Processes Appl 104:117–130

  16. Phelan MJ (1990a) Bayes estimation from a Markov renewal process. Ann Statist 18:603–616

  17. Phelan MJ (1990b) Estimating the transition probabilities from censored Markov renewal processes. Statist Probab Lett 10:43–47

  18. Pyke R (1961a) Markov renewal processes: definitions and preliminary properties. Ann Math Statist 32:1231–1242

  19. Pyke R (1961b) Markov renewal processes with finitely many states. Ann Math Statist 32:1243–1259

  20. Pyke R, Schaufele R (1964) Limit theorems for Markov renewal processes. Ann Math Statist 35:1746–1764

  21. Smith WL (1955) Regenerative stochastic processes. Proc R Soc (Lond) Ser. A 232:6–31

  22. Voelkel JG, Crowley J (1984) Nonparametric inference for a class of semi-Markov processes with censored observations. Ann Statist 12:142–160

  23. Walker S, Muliere P (1997) Beta-Stacy processes and a generalization of the Pólya-urn scheme. Ann Statist 25:1762–1780

Download references

Author information

Correspondence to Paolo Bulla.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bulla, P., Muliere, P. Bayesian Nonparametric Estimation for Reinforced Markov Renewal Processes. Stat Infer Stoch Process 10, 283–303 (2007). https://doi.org/10.1007/s11203-006-9000-x

Download citation


  • Markov renewal processes
  • Estimation
  • Bayesian nonparametrics

AMS Classification

  • 62M05
  • 62G05