pp 1–26 | Cite as

A general guide in Bayesian and robust Bayesian estimation using Dirichlet processes

  • Ali KarimnezhadEmail author
  • Mahmoud Zarepour


In this paper, we investigate Bayesian and robust Bayesian estimation of a wide range of parameters of interest in the context of Bayesian nonparametrics under a broad class of loss functions. Dealing with uncertainty regarding the prior, we consider the Dirichlet and the Dirichlet invariant priors, and provide explicit form of the resulting Bayes and robust Bayes estimators. Tractability of the results is supported by numerous examples of different well-known loss functions. The practical utility of the proposed Bayes and robust Bayes estimators are examined for a real data set.


Bayesian estimation Bayesian nonparametrics Dirichlet process Dirichlet invariant process 



The authors are cordially grateful to the Editor in Chief and two anonymous reviewers for raising several helpful comments and suggestions which led to a substantial improvement in the quality of our work. Research of the second author was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) (RGPIN-2018-04008).

Compliance with ethical standards

Conflict of interest

The author declares that there is no conflict of interest.


  1. Al-Labadi L, Evans M (2016) Prior based model checking. arXiv preprint. arXiv:1606.08106
  2. Al-Labadi L, Evans M (2017) Optimal robustness results for relative belief inferences and the relationship to prior-data conflict. Bayesian Anal 12(3):705–728MathSciNetzbMATHCrossRefGoogle Scholar
  3. Al-Labadi L, Zarepour M (2013) On asymptotic properties and almost sure approximation of the normalized inverse-Gaussian process. Bayesian Anal 8(3):553–568MathSciNetzbMATHCrossRefGoogle Scholar
  4. Antoniak CE (1974) Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann Stat 2(6):1152–1174MathSciNetzbMATHCrossRefGoogle Scholar
  5. Arias-Nicolás JP, Martín J, Ruggeri F, Suárez-Llorens A (2009) Optimal actions in problems with convex loss functions. Int J Approx Reason 50(2):303–314MathSciNetzbMATHCrossRefGoogle Scholar
  6. Banerjee A, Merugu S, Dhillon IS, Ghosh J (2005) Clustering with Bregman divergences. J Mach Learn Res 6:1705–1749MathSciNetzbMATHGoogle Scholar
  7. Basu S (2000) Bayesian robustness and Bayesian nonparametrics. In: Insua DR, Ruggeri F (eds) Robust Bayesian analysis. Springer, New York, pp 223–240zbMATHCrossRefGoogle Scholar
  8. Benavoli A, Mangili F, Ruggeri F, Zaffalon M (2015) Imprecise Dirichlet process with application to the hypothesis test on the probability that \(X\le Y\). J Stat Theory Pract 9(3):658–684MathSciNetzbMATHCrossRefGoogle Scholar
  9. Berger JO (1990) Robust Bayesian analysis: sensitivity to the prior. J Stat Plan Inference 25(3):303–328MathSciNetzbMATHCrossRefGoogle Scholar
  10. Berger JO (2013) Statistical decision theory and Bayesian analysis, 2nd edn. Springer, New YorkGoogle Scholar
  11. Berger JO, Moreno E, Pericchi LR, Bayarri MJ, Bernardo JM, Cano JA, De la Horra J, Martín J, Ríos-Insúa D, Betrò B et al (1994) An overview of robust Bayesian analysis. Test 3(1):5–124MathSciNetCrossRefGoogle Scholar
  12. Bose S (2017) Robustness in Bayesian nonparametrics. Int J Approx Reason 82:161–169MathSciNetzbMATHCrossRefGoogle Scholar
  13. Bregman LM (1967) The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput Math Math Phys 7(3):200–217MathSciNetzbMATHCrossRefGoogle Scholar
  14. Brown L (1968) Inadmissibility of the usual estimators of scale parameters in problems with unknown location and scale parameters. Ann Math Stat 39(1):29–48zbMATHCrossRefGoogle Scholar
  15. Cagno E, Caron F, Mancini M, Ruggeri F (2000) Using AHP in determining the prior distributions on gas pipeline failures in a robust Bayesian approach. Reliab Eng Syst Saf 67(3):275–284CrossRefGoogle Scholar
  16. Calabria R, Pulcini G (1994) An engineering approach to Bayes estimation for the Weibull distribution. Microelectron Reliab 34(5):789–802zbMATHCrossRefGoogle Scholar
  17. Carlton MA (1999) Applications of the two-parameter Poisson–Dirichlet distribution. Ph.D. thesis, Department of Statistics, University of California, Los AngelesGoogle Scholar
  18. Dalal SR (1979) Dirichlet invariant processes and applications to nonparametric estimation of symmetric distribution functions. Stoch Process Their Appl 9(1):99–107MathSciNetzbMATHCrossRefGoogle Scholar
  19. Ferguson TS (1973) A Bayesian analysis of some nonparametric problems. Ann Stat 1(2):209–230MathSciNetzbMATHCrossRefGoogle Scholar
  20. Ferguson TS, Klass MJ (1972) A representation of independent increment processes without Gaussian components. Ann Math Stat 43(5):1634–1643MathSciNetzbMATHCrossRefGoogle Scholar
  21. Févotte C, Bertin N, Durrieu JL (2009) Nonnegative matrix factorization with the Itakura-Saito divergence: with application to music analysis. Neural Comput 21(3):793–830zbMATHCrossRefGoogle Scholar
  22. Golparver L, Karimnezhad A, Parsian A (2013) Optimal rules and robust Bayes estimation of a gamma scale parameter. Metrika 76(5):595–622MathSciNetzbMATHCrossRefGoogle Scholar
  23. Gupta AK, Nadarajah S (2004) Mathematical properties of the Beta distribution. In: Gupta AK, Nadarajah S (eds) Handbook of beta distribution and its applications. Marcel Dekker Inc., New York, pp 33–53zbMATHCrossRefGoogle Scholar
  24. Hjort NL (1990) Nonparametric Bayes estimators based on beta processes in models for life history data. Ann Stat 18(3):1259–1294MathSciNetzbMATHCrossRefGoogle Scholar
  25. Hosseini R, Zarepour M (2018) A note on Bayesian nonparametric inference for spherically symmetric distribution. arXiv preprint. arXiv:1807.11066v2
  26. Ishwaran H, James LF (2001) Gibbs sampling methods for stick-breaking priors. J Am Stat Assoc 96(453):161–173MathSciNetzbMATHCrossRefGoogle Scholar
  27. Ishwaran H, Zarepour M (2000) Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models. Biometrika 87(2):371–390MathSciNetzbMATHCrossRefGoogle Scholar
  28. Jagers P (1974) Aspects of random measures and point processes. Adv Probab Relat Top 3:179–239MathSciNetzbMATHGoogle Scholar
  29. James LF, Lijoi A, Prünster I (2006) Conjugacy as a distinctive feature of the Dirichlet process. Scand J Stat 33(1):105–120MathSciNetzbMATHCrossRefGoogle Scholar
  30. James W, Stein C (1992) Estimation with quadratic loss. In: Kotz S, Johnson NL (eds) Breakthroughs in statistics. Springer Series in Statistics (Perspectives in Statistics). Springer, New York, pp 443–460CrossRefGoogle Scholar
  31. Jang GH, Lee J, Lee S (2010) Posterior consistency of species sampling priors. Stat Sinica 20:581–593MathSciNetzbMATHGoogle Scholar
  32. Kalbfleisch JD (1978) Non-parametric Bayesian analysis of survival time data. J R Stat Soc Ser B (Methodol) 40(2):214–221MathSciNetzbMATHGoogle Scholar
  33. Karimnezhad A, Parsian A (2014) Robust Bayesian methodology with applications in credibility premium derivation and future claim size prediction. AStA Adv Stat Anal 98(3):287–303MathSciNetzbMATHCrossRefGoogle Scholar
  34. Karimnezhad A, Parsian A (2018) Most stable sample size determination in clinical trials. Stat Methods Appl. MathSciNetCrossRefzbMATHGoogle Scholar
  35. Karimnezhad A, Parsian A (2019) Bayesian and robust Bayesian analysis in a general setting. Commun Stat Theory Methods 48(15):3899–3920MathSciNetCrossRefGoogle Scholar
  36. Karimnezhad A, Lucas PJ, Parsian A (2017) Constrained parameter estimation with uncertain priors for Bayesian networks. Electron J Stat 11(2):4000–4032MathSciNetzbMATHCrossRefGoogle Scholar
  37. Kiapour A, Nematollahi N (2011) Robust Bayesian prediction and estimation under a squared log error loss function. Stat Probab Lett 81(11):1717–1724MathSciNetzbMATHCrossRefGoogle Scholar
  38. Lehmann EL, Casella G (1998) Theory of point estimation. Springer, New YorkzbMATHGoogle Scholar
  39. Lo AY (1984) On a class of Bayesian nonparametric estimates: I. Density estimates. Ann Stat 12(1):351–357MathSciNetzbMATHCrossRefGoogle Scholar
  40. Lijoi A, Mena RH, Prünster I (2005) Hierarchical mixture modeling with normalized inverse-Gaussian priors. J Am Stat Assoc 100(472):1278–1291MathSciNetzbMATHCrossRefGoogle Scholar
  41. Makov UE (1995) Loss robustness via Fisher-weighted squared-error loss function. Insur Math Econ 16(1):1–6MathSciNetzbMATHCrossRefGoogle Scholar
  42. Masi I, Tran AT, Hassner T, Leksut JT, Medioni G (2016) Do we really need to collect millions of faces for effective face recognition? In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision, ECCV. Lecture Notes in Computer Science, Col 9909. Springer, ChamGoogle Scholar
  43. Müller P, Quintana FA, Jara A, Hanson T (2016) Bayesian nonparametric data analysis. Springer, New YorkzbMATHGoogle Scholar
  44. Norstrom JG (1996) The use of precautionary loss functions in risk analysis. IEEE Trans Reliab 45(3):400–403CrossRefGoogle Scholar
  45. Parsian A, Kirmani S (2002) Estimation under LINEX loss function. In: Ullah A, Wan ATK, Chaturvedi A (eds) Handbook of applied econometrics and statistical inference. Marcel Dekker Inc., New York, pp 53–75Google Scholar
  46. Phadia EG (1973) Minimax estimation of a cumulative distribution function. Ann Stat 1(6):1149–1157MathSciNetzbMATHCrossRefGoogle Scholar
  47. Phadia EG (2016) Prior processes and their applications, 2nd edn. Springer, ChamzbMATHCrossRefGoogle Scholar
  48. Pitman J (1996) Blackwell-macqueen urn scheme. Stat Probab Game Theory Pap Honor David Blackwell 30:245MathSciNetCrossRefGoogle Scholar
  49. Pitman J, Yor M (1997) The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann Probab 2:855–900MathSciNetzbMATHGoogle Scholar
  50. Regazzini E, Lijoi A, Prünster I (2003) Distributional results for means of normalized random measures with independent increments. Ann Stat 31(2):560–585MathSciNetzbMATHCrossRefGoogle Scholar
  51. Ruggeri F (2010) Nonparametric Bayesian robustness. Chil J Stat 2:51–68MathSciNetzbMATHGoogle Scholar
  52. Ruggeri F (2014) On some optimal Bayesian nonparametric rules for estimating distribution functions. Econ Rev 33(1–4):289–304MathSciNetCrossRefGoogle Scholar
  53. Ríos Insua D, Ruggeri F, Wiper M (2012) Bayesian analysis of stochastic process models. Wiley, ChichesterzbMATHCrossRefGoogle Scholar
  54. Sethuraman J (1994) A constructive definition of Dirichlet priors. Stat Sinica 4:639–650MathSciNetzbMATHGoogle Scholar
  55. Soliman AA (2005) Estimation of parameters of life from progressively censored data using Burr-xii model. IEEE Trans Reliab 54(1):34–42CrossRefGoogle Scholar
  56. Tanner MA, Wong WH (1987) The calculation of posterior distributions by data augmentation. J Am Stat Assoc 82:528–550MathSciNetzbMATHCrossRefGoogle Scholar
  57. Teh YW (2006) A hierarchical Bayesian language model based on Pitman–Yor processes. In: Proceedings of the 21st international conference on computational linguistics and the 44th annual meeting of the association for computational linguistics. Association for Computational Linguistics pp 985–992Google Scholar
  58. Thibaux R, Jordan MI (2007) Hierarchical beta processes and the Indian buffet process. In artificial intelligence and statistics. In: Meila M, Shen X (eds.) Proceedings of the 10th conference on artificial intelligence and statistics. Society for Artificial Intelligence and Statistics, pp 564–571Google Scholar
  59. van Dyk DA, Meng XL (2001) The art of data augmentation. J Comput Graph Stat 10(1):1–50MathSciNetCrossRefGoogle Scholar
  60. Varian HR (1975) A Bayesian approach to real estate assessment. Studies in Bayesian Econometric and Statistics in Honor of Leonard J. Savage 195–208Google Scholar
  61. Walker S, Muliere P (1997) Beta-Stacy processes and a generalization of the Pólya-urn scheme. Ann Stat 25(4):1762–1780zbMATHCrossRefGoogle Scholar
  62. Zarepour M, Al-Labadi L (2012) On a rapid simulation of the Dirichlet process. Stat Probab Lett 82(5):916–924MathSciNetzbMATHCrossRefGoogle Scholar
  63. Zellner A (1986) Bayesian estimation and prediction using asymmetric loss functions. J Am Stat Assoc 81(394):446–451MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Biochemistry, Microbiology and Immunology, Faculty of MedicineUniversity of OttawaOttawaCanada
  2. 2.Department of StatisticsUniversity of OttawaOttawaCanada

Personalised recommendations