Advertisement

Statistics and Computing

, Volume 27, Issue 1, pp 3–17 | Cite as

A moment-matching Ferguson & Klass algorithm

  • Julyan Arbel
  • Igor Prünster
Article

Abstract

Completely random measures (CRM) represent the key building block of a wide variety of popular stochastic models and play a pivotal role in modern Bayesian Nonparametrics. The popular Ferguson & Klass representation of CRMs as a random series with decreasing jumps can immediately be turned into an algorithm for sampling realizations of CRMs or more elaborate models involving transformed CRMs. However, concrete implementation requires to truncate the random series at some threshold resulting in an approximation error. The goal of this paper is to quantify the quality of the approximation by a moment-matching criterion, which consists in evaluating a measure of discrepancy between actual moments and moments based on the simulation output. Seen as a function of the truncation level, the methodology can be used to determine the truncation level needed to reach a certain level of precision. The resulting moment-matching Ferguson & Klass algorithm is then implemented and illustrated on several popular Bayesian nonparametric models.

Keywords

Bayesian Nonparametrics Completely random measures Ferguson & Klass algorithm Moment-matching Normalized random measures Posterior sampling 

Notes

Acknowledgments

The authors are grateful to an Associate Editor and two anonymous Referees for valuable comments and suggestions.

References

  1. Argiento, R., Bianchini, I., Guglielmi, A.: A priori truncation method for posterior sampling from homogeneous normalized completely random measure mixture models. preprint. arXiv:1507.04528 (2015)
  2. Argiento, R., Bianchini, I., Guglielmi, A.: A blocked Gibbs sampler for NGG-mixture models via a priori truncation. Stat. Comput. 26(3), 641–661 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  3. Barrios, E., Lijoi, A., Nieto-Barajas, L.E., Prünster, I.: Modeling with normalized random measure mixture models. Stat. Sci. 28(3), 313–334 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  4. Brix, A.: Generalized gamma measures and shot-noise Cox processes. Adv. Appl. Probab. 31, 929–953 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  5. Burden, R., Faires, J.: Numerical Analysis. PWS Publishing Company, Boston (1993)zbMATHGoogle Scholar
  6. Campbell, T., Huggins, J., Broderick, T., How, J.: Truncated completely random measures. In: Bayesian Nonparametrics, the Next Generation (NIPS workshop) (2015)Google Scholar
  7. Cont, R., Tankov, P.: Financial Modelling with Jump Processes. Chapman & Hall / CRC Press, London (2008)zbMATHGoogle Scholar
  8. Daley, D.J., Vere-Jones, D.: An Introduction to the Theory of Point Processes. Vol. II. General Theory and Structure. Probability and Its Applications. Springer, New York (2008)Google Scholar
  9. De Blasi, P., Favaro, S., Muliere, P.: A class of neutral to the right priors induced by superposition of beta processes. J. Stat. Plan. Inference 140(6), 1563–1575 (2010)Google Scholar
  10. De Blasi, P., Lijoi, A., Prünster, I.: An asymptotic analysis of a class of discrete nonparametric priors. Stat. Sin. 23, 1299–1322 (2012)Google Scholar
  11. Doshi, F., Miller, K., Gael, J. V., and Teh, Y. W.: Variational inference for the Indian buffet process. In: International Conference on Artificial Intelligence and Statistics, pp. 137–144 (2009)Google Scholar
  12. Epifani, I., Lijoi, A., Prünster, I.: Exponential functionals and means of neutral-to-the-right priors. Biometrika 90(4), 791–808 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  13. Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1(2), 209–230 (1973)MathSciNetCrossRefzbMATHGoogle Scholar
  14. Ferguson, T.S.: Prior distributions on spaces of probability measures. Ann. Stat. 2(4), 615–629 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  15. Ferguson, T.S., Klass, M.J.: A representation of independent increment processes without Gaussian components. Ann. Math. Stat. 43(5), 1634–1643 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  16. Ghahramani, Z., Griffiths, T. L.: Infinite latent feature models and the Indian buffet process. In: Advances in Neural Information Processing Systems 18 (NIPS 2005), pp. 475–482 (2005)Google Scholar
  17. Griffin, J.E.: An adaptive truncation method for inference in Bayesian nonparametric models. Stat. Comput. 26(1–2), 423–441 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  18. Griffin, J.E., Walker, S.G.: Posterior simulation of normalized random measure mixtures. J. Comput. Graph. Stat. 20(1), 241–259 (2011)MathSciNetCrossRefGoogle Scholar
  19. Hjort, N.L.: Nonparametric bayes estimators based on beta processes in models for life history data. Ann. Stat. 18(3), 1259–1294 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  20. Ishwaran, H., James, L.: Gibbs sampling methods for stick-breaking priors. J. Am. Stat. Assoc. 96(453), 161–173 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  21. James, L.F., Lijoi, A., Prünster, I.: Conjugacy as a distinctive feature of the Dirichlet process. Scand. J. Stat. 33(1), 105–120 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  22. James, L.F., Lijoi, A., Prünster, I.: Posterior analysis for normalized random measures with independent increments. Scand. J. Stat. 36(1), 76–97 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  23. Jordan, M.I.: Hierarchical models, nested models and completely random measures. In: Chen, M.-H., Müller, P., Sun, D., Ye, K., Dey, D. (eds.) Frontiers of Statistical Decision Making and Bayesian Analysis: in Honor of James O. Berger, pp. 207–218. Springer, New York (2010)Google Scholar
  24. Lijoi, A., Mena, R.H., Prünster, I.: Controlling the reinforcement in Bayesian non-parametric mixture models. J. R Stat. Soc. B 69(4), 715–740 (2007)MathSciNetCrossRefGoogle Scholar
  25. Lijoi, A., Prünster, I.: Models beyond the Dirichlet process. In: Hjort, N.L., Holmes, C.C., Müller, P., Walker, S.G. (eds.) Bayesian Nonparametrics, pp. 80–136. Cambridge University Press, Cambridge (2010)CrossRefGoogle Scholar
  26. Nieto-Barajas, L.E., Prünster, I., Walker, S.G.: Normalized random measures driven by increasing additive processes. Ann. Stat. 32(6), 2343–2360 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  27. Nieto-Barajas, L.E.: Bayesian semiparametric analysis of short- and long-term hazard ratios with covariates. Comput. Stat. Data Anal. 71, 477–490 (2014)MathSciNetCrossRefGoogle Scholar
  28. Nieto-Barajas, L.E., Prünster, I.: A sensitivity analysis for Bayesian nonparametric density estimators. Stat. Sin. 19(2), 685 (2009)MathSciNetzbMATHGoogle Scholar
  29. Nieto-Barajas, L.E., Walker, S.G.: Markov beta and gamma processes for modelling hazard rates. Scand. J. Stat. 29(3), 413–424 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  30. Nieto-Barajas, L.E., Walker, S.G.: Bayesian nonparametric survival analysis via Lévy driven Markov processes. Stat. Sin. 14(4), 1127–1146 (2004)zbMATHGoogle Scholar
  31. Orbanz, P., and Williamson, S.: Unit-rate poisson representations of completely random measures. Technical report (2012)Google Scholar
  32. Paisley, J. W., Blei, D. M., and Jordan, M. I.: Stick-breaking beta processes and the Poisson process. In: International Conference on Artificial Intelligence and Statistics, pp. 850–858 (2012)Google Scholar
  33. Regazzini, E., Lijoi, A., Prünster, I.: Distributional results for means of normalized random measures with independent increments. Ann. Stat. 31(2), 560–585 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  34. Rosiński, J.: Series representations of Lévy processes from the perspective of point processes. In: Barndorff-Nielsen, O., Mikosch, T., Resnick, S.I. (eds.) Lévy Processes, pp. 401–415. Birkhäuser Boston, Boston, MA (2001)Google Scholar
  35. Teh, Y. W., and Görür, D.: Indian buffet processes with power-law behavior. In: Advances in Neural Information Processing Systems 18 (NIPS 2009), pp. 1838–1846 (2009)Google Scholar
  36. Thibaux, R., and Jordan, M. I.: Hierarchical beta processes and the Indian buffet process. In: International Conference on Artificial Intelligence and Statistics, pp. 564–571 (2007)Google Scholar
  37. Walker, S.G., Damien, P.: Miscellanea. Representations of Lévy processes without Gaussian components. Biometrika 87(2), 477–483 (2000)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Department of Decision SciencesBIDSA and IGIER, Bocconi UniversityMilanItaly
  2. 2.Collegio Carlo AlbertoMoncalieriItaly

Personalised recommendations