Statistics and Computing

, Volume 25, Issue 1, pp 67–78 | Cite as

On a class of \(\sigma \)-stable Poisson–Kingman models and an effective marginalized sampler

Article
First Online:
Received:
Accepted:

Abstract

We investigate the use of a large class of discrete random probability measures, which is referred to as the class \(\mathcal {Q}\), in the context of Bayesian nonparametric mixture modeling. The class \(\mathcal {Q}\) encompasses both the the two-parameter Poisson–Dirichlet process and the normalized generalized Gamma process, thus allowing us to comparatively study the inferential advantages of these two well-known nonparametric priors. Apart from a highly flexible parameterization, the distinguishing feature of the class \(\mathcal {Q}\) is the availability of a tractable posterior distribution. This feature, in turn, leads to derive an efficient marginal MCMC algorithm for posterior sampling within the framework of mixture models. We demonstrate the efficacy of our modeling framework on both one-dimensional and multi-dimensional datasets.

Keywords

Bayesian nonparametrics Normalized generalized Gamma process Marginalized MCMC sampler Mixture model \(\sigma \)-Stable Poisson–Kingman model Two parameter Poisson–Dirichlet process 
This is a preview of subscription content, log in to check access

Notes

Acknowledgments

The authors are grateful to an Associate Editor and two Referees for their valuable remarks and suggestions that have led to a substantial improvement of the paper. We would also like to thank Lancelot F. James for helpful suggestions. Stefano Favaro is supported by the European Research Council (ERC) through StG N-BNP 306406. Yee Whye Teh’s research leading to these results is supported by the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007–2013) ERC Grant Agreement No. 617411. Maria Lomeli is supported by the Gatsby Charitable Foundation.

Supplementary material

11222_2014_9499_MOESM1_ESM.pdf (195 kb)
ESM (PDF 196 kb)

References

  1. Barrios, E., Lijoi, A., Nieto-Barajas, L.E., Prünster, I.: Modeling with normalized random measure mixture models. Stat. Sci. 28, 313–334 (2013)CrossRefGoogle Scholar
  2. Charalambides, C.A.: Combinatorial Methods in Discrete Distributions. Wiley-Interscience, Hoboken (2005)CrossRefMATHGoogle Scholar
  3. De Blasi, P., Favaro, S., Lijoi, A., Mena, R.H., Prun̈ster, I., Ruggiero, M.: Are Gibbs-type priors the most natural generalization of the Dirichlet process? IEEE Trans. Pattern Anal. Mach. Intell. (in press) (2013)Google Scholar
  4. De la Mata-Espinosa, P., Bosque-Sendra, J.M., Bro, R., Cuadros-Rodriguez, L.: Discriminating olive and non-olive oils using HPLC-CAD and chemometrics. Anal Bioanal Chem 399, 2083–2092 (2011)Google Scholar
  5. Escobar, M.D.: Estimating normal means with a Dirichlet process prior. J. Am. Stat. Assoc. 89, 268–277 (1994)CrossRefMATHMathSciNetGoogle Scholar
  6. Escobar, M.D., West, M.: Bayesian density estimation and inference using mixtures. J. Am. Stat. Assoc. 90, 577–588 (1995)CrossRefMATHMathSciNetGoogle Scholar
  7. Favaro, S., Lijoi, A., Prünster, I.: On the stick-breaking representation of normalized inverse Gaussian priors. Biometrika 99, 663–674 (2012)CrossRefMATHMathSciNetGoogle Scholar
  8. Favaro, S., Teh, Y.W.: MCMC for normalized random measure mixture models. Stat. Sci. 28, 335–359 (2013)CrossRefMathSciNetGoogle Scholar
  9. Favaro, S., Walker, S.G.: Slice sampling \(\sigma \)-stable Poisson–Kingman mixture models. J. Comput. Gr. Stat. 22, 830–847 (2013)CrossRefMathSciNetGoogle Scholar
  10. Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209–230 (1973)CrossRefMATHMathSciNetGoogle Scholar
  11. Gnedin, A., Pitman, J.: Exchangeable Gibbs partitions and Stirling triangles. Zap. Nauchn. Sem. S. Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 325, 83–102 (2005)Google Scholar
  12. Griffin, J.E., Walker, S.G.: Posterior simulation of normalized random measure mixtures. J. Comput. Gr. Stat. 20, 241–259 (2009)CrossRefMathSciNetGoogle Scholar
  13. Ishwaran, H., James, L.F.: Gibbs sampling methods for stick-breaking priors. J. Am. Stat. Assoc. 96, 161–173 (2001)CrossRefMATHMathSciNetGoogle Scholar
  14. James, L.F.: Poisson process partition calculus with applications to exchangeable models and Bayesian nonparametrics (preprint) (2002). arXiv:math/0205093
  15. James, L.F.: Coag–Frag duality for a class of stable Poisson–Kingman mixtures (preprint) (2010). arXiv:math/1008.2420
  16. James, L.F.: Stick-breaking PG\((\alpha ,\zeta )\)-generalized Gamma processes (preprint) (2013). arXiv:math/1308.6570
  17. James, L.F., Lijoi, A., Prünster, I.: Distributions of linear functionals of two parameter Poisson–Dirichlet random measures. Ann. Appl. Probab. 18, 521–551 (2008)CrossRefMATHMathSciNetGoogle Scholar
  18. James, L.F., Lijoi, A., Prünster, I.: Posterior analysis for normalized random measures with independent increments. Scand. J. Stat. 36, 76–97 (2009)CrossRefMATHMathSciNetGoogle Scholar
  19. Kalli, M., Griffin, J.E., Walker, S.G.: Slice sampling mixture models. Stat. Comput. 21, 93–105 (2011)CrossRefMATHMathSciNetGoogle Scholar
  20. Kingman, J.F.C.: Completely random measures. Pac. J. Math. 21, 59–78 (1967)CrossRefMATHMathSciNetGoogle Scholar
  21. Kingman, J.F.C.: Random discrete distributions. J. R. Stat. Soc. B 37, 1–22 (1975)MATHMathSciNetGoogle Scholar
  22. Lijoi, A., Mena, R.H., Prünster, I.: Hierarchical mixture modelling with normalized inverse-Gaussian priors. J. Am. Stat. Assoc. 100, 1278–1291 (2005)CrossRefMATHGoogle Scholar
  23. Lijoi, A., Mena, R.H., Prünster, I.: Controlling the reinforcement in Bayesian nonparametric mixture models. J. R. Stat. Soc. B 69, 715–740 (2007)CrossRefGoogle Scholar
  24. Lijoi, A., Prünster, I.: Models beyond the Dirichlet process. In: Hjort, N.L., Holmes, C.C., Müller, P., Walker, S.G. (eds.) Bayesian Nonparametrics, pp. 80–136. Cambridge University Press, Cambridge (2010)CrossRefGoogle Scholar
  25. Lo, A.I.: On a class of Bayesian nonparametric estimates: I. Density estimates. Ann. Stat. 12, 351–357 (1984)CrossRefMATHGoogle Scholar
  26. MacEachern, S.N.: Estimating normal means with a conjugate style Dirichlet process prior. Commun. Stat. Simul. Comput. 23, 727–741 (1994)CrossRefMATHMathSciNetGoogle Scholar
  27. Muliere, P., Tardella, L.: Approximating distributions of random functionals of Ferguson–Dirichlet priors. Can. J. Stat. 26, 283–298 (1998)CrossRefMATHMathSciNetGoogle Scholar
  28. Neal, R.M.: Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Gr. Stat. 9, 249–265 (2000)MathSciNetGoogle Scholar
  29. Neal, R.M.: Slice sampling. Ann. Stat. 31, 705–767 (2003)CrossRefMATHMathSciNetGoogle Scholar
  30. Nieto-Barajas, L.E., Prünster, I., Walker, S.G.: Normalized random measures driven by increasing additive processes. Ann. Stat. 32, 2343–2360 (2004)CrossRefMATHGoogle Scholar
  31. Papaspiliopoulos, O.: A note on posterior sampling from Dirichlet mixture models. Unpublished manuscript (2008)Google Scholar
  32. Papaspiliopoulos, O., Roberts, G.O.: Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models. Biometrika 95, 169–186 (2008)CrossRefMATHMathSciNetGoogle Scholar
  33. Perman, M., Pitman, J., Yor, M.: Size-biased sampling of Poisson point processes and excursions. Probab. Theory Relat. Fields 92, 21–39 (1992)CrossRefMATHMathSciNetGoogle Scholar
  34. Pitman, J.: Exchangeable and partially exchangeable random partitions. Probab. Theory Relat. Fields 102, 145–158 (1995)CrossRefMATHMathSciNetGoogle Scholar
  35. Pitman, J.: Some developments of the Blackwell–MacQueen urn scheme. In: Ferguson, T.S., et al. (eds.) Statistics, Probability and Game Theory: Papers in Honor of David Blackwell. Lecture Notes Monograph Series, vol. 30, pp. 245–267. IMS, Beachwood (1996a)CrossRefGoogle Scholar
  36. Pitman, J., Yor, M.: The two parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann. Probab. 25, 855–900 (1997) Google Scholar
  37. Pitman, J.: Poisson–Kingman partitions. In: Goldstein, D.R. (ed.) Science and Statistics: A Festschrift for Terry Speed. Lecture Notes Monograph Series, pp. 1–34. IMS, Beachwood (2003)Google Scholar
  38. Pitman, J.: Combinatorial stochastic processes. Ecole d’Eté de Probabilités de Saint-Flour XXXII. Lecture Notes in Mathematics N. 1875. Springer, New York (2006)Google Scholar
  39. Regazzini, E., Lijoi, A., Prünster, I.: Distributional results for means of normalized random measures with independent increments. Ann. Stat. 31, 560–585 (2002)Google Scholar
  40. Roeder, K.: Density estimation with confidence sets exemplified by superclusters and voids in the galaxies. J. Am. Stat. Assoc. 36, 45–54 (1990)Google Scholar
  41. Walker, S.G.: Sampling the Dirichlet mixture model with slices. Commun. Stat. Simul. Comput. 36, 45–54 (2007)CrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.University of Torino and Collegio Carlo AlbertoTurinItaly
  2. 2.Gatsby Computational Neuroscience UnitUniversity College LondonLondonUK
  3. 3.Department of StatisticsUniversity of OxfordOxfordUK

Personalised recommendations