Skip to main content
SpringerLink
Log in

On a class of \(\sigma \)-stable Poisson–Kingman models and an effective marginalized sampler

Statistics and Computing volume 25, pages 67–78 (2015)Cite this article

Abstract

We investigate the use of a large class of discrete random probability measures, which is referred to as the class \(\mathcal {Q}\), in the context of Bayesian nonparametric mixture modeling. The class \(\mathcal {Q}\) encompasses both the the two-parameter Poisson–Dirichlet process and the normalized generalized Gamma process, thus allowing us to comparatively study the inferential advantages of these two well-known nonparametric priors. Apart from a highly flexible parameterization, the distinguishing feature of the class \(\mathcal {Q}\) is the availability of a tractable posterior distribution. This feature, in turn, leads to derive an efficient marginal MCMC algorithm for posterior sampling within the framework of mixture models. We demonstrate the efficacy of our modeling framework on both one-dimensional and multi-dimensional datasets.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. This dataset is included in the MASS package in the R statistical computing environment.

  2. This dataset can be obtained from the University of Copenhagen, Department of Food Science repository of public datasets for multivariate analysis: http://www.models.life.ku.dk/oliveoil.

References

  • Barrios, E., Lijoi, A., Nieto-Barajas, L.E., Prünster, I.: Modeling with normalized random measure mixture models. Stat. Sci. 28, 313–334 (2013)

    Article  Google Scholar 

  • Charalambides, C.A.: Combinatorial Methods in Discrete Distributions. Wiley-Interscience, Hoboken (2005)

    Book  MATH  Google Scholar 

  • De Blasi, P., Favaro, S., Lijoi, A., Mena, R.H., Prun̈ster, I., Ruggiero, M.: Are Gibbs-type priors the most natural generalization of the Dirichlet process? IEEE Trans. Pattern Anal. Mach. Intell. (in press) (2013)

  • De la Mata-Espinosa, P., Bosque-Sendra, J.M., Bro, R., Cuadros-Rodriguez, L.: Discriminating olive and non-olive oils using HPLC-CAD and chemometrics. Anal Bioanal Chem 399, 2083–2092 (2011)

  • Escobar, M.D.: Estimating normal means with a Dirichlet process prior. J. Am. Stat. Assoc. 89, 268–277 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  • Escobar, M.D., West, M.: Bayesian density estimation and inference using mixtures. J. Am. Stat. Assoc. 90, 577–588 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  • Favaro, S., Lijoi, A., Prünster, I.: On the stick-breaking representation of normalized inverse Gaussian priors. Biometrika 99, 663–674 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  • Favaro, S., Teh, Y.W.: MCMC for normalized random measure mixture models. Stat. Sci. 28, 335–359 (2013)

    Article  MathSciNet  Google Scholar 

  • Favaro, S., Walker, S.G.: Slice sampling \(\sigma \)-stable Poisson–Kingman mixture models. J. Comput. Gr. Stat. 22, 830–847 (2013)

    Article  MathSciNet  Google Scholar 

  • Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209–230 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  • Gnedin, A., Pitman, J.: Exchangeable Gibbs partitions and Stirling triangles. Zap. Nauchn. Sem. S. Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 325, 83–102 (2005)

  • Griffin, J.E., Walker, S.G.: Posterior simulation of normalized random measure mixtures. J. Comput. Gr. Stat. 20, 241–259 (2009)

    Article  MathSciNet  Google Scholar 

  • Ishwaran, H., James, L.F.: Gibbs sampling methods for stick-breaking priors. J. Am. Stat. Assoc. 96, 161–173 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  • James, L.F.: Poisson process partition calculus with applications to exchangeable models and Bayesian nonparametrics (preprint) (2002). arXiv:math/0205093

  • James, L.F.: Coag–Frag duality for a class of stable Poisson–Kingman mixtures (preprint) (2010). arXiv:math/1008.2420

  • James, L.F.: Stick-breaking PG\((\alpha ,\zeta )\)-generalized Gamma processes (preprint) (2013). arXiv:math/1308.6570

  • James, L.F., Lijoi, A., Prünster, I.: Distributions of linear functionals of two parameter Poisson–Dirichlet random measures. Ann. Appl. Probab. 18, 521–551 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  • James, L.F., Lijoi, A., Prünster, I.: Posterior analysis for normalized random measures with independent increments. Scand. J. Stat. 36, 76–97 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  • Kalli, M., Griffin, J.E., Walker, S.G.: Slice sampling mixture models. Stat. Comput. 21, 93–105 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  • Kingman, J.F.C.: Completely random measures. Pac. J. Math. 21, 59–78 (1967)

    Article  MATH  MathSciNet  Google Scholar 

  • Kingman, J.F.C.: Random discrete distributions. J. R. Stat. Soc. B 37, 1–22 (1975)

    MATH  MathSciNet  Google Scholar 

  • Lijoi, A., Mena, R.H., Prünster, I.: Hierarchical mixture modelling with normalized inverse-Gaussian priors. J. Am. Stat. Assoc. 100, 1278–1291 (2005)

    Article  MATH  Google Scholar 

  • Lijoi, A., Mena, R.H., Prünster, I.: Controlling the reinforcement in Bayesian nonparametric mixture models. J. R. Stat. Soc. B 69, 715–740 (2007)

    Article  Google Scholar 

  • Lijoi, A., Prünster, I.: Models beyond the Dirichlet process. In: Hjort, N.L., Holmes, C.C., Müller, P., Walker, S.G. (eds.) Bayesian Nonparametrics, pp. 80–136. Cambridge University Press, Cambridge (2010)

    Chapter  Google Scholar 

  • Lo, A.I.: On a class of Bayesian nonparametric estimates: I. Density estimates. Ann. Stat. 12, 351–357 (1984)

    Article  MATH  Google Scholar 

  • MacEachern, S.N.: Estimating normal means with a conjugate style Dirichlet process prior. Commun. Stat. Simul. Comput. 23, 727–741 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  • Muliere, P., Tardella, L.: Approximating distributions of random functionals of Ferguson–Dirichlet priors. Can. J. Stat. 26, 283–298 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • Neal, R.M.: Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Gr. Stat. 9, 249–265 (2000)

    MathSciNet  Google Scholar 

  • Neal, R.M.: Slice sampling. Ann. Stat. 31, 705–767 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  • Nieto-Barajas, L.E., Prünster, I., Walker, S.G.: Normalized random measures driven by increasing additive processes. Ann. Stat. 32, 2343–2360 (2004)

    Article  MATH  Google Scholar 

  • Papaspiliopoulos, O.: A note on posterior sampling from Dirichlet mixture models. Unpublished manuscript (2008)

  • Papaspiliopoulos, O., Roberts, G.O.: Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models. Biometrika 95, 169–186 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  • Perman, M., Pitman, J., Yor, M.: Size-biased sampling of Poisson point processes and excursions. Probab. Theory Relat. Fields 92, 21–39 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  • Pitman, J.: Exchangeable and partially exchangeable random partitions. Probab. Theory Relat. Fields 102, 145–158 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  • Pitman, J.: Some developments of the Blackwell–MacQueen urn scheme. In: Ferguson, T.S., et al. (eds.) Statistics, Probability and Game Theory: Papers in Honor of David Blackwell. Lecture Notes Monograph Series, vol. 30, pp. 245–267. IMS, Beachwood (1996a)

    Chapter  Google Scholar 

  • Pitman, J., Yor, M.: The two parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann. Probab. 25, 855–900 (1997)

  • Pitman, J.: Poisson–Kingman partitions. In: Goldstein, D.R. (ed.) Science and Statistics: A Festschrift for Terry Speed. Lecture Notes Monograph Series, pp. 1–34. IMS, Beachwood (2003)

  • Pitman, J.: Combinatorial stochastic processes. Ecole d’Eté de Probabilités de Saint-Flour XXXII. Lecture Notes in Mathematics N. 1875. Springer, New York (2006)

  • Regazzini, E., Lijoi, A., Prünster, I.: Distributional results for means of normalized random measures with independent increments. Ann. Stat. 31, 560–585 (2002)

    Google Scholar 

  • Roeder, K.: Density estimation with confidence sets exemplified by superclusters and voids in the galaxies. J. Am. Stat. Assoc. 36, 45–54 (1990)

    Google Scholar 

  • Walker, S.G.: Sampling the Dirichlet mixture model with slices. Commun. Stat. Simul. Comput. 36, 45–54 (2007)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The authors are grateful to an Associate Editor and two Referees for their valuable remarks and suggestions that have led to a substantial improvement of the paper. We would also like to thank Lancelot F. James for helpful suggestions. Stefano Favaro is supported by the European Research Council (ERC) through StG N-BNP 306406. Yee Whye Teh’s research leading to these results is supported by the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007–2013) ERC Grant Agreement No. 617411. Maria Lomeli is supported by the Gatsby Charitable Foundation.

Author information

Authors and Affiliations

  1. University of Torino and Collegio Carlo Alberto, Turin, Italy

    S. Favaro

  2. Gatsby Computational Neuroscience Unit, University College London, London, UK

    M. Lomeli

  3. Department of Statistics, University of Oxford, Oxford, UK

    Y. W. Teh

Authors
  1. S. Favaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Lomeli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Y. W. Teh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Y. W. Teh.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM (PDF 196 kb)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Favaro, S., Lomeli, M. & Teh, Y.W. On a class of \(\sigma \)-stable Poisson–Kingman models and an effective marginalized sampler. Stat Comput 25, 67–78 (2015). https://doi.org/10.1007/s11222-014-9499-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-014-9499-4

Keywords

search

Navigation