Abstract
We study the construction of random discrete distributions, taking values in the infinite dimensional simplex, by means of a latent random subset of the natural numbers. The derived sequences of random weights are then used to establish a Bayesian non-parametric prior. A sufficient condition on the distribution of the random set is given, that assures the corresponding prior has full support, and taking advantage of the construction, we propose a general MCMC algorithm for density estimation purposes. This method is illustrated by building a new distribution over the space of all finite and non-empty subsets of \(\mathbb {N}\), that subsequently leads to a general class of random probability measures termed Geometric product stick-breaking process. It is shown that Geometric product stick-breaking process approximate, in distribution, Dirichlet and Geometric processes, and that the respective weights sequences have heavy tails, thus leading to very flexible mixture models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Recall that a probability kernel ψ from \({\mathcal {S}}\) into \(({\mathcal {R}},\mathcal {B}({\mathcal {R}}))\) is a function, \(\psi :\mathcal {B}({\mathcal {R}})\times {\mathcal {S}} \to [0,1]\), such that for every \(s \in {\mathcal {S}}\) fixed ψ(⋅|s) is a probability measure, and for every \(B \in \mathcal {B}({\mathcal {R}})\) fixed, ψ(B|⋅) is a measurable with respect to \(\mathcal {B}({\mathcal {S}})\) and \(\mathcal {B}([0,1])\).
References
Billingsley, P.: Convergence of Probability Measures. Wiley Series in Probability And Statistics. John Wiley and Sons Inc., New York (1968)
Bissiri, P.G., Ongaro, A.: On the topological support of species sampling priors. Electron. J. Stat. 8, 861–882 (2014)
Blackwell, D., MacQueen, J.: Ferguson distributions via Pólya urn schemes. Ann. Stat. 1, 353–355 (1973)
De Blasi, P., Martínez, A.F., Mena, R.H., Prünster, I.: On the inferential implications of decreasing weight structures in mixture models. Comput. Stat. Data Anal. 147, 106940 (2020)
Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209–230 (1973)
Frühwirth-Schnatter, S., Celeux, G., Robert, C.P.: Handbook of Mixture Analysis. Chapman and Hall/CRC, Boca Raton (2019)
Fuentes-García, R., Mena, R.H., Walker, S.G.: A new Bayesian Nonparametric mixture model. Commun. Stat. Simul. Comput. 39, 669–682 (2010)
Ghosal, S., van der Vaart, A.: Fundamentals of Nonparametric Bayesian Inference. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge (2017)
Hjort, N., Holmes, C., Müller, P., Walker, S.G.: Bayesian Nonparametrics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge (2010)
Ishwaran, H., James, L.F.: Gibbs sampling methods for stick-breaking priors. J. Am. Stat. Assoc. 96, 161–173 (2001)
James, L.F., Lijoi, A., Prünster, I.: Posterior analysis for normalized random measures with independent increments. J. R. Stat. Soc. B (Stat. Methodol. ) 36, 76–97 (2009)
Kallenberg, O.: Random Measures, Theory and Applications. Probability Theory and Stochastic Modelling, vol. 77, pp. 1–680. Springer International Publishing, Cham (2017)
Kingman, J.F.C.: Random discrete distributions (with discussion). J. R. Stat. Soc. B (Stat. Methodol.) 37, 1–22 (1975)
McCloskey, T.: A model for the distribution of individuals by species in an environment. Michigan State University Department of Statistics (1965)
Parthasarathy, K.R.: Probability Measures on Metric Spaces. Academic Press, New York (1967)
Pitman, J.: Some developments of the Blackwell-MacQueen urn scheme. In: Ferguson, T.S., Shapley, L.S., MacQueen, J.B. (eds.) Statistics Probability and Game Theory. IMS Lecture Notes Monograph Series, vol. 30, pp. 245–267 (1996)
Pitman, J.: Combinatorial stochastic processes. In: École d’été de probabilités de Saint-Flour, vol. 1875, pp. 1–260. Springer, Berlin Heidelberg (2006)
Regazzini, E., Lijoi, A., Prünster, I.: Distributional results for means of normalized random measures with independent increments. Ann. Stat. 31, 560–585 (2003)
Sethuraman, J.: A constructive definition of Dirichlet priors. Stat. Sin. 4, 639–650 (1994)
Walker, S.G.: Sampling the Dirichlet mixture model with slices. Commun. Stat. Simul. Comput. 36, 45–54 (2007)
Acknowledgements
We thank an anonymous referee for his/her careful review that led to substantial improvements, and to the project PAPIIT-UNAM IG100221. Many thanks to the CONACyT PhD scholarship program and the support of CONTEX project 2018-9B.
Appendix
The purpose of this section is to prove that under mild conditions of ψ, the mapping
is continuous with respect to the weak topology. This assures that if \({\mathbf {W}}^{(n)} = \left ({\mathbf {w}}^{(n)}_j\right )_{j \geq 1}\) converges in distribution to \({\mathbf {W}} = \left ({\mathbf {w}}_j\right )_{j \geq 1}\), and \({\boldsymbol {\Xi }}^{(n)} = \left ({\boldsymbol {\xi }}^{(n)}_j\right )_{j \geq 1}\) converges in distribution to \({\boldsymbol {\Xi }} = \left ({\boldsymbol {\xi }}_j\right )_{j \geq 1}\), then \(\sum _{j\geq 1}{\mathbf {w}}^{(n)}_{j}\,\psi \left (\cdot \,\middle |\,{\boldsymbol {\xi }}^{(n)}_j\right )\) converges weakly in distribution to ∑j≥1 w jψ(⋅∣ξ j).
Proposition A.1
Let\(({\mathcal {S}},\mathcal {B}({\mathcal {S}}))\)and\((\mathcal {R},\mathcal {B}(\mathcal {R}))\)be Polish spaces and let ψ be a probability kernel from\({\mathcal {S}}\)into\(\mathcal {R}\)such that ψ(⋅∣s n) converges weakly to ψ(⋅∣s) as s n → s in\({\mathcal {S}}\) . Then the mapping
from \(\Delta _{\infty }\times \mathcal {S}^{\infty }\) into the space of all probability measures over \((S,\mathcal {B}(\mathcal {S}))\) is continuous with respect to the weak topology.
Proof
Let W = (w 1, w 2, …), \(\left \{W^{(n)}= \left (w^{(n)}_1,w^{(n)}_2,\ldots \right )\right \}_{n \geq 1}\) be elements of Δ∞, and S = (s 1, s 2, …), \(\left \{S^{(n)}=\left (s^{(n)}_1,s^{(n)}_2,\ldots \right )_{n \geq 1}\right \}\), be elements of \({\mathcal {S}}^{\infty }\), such that \(w_j^{(n)} \to w_j\) and \(s^{(n)}_j \to s_j\), for every j ≥ 1. Define \(p^{(n)} = \sum _{j \geq 1}w^{(n)}_j\,\psi \left (\cdot \,\middle |\, s^{(n)}_j\right )\) and p =∑j≥1 w jψ(⋅∣s j). By the Portmanteau theorem (see for instance [1],[12] or [15]) it suffices to prove that for every continuous and bounded function \(f:{\mathcal {S}} \to \mathbb {R}\),
So fix a continuous and bounded function \(f: S \to \mathbb {R}\). First note that by hypothesis \(\psi \left (\cdot \,\middle |\, s^{(n)}_j\right )\) converges weakly to ψ(⋅∣s j), for every j ≥ 1, by the Portmanteau theorem this implies
thus \(w^{(n)}_j\psi \left (f\,\middle |\, s^{(n)}_j\right ) \to w_j\psi (f \mid s_j)\), for j ≥ 1. Since f is bounded, there exist M such that |f|≤ M, hence, using the fact that \(\psi \left (\cdot \,\middle |\, s^{(n)}_j\right )\) is a probability measure, we obtain
for every n, j ≥ 1. Evidently, \(Mw^{(n)}_j \to Mw_j\), and \(\sum _{j \geq 1}Mw^{(n)}_j = M = \sum _{j \geq 1}Mw_j\). Then, by general Lebesgue dominated convergence theorem, we conclude
□
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gil-Leyva, M.F. (2021). Bayesian Non-parametric Priors Based on Random Sets. In: Hernández‐Hernández, D., Leonardi, F., Mena, R.H., Pardo Millán, J.C. (eds) Advances in Probability and Mathematical Statistics. Progress in Probability, vol 79. Birkhäuser, Cham. https://doi.org/10.1007/978-3-030-85325-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-85325-9_5
Published:
Publisher Name: Birkhäuser, Cham
Print ISBN: 978-3-030-85324-2
Online ISBN: 978-3-030-85325-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)