Strongly consistent model selection for densities

Abstract

Let f be an unknown multivariate density belonging to a set of densities $\mathcal{F}_{k^{*}}$ of finite associated Vapnik–Chervonenkis dimension, where the complexity k ^* is unknown, and ℱ _k ⊂ℱ _k+1 for all k. Given an i.i.d. sample of size n drawn from f, this article presents a density estimate $\hat{f}_{K_{n}}$ yielding almost sure convergence of the estimated complexity K _n to the true but unknown k ^* and with the property $\mathbf{E}\{\int|\hat{f}_{K_{n}}-f|\}=\mbox{O}(1/\sqrt{n}\,)$ . The methodology is inspired by the combinatorial tools developed in Devroye and Lugosi (Combinatorial methods in density estimation. Springer, New York, 2001) and it includes a wide range of density models, such as mixture models and exponential families.