The Generalized Cross Entropy Method, with Applications to Probability Density Estimation
Nonparametric density estimation aims to determine the sparsest model that explains a given set of empirical data and which uses as few assumptions as possible. Many of the currently existing methods do not provide a sparse solution to the problem and rely on asymptotic approximations. In this paper we describe a framework for density estimation which uses information-theoretic measures of model complexity with the aim of constructing a sparse density estimator that does not rely on large sample approximations. The effectiveness of the approach is demonstrated through an application to some well-known density estimation test cases.
KeywordsCross entropy Information theory Monte Carlo simulation Statistical modeling Kernel smoothing Functional optimization Bandwidth selection Calculus of variations
AMS 2000 Subject ClassificationsPrimary 94A17 60K35; Secondary 68Q32 93E14
Unable to display preview. Download preview PDF.
- Biernacki C, Celeux C, Govaert G (1998) Assessing a mixture model for clustering with the integrated classification likelihood. Technical report no. 3521. Rhône-Alpes, INRIAGoogle Scholar
- Botev ZI (2005) Stochastic methods for optimization and machine learning. ePrintsUQ, BSc (Hons) thesis, Department of Mathematics, School of Physical Sciences, The University of Queensland. http://eprint.uq.edu.au/archive/00003377/
- Kapur JN, Kesavan HK (1992) Entropy optimization principles with applications. Academic, New YorkGoogle Scholar
- Mclachlan GJ, Peel D (1997) Contribution to the discussion of paper by S. Richardson and P. J. Green. J R Stat Soc Ser B Stat Methodol 59:779–780Google Scholar
- Mukherjee S, Vapnik V (1999) Multivariate density estimation: a support vector machine approach. Massachusetts Institute of Technology. ftp://publications.ai.mit.edu/ai-publications/1500-1999/AIM-1653.ps
- Pawitan Y (2001) In all likelihood: statistical modeling and inference using likelihood. Carendon, OxfordGoogle Scholar