Abstract
In the paper the problem of estimation of Fisher information I f for a univariate density supported on [0, 1] is discussed. A starting point is an observation that when the density belongs to an exponential family of a known dimension, an explicit formula for I f there allows for its simple estimation. In a general case, for a given random sample, a dimension of an exponential family which approximates it best is sought and then estimator of I f is constructed for the chosen family. As a measure of quality of fit a modified Bayes Information Criterion is used. The estimator, which is an instance of Post Model Selection Estimation method is proved to be consistent and asymptotically normal when the density belongs to the exponential family. Its consistency is also proved under misspecification when the number of exponential models under consideration increases in a suitable way. Moreover we provide evidence that in most of considered parametric cases the small sample performance of proposed estimator is superior to that of kernel estimators.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barron AR, Sheu CH (1991) Approximation of density functions by sequences of exponential families. Ann Stat 19: 1347–1369
Bickel P, Klassen C, Ritov Y, Wellner J (1998) Semiparametrics. In: Kotz S, Read C, Banks D (eds) Encyclopedia of statistical sciences, vol 2. Wiley, New York, pp 602–614
Bogdan K, Bogdan M (2000) On existence of maximum likelihood estimators in exponential families. Statistics 34: 137–149
Csörgő M, Révész P (1986) A nearest neighbour-estimator for the score function. Prob Theory Relat Fields 71: 293–305
Haughton DMA (1988) On the choice of a model to fit data from an exponential family. Ann Stat 16: 342–355
Inglot T, Ledwina T (1996) Asymptotic optimality of data-driven Neyman’s tests for uniformity. Ann Stat 24: 1982–2019
Inglot T, Ledwina T (2001) Intermediate approach to comparison of some goodness-of-fit tests. Ann Inst Stat Math 53: 810–834
Kallenberg WCM, Ledwina T (1995) Consistency and Monte Carlo simulation of a data driven version of smooth goodness-of-fit tests. Ann Stat 23: 1594–1608
Kallenberg WCM, Ledwina T (1997) Data driven smooth test when the hypothesis is composite. J Am Stat Assoc 92: 1094–1104
Ledwina T (1994) Data-driven version of Neyman’s smooth test of fit. J Am Stat Assoc 89: 1000–1005
Ledwina T (2000) Limiting behaviour of data driven Neyman’s statistic under fixed alternatives. Manuscript
Leeb H, Pötscher BM (2006) Model Selection, manuscript. http://www.stat.yale.edu/hl284/handbook.pdf
Schuster EF (1985) Incorportating support constraints into nonparametric estimation of densities. Commun Stat Theory Methods 14: 1123–1136
Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6: 461–464
Sheather SJ, Jones MC (1991) A reliable data-based bandwidth selection method for kernel density estimation. J R Stat Soc Ser B 53: 683–690
Stone CJ (1975) Adaptive maximum likelihood estimator of a location parameter. Ann Stat 3: 267–284
van der Vaart AW (2000) Asymptotic statistics. Cambridge University Press, Cambridge
Yurinskii VV (1976) Exponential inequalities for sums of random vectors. J Mult Anal 6: 473–499
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mielniczuk, J., Wojtyś, M. Estimation of Fisher information using model selection. Metrika 72, 163–187 (2010). https://doi.org/10.1007/s00184-009-0246-3
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00184-009-0246-3