Abstract
Divergence measures play an important role in statistical theory, especially in large sample theories of estimation and testing. The underlying reason is that they are indices of statistical distance between probability distributions P and Q; the smaller these indices are the harder it is to discriminate between P and Q. Many divergence measures have been proposed since the publication of the paper of Kullback and Leibler (1951). Renyi (1961) gave the first generalization of Kullback-Leibler divergence, Jeffreys (1946) defined the J-divergences, Burbea and Rao (1982) introduced the R-divergences, Sharma and Mittal (1977) the (r,s)-divergences, Csiszar (1967) the ϕ-divergences, Taneja (1989) the generalized J-divergences and the generalized R-divergences and so on. In order to do a unified study of their statistical properties, here we propose a generalized divergence, called (h,ϕ)-divergence, which include as particular cases the above mentioned divergence measures. Under different assumptions, it is shown that the asymptotic distributions of the (h,ϕ)-divergence statistics are either normal or chi square. The chi square and the likelihood ratio test statistics are particular cases of the (h,ϕ)-divergence test statistics considered. From the previous results, asymptotic distributions of entropy statistics are derived too. Applications to testing statistical hypothesis in multinomial populations are given. The Pitman and Bahadur efficiencies of tests of goodness of fit and independence based on these statistics are obtained. To finish, apendices with the asymptotic variances of many well known divergence and entropy statistics are presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arimoto, S. (1971): “Information theoretical considerations on estimation problems”.Information and Control, 19, 181–194.
Bahadur, R.R. (1971): “Some Limits Theorems in Statistics”. SIAM, Philadelfia.
Cressie, N. and Read, T.R.C. (1984): “Multinomial goodness of fit tests”.J. Roy. Statist. Soc., B, 46, 440–464.
Csiszar, I. (1967): “Information type measures of difference of probability distributions and indirect observations”.Studia Sci. Mat. Hung., 2, 299–318.
Dik, J.J. and de Gunst, M.C.M. (1985): “The distribution of general quadratic forms in normal variables”.Statistica Neerlandica, 39, 14–26.
Havrda, J. and Charvát, F. (1967): “Quantification of classification processes. Concept of structural α-entropy”.Kybernetika, 2, 30–35.
Kullback, S. and Leibler, A. (1951): “On the information and sufficiency”.Ann. Math. Statist., 27, 986–1005.
Menéndez, M.L., Morales, D., Pardo, L. and Salicrú, M. (1992): “Some statistical applications of (r,s)-directed divergences”.Utilit. Mathemat., 42, 115–127.
Morales, D., Pardo, L., Salicrú, M. and M. L. Menéndez (1993): “Asymptotic properties of (r,s)-directed divergence in a stratified sampling”.Applied Mathematics and Computation, 15, 131–152.
Onicescu, O. (1966): “Energie informationalle”.C. R. Acad. Sc. Paris Ser. A, 263, 841–842.
Pardo, L., Morales, D., Salicrú, M. and M. L. Menéndez (1993): “The π-divergence statistics in bivariate multinomial populations including stratification”.Metrika, 40, 223–235.
Rao, C. R. (1973):Linear Statistical Inference and its Applications. John Wiley, New York.
Renyi, A. (1961): “On measures of entropy and information”.Proced. 4th Berkeley Symp. Math. Statist. and Prob., 1, 547–561.
Shannon, C. (1948): “A mathematical theory of comunications”.Bell System Tech. J., 27, 379–423.
Sharma B. D. and Mittal D. P. (1977): “New nonadditive measures of entropy for discrete probability distributions”.J. Math. Sci., 10, 28–40.
Taneja, I. J. (1989): “On generalized information measures and their applications”.Adv. Elect. and Elect. Phis. 76, 327–413.
Vajda, I. (1989): “Theory of Statistical Inference and Information”. Kluwer Academic Publishers, Dordrecht.
Zografos, K. (1992): “Asymptotic properties of ϕ-divergence statistic and its applications in contingency tables”. Univ. of Ioannina. Dep. of Mathematics. Probab.-Statist.-Operat. Research Unit.
Zografos, K., Ferentinos, K. and Papaioannou, T. (1990): “π-Divergence statistics: sampling properties and multivariate goodness of fit and divergence tests”.Commun. Statist. (Theory and Meth.), 19(5) 1785–1802.
Author information
Authors and Affiliations
Additional information
The research in this paper was supported in part by DGICYT Grants N. PB91-0387 and N. PB91-0155. Their financial support is gratefully acknowledged.
Rights and permissions
About this article
Cite this article
Menéndez, M.L., Morales, D., Pardo, L. et al. Asymptotic behaviour and statistical applications of divergence measures in multinomial populations: a unified study. Stat Papers 36, 1–29 (1995). https://doi.org/10.1007/BF02926015
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF02926015