Estimation of entropy and other functionals of a multivariate density

Abstract

For a multivariate density f with respect to Lebesgue measure μ, the estimation of % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% Waa8qaaeaacaWGkbGaaiikaiaadAgacaGGPaGaamOzaiaadsgacqaH% 8oqBaSqabeqaniabgUIiYdaaaa!4404!\[\int {J(f)fd\mu } \], and in particular % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% Waa8qaaeaacaWGMbWaaWbaaSqabeaacaaIYaaaaOGaamizaiabeY7a% TbWcbeqab0Gaey4kIipaaaa!41E4!\[\int {f^2 d\mu } \] and % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% Waa8qaaeaacaWGMbGaciiBaiaac+gacaGGNbGaamOzaiaadsgacqaH% 8oqBaSqabeqaniabgUIiYdaaaa!44AC!\[\int {f\log fd\mu } \], is studied. These two particular functionals are important in a number of contexts. Asymptotic bias and variance terms are obtained for the estimators % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% WaaybyaeqaleqabaGaey4jIKnaneaacaWGjbaaaOGaeyypa0Zaa8qa% aeaacaWGkbGaaiikamaawagabeWcbeqaaiabgEIizdqdbaGaamOzaa% aakiaacMcacaWGKbGaamOramaaBaaaleaacaWGobaabeaaaeqabeqd% cqGHRiI8aaaa!4994!\[\mathop I\limits^ \wedge = \int {J(\mathop f\limits^ \wedge )dF_N } \] and % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% WaaybyaeqaleqabaGaeSipIOdaneaacaWGjbaaaOGaeyypa0Zaa8qa% aeaacaWGkbGaaiikamaawagabeWcbeqaaiabgEIizdqdbaGaamOzaa% aakiaacMcadaGfGbqabSqabeaacqGHNis2a0qaaiaadAgaaaGccaWG% KbGaeqiVd0galeqabeqdcqGHRiI8aaaa!4C40!\[\mathop I\limits^ \sim = \int {J(\mathop f\limits^ \wedge )\mathop f\limits^ \wedge d\mu } \], where % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% WaaybyaeqaleqabaGaey4jIKnaneaacaWGMbaaaaaa!3E9C!\[{\mathop f\limits^ \wedge }\] is a kernel density estimate of f and F _n is the empirical distribution function based on the random sample X ₁ ,..., X _n from f. For the two functionalsmentioned above, a first order bias term for Î can be made zero by appropriate choices of non-unimodal kernels. Suggestions for the choice of bandwidth are given; for % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiiYdd9qrFfea0dXdf9vqai-hEir8Ve% ea0de9qq-hbrpepeea0db9q8as0-LqLs-Jirpepeea0-as0Fb9pgea% 0lrP0xe9Fve9Fve9qapdbaqaaeGacaGaaiaabeqaamaabaabcaGcba% WaaybyaeqaleqabaGaey4jIKnaneaacaWGjbaaaOGaeyypa0Zaa8qa% aeaadaGfGbqabSqabeaacqGHNis2a0qaaiaadAgaaaGccaWGKbGaam% OramaaBaaaleaacaWGobaabeaaaeqabeqdcqGHRiI8aaaa!476C!\[\mathop I\limits^ \wedge = \int {\mathop f\limits^ \wedge dF_N } \], a study of optimal bandwidth is possible.