Universal consistency of delta estimators

  • Jose M. Vidal-Sanz
  • Miguel A. Delgado
Estimation

Abstract

This paper considers delta estimators of the Radon-Nikodym derivative of a probability function with respect to a σ-finite measure. We provide sufficient conditions for universal consistency, which are checked for some wide classes of nonparametric estimators.

Key words and phrases

Nonparametric density estimation delta estimators universal consistency 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abou-Jaoude, S. (1976a). Sur une condition nécessaire et suffisante deL 1 convergence presque complete de l'estimateur de la partition fixe pour une density,Comptes Rendus de l'Academie des Sciences de Paris, Serie A,283, 1107–1110.MathSciNetMATHGoogle Scholar
  2. Abou-Jaoude, S. (1976b). Sur la convergenceL 1 etL de l'estimateur de la partition aléatorie pour une densité,Annales de l'Institut Henry Poincairé,12, 299–317.MathSciNetGoogle Scholar
  3. Abou-Jaoude, S. (1976c). Conditions nécessaries et suffisantes de convergenceL 1 in probabilité de l'histogram pour une densité,Annales de l'Institut Henry Poincairé,12, 213–231.MathSciNetGoogle Scholar
  4. Araujo, A. and Giné, E. (1980).The Central Limit Theorem for Real and Banach Valued Random Variables, John Wiley & Sons, New York.MATHGoogle Scholar
  5. Basawa, I. V. and Prakasa Rao, B. L. S. (1980).Statistical Inference for Stochastic Processes, Academic Press, New York.MATHGoogle Scholar
  6. Billingsley, P. (1986).Probability and Measure, 2nd ed., John Wiley & Sons, New York.MATHGoogle Scholar
  7. Bosq, D. (1969). Sur l'estimation d'une densité multivariée par une serie de fonctions orthogonales,Comptes Rendus de l'Academie des Sciences de Paris,268, 555–557.MathSciNetMATHGoogle Scholar
  8. Butzer, P. L. and Nessel, R. J. (1971).Fourier Analysis and Approximation, Vol. 1, Birkhäuser Verlag, Bassel and Stuttgart.MATHGoogle Scholar
  9. Čencov, N. N. (1962). Evaluation of an unknown density by orthogonal series,Soviet Mathematics Doklady,3, 1559–1562.Google Scholar
  10. Cheney, E. W. (1982).Introduction to Approximation Theory, Chelsea Publishing Company, New York.MATHGoogle Scholar
  11. Chung, K. L. (1974).A Course in Probability Theory, 2nd ed., Academic Press, San Diego, California.MATHGoogle Scholar
  12. Davis, P. J. (1975).Interpolation and Approximation, Dover, New York.MATHGoogle Scholar
  13. de Guzman, M. (1975). Differentiation of Integrals in ℝn, Lecture Notes in Mathematics,481, Springer Verlag, Berlin.Google Scholar
  14. DeVore, R. A. and Lorentz, G. G. (1993).Constructive Approximation.Grundlehren der mathemastischen Wissenschaften,303, Springer Verlag, Berlin.MATHGoogle Scholar
  15. Devroye, L. (1983). The equivalence of weak, strong and complete convergence inL 1 for kernel density estimates,Annals of Statistics,11, 896–904.MathSciNetMATHGoogle Scholar
  16. Devroye, L. (1987). A course in density estimation,Progress in Probability and Statistics, Birkhäuser, Boston.MATHGoogle Scholar
  17. Devroye, L. (1991). Exponential inequalities in nonparametric estimation,Nonparametric Functional Estimation and Related Topics (ed. G. Roussas), 31–44, Kluwer Academic Publishers, Dordrecht.Google Scholar
  18. Devroye, L. and Györfi, L. (1983). Distribution-free exponential bound on theL 1 error of partitioning estimates of a regression function,Proceedings of the Fourth Pannonian Symposium on Mathematical Statistics (eds. F. Konecny, J. Mogyorodi and W. Wertz), Akadémiai Kiadó, Budapest, Hungary.Google Scholar
  19. Devroye, L. and Györfi, L. (1985a).Nonparametric Density Estimation, The L 1 View, John Wiley & Sons, New York.MATHGoogle Scholar
  20. Devroye, L. and Györfi, L. (1985b). Distribution free exponential bound for theL 1 error of the partitioning estimates of a regression function,Probability and Statistical Decision Theory, Proceedings of the Fourth Pannonian Symposium on Mathematica Statistics (eds. F. Konecny, J. Mogyorodi and W. Wertz), 67–76, Reidel, Dordrecht.Google Scholar
  21. Devroye, L. and Krzyżak, A. (1989). An equivalence theorem forL 1 convergence of nearest neighbor regression function estimates,Annals of Statistics,22, 1371–1385.Google Scholar
  22. Devroye, L. and Lugosi, G. (2001).Combinatorial Methods in Density Estimation, Springer Verlag, New York.MATHGoogle Scholar
  23. Devroye, L. and Wagner, T. J. (1979). On theL 1 convergence of kernel density estimators,Annals of Statistics,7, 1136–1139.MathSciNetMATHGoogle Scholar
  24. Devroye, L. and Wagner, T. J. (1980a). Distribution free consistency results in nonparametric discrimination and regression function estimates,Annals of Statistics,8 231–239.MathSciNetMATHGoogle Scholar
  25. Devroye, L. and Wagner, T. J. (1980b). On theL 1 convergence of kernel estimators of regression functions with application in discrimination,Zeitschrift für Wahrsheinlichkeitstheorie und verwandte Gebiete,51, 15–25.MathSciNetCrossRefMATHGoogle Scholar
  26. Devroye, L. and Wagner, T. J. (1982). Nearest neighbor methods in discrimination,Handbook of Statistics (eds. P. Krishnaiah and L. Kanal), Vol. 2, 193–197, North Holland, Amsterdam.Google Scholar
  27. Devroye, L., Györfi, L., Krzyżak, A. and Lugosi, G. (1996a). On the strong universal consistency of nearest neighbor regression fucntion estimates,Annals of Statistics,22, 1371–1385.Google Scholar
  28. Devroye, L., Györfi, L. and Lugosi, G. (1996b). A probabilistic theory of pattern recognition,Applications of Mathematics, Stochastic Modelling and Applied Probability, Springer Verlag, New York.MATHGoogle Scholar
  29. Dunford, N. and Schwartz, J. T. (1957).Linear Operators. Part I. General Theory, Wiley Classics Library Edition, 1988. John Wiley & Sons, New York.Google Scholar
  30. Edgar, G. A. and Sucheston, L. (1992). Stopping times and directed processes,Encyclopedia of Mathematics and Its Applications (ed. G. C. Rota),47, Cambridge University Press, Cambridge.Google Scholar
  31. Freedman, D. and Diaconis, P. (1981). On the histogram as a density estimator:L 2 theory,Zeitschrift fur Wahrsheinlichkeitstheorie und verwandte Gebiete,58, 139–157.MathSciNetCrossRefMATHGoogle Scholar
  32. Graunt, J. (1662).Natural and Political Observations Made upon the Bills of Mortality, Martyn, London.Google Scholar
  33. Györfi, L. (1981). The rate of convergence ofK nNN regression estimation and classification,IEEE Transactions on Information Theory,IT-27, 500–509.CrossRefGoogle Scholar
  34. Györfi, L., Kohler, M., Krzyżak, A. and Walk, H. (2002).A Distribution-free Theory of Nonparametric Regression, Springer Verlag, New York.MATHGoogle Scholar
  35. Hoffmann-Jørgensen, J. (1974). Sums of independent Banach space valued random variables,Studia Mathematica,52, 159–186.MathSciNetGoogle Scholar
  36. Hoffmann-Jørgensen, J. (1976).Probability in Banach Spaces, Lecture Notes in Mathematics,598, Springer Verlag, New York.Google Scholar
  37. Hong, D. H., Ordoñez-Cabrera, M., Sung, S. H. and Volodin, A. I. (2000). On the weak law for randomly indexed partial sums for arrays of random elements in martingale typep Banach spaces,Statistics and Probability Letters,46, 177–185.MathSciNetCrossRefMATHGoogle Scholar
  38. Hu, T. C., Ordoñez-Cabrera, M. and Volodin, A. I. (2001). Convergence of randomly weighted sums of B-space valued random elements and uniform integrability concerning random weights,Statistics and Probability Letters,51, 155–164.MathSciNetCrossRefMATHGoogle Scholar
  39. Hu, T. H. and Chang, H. H. (1997). Complete convergence and the law of large numbers for arrays of random elements,Proceedings, 2nd World Congress of Nonlinear Analysis, Elsevier Science, U.K. (Nonlinear Analysis. Methods and Applications,30, 4257–4266).Google Scholar
  40. Kantorovich, L. V. and Akilov, G. P. (1982).Functional Analysis, 2nd ed., Pergamon Press, Oxford.MATHGoogle Scholar
  41. Kreyszig, E. (1978).Introductory Functional Analysis with Applications, Wiley Classics Library, John Wiley & Sons, New York.MATHGoogle Scholar
  42. Ledoux, M. and Talagrand, M. (1991).Probability in Banach Spaces: Isoperimetry and Processes, Springer Verlag, New York.MATHGoogle Scholar
  43. Linde, W. (1986).Probability in Banach Spaces-Stable and Infinitely Divisible Distributions, John Wiley & Sons, New York.MATHGoogle Scholar
  44. Lugosi, G. and Nobel, A. (1996). Consistency of data driven histogram methods for density estimation and classification,Annals of Statistics,24, 687–706.MathSciNetCrossRefMATHGoogle Scholar
  45. Lugosi, G. and Zeger, K. (1995). Nonparametric estimation via empirical risk minimization,IEEE Transactions on Information Theory,41, 677–678.MathSciNetCrossRefMATHGoogle Scholar
  46. McDiarmid, C. (1989). On the method of bounded differences,Surveys in Combinatorics, 148–188, Cambridge University Press, Cambridge.Google Scholar
  47. Ordoñez-Cabrera, M. (1994). Convergence of weighted sums of random variables and uniform integrability concerning the weights,Collectanea Mathematica,45, 121–132.MathSciNetMATHGoogle Scholar
  48. Patterson, R. F. and Taylor, R. L. (1997). Strong Laws of Large Numbers for negatively dependent random elements,Proceedings, 2nd World Congress of Nonlinear Analysis, Elsevier Science, U.K. (Nonlinear Analysis. Methods and Applications,30, 4229–4235).Google Scholar
  49. Pisier, G. (1986).Probabilistic Methods in the Geometry of Banach Spaces, Lecture Notes in Mathematics,1206, 167–241, Springer, Berlin.Google Scholar
  50. Pisier, G. (1989).The Volume of Convex Bodies and Banach Space Geometry, Cambridge University Press, Cambridge.MATHGoogle Scholar
  51. Pollard, D. (1984).Convergence of Stochastic Processes, Springer Verlag, New York.MATHGoogle Scholar
  52. Prakasa Rao, B. L. S. (1983).Nonparametric Functional Estimation, Academic Press, London.MATHGoogle Scholar
  53. Révesz, P. (1971). Testing of density functions,Periodica Mathematica Hungarica,1, 35–44.MathSciNetCrossRefMATHGoogle Scholar
  54. Révesz, P. (1972). On empirical density function,Periodica Mathematica Hungarica,2, 85–110.MathSciNetCrossRefMATHGoogle Scholar
  55. Révesz, P. (1973). A strong law of the empirical density function,Transcations of the 6th Prague Conference on Information Theory, 469–472.Google Scholar
  56. Révesz, P. (1974). On empirical density function,Probability and Statistical MethodsSummer School, Bulgarian Academy of Science, Varna, Bulgaria.Google Scholar
  57. Rudin, W. (1966).Real and Complex Analysis, 2nd ed., McGraw Hill, New York.MATHGoogle Scholar
  58. Schwartz, L. (1981).Geometry and Probability in Banach Spaces, Lecture Notes in Mathematics,852, Springer Verlag, New York.MATHGoogle Scholar
  59. Scott, D. W. (1979). On optimal data based histograms,Biometrica,66, 605–610.CrossRefMATHGoogle Scholar
  60. Scott, D. W. (1992).Multivariate Density Estimation. Theory, Practice, and Visualization, John Wiley & Sons, New York.MATHGoogle Scholar
  61. Shilov, G. E. and Gurevich, B. L. (1997).Integral, Measure & Derivative: A Unified Approach, Dover, New York.Google Scholar
  62. Silverman, A. N. (1978). Weak and strong uniform consistency of the kernel estimate of a density and its derivatives,Annals of Statistics,6, 177–184.MathSciNetMATHGoogle Scholar
  63. Stein, E. M. (1970).Singular Integrals and Differentiability Properties of Functions, Princeton University Press, Princeton, New Jersey.MATHGoogle Scholar
  64. Stone, C. J. (1977). Consistent nonparametric regression,Annals of Statistics,5, 595–645.MathSciNetMATHGoogle Scholar
  65. Stute, W. (1982). A law of the logarithm for kernel density estimators,Annals of Probability,10, 414–422.MathSciNetMATHGoogle Scholar
  66. Taylor, R. L. and Hu, T. H. (1987). Strong Laws of Large Numbers for arrays of rowise independents random elements,International Journal of Mathematics and Mathematical Sciences,10, 804–814.MathSciNetCrossRefGoogle Scholar
  67. Terrell, G. R. (1984). Efficiency of nonparametric density estimators, Tech. Report, Department of Mathematical Sciences, Rice University, Houston, Texas.Google Scholar
  68. Terrell, G. R. and Scott, D. W. (1992). Variable kernel density estimation,Annals of Statistics,20, 1236–1265.MathSciNetMATHGoogle Scholar
  69. Tukey, J. W. (1977).Exploratory Data Analysis, Addison-Wesley, Reading, Massachusetts.MATHGoogle Scholar
  70. Vakhania, N. N. (1981).Probability Distributions on Linear Spaces, North Holland, New York.MATHGoogle Scholar
  71. Vakhania, N. N., Tarieladze, V. I. and Chobanyan, S. A. (1987). Probability distributions on Banach spaces,Mathematics and Its Applications (Soviet Series), Reldel Publishing Company, Dordrecht (translated from Russian, 1985, Nauka, Moscow).MATHGoogle Scholar
  72. van der Vaart, A. W. and Wellner, J. A. (1996).Weak Convergence and Empirical Processes with Applications to Statistics, Springer Verlag, New York.MATHGoogle Scholar
  73. Vapnik, V. (1982).Estimation of Dependencies Based on Empirical Data, Springer Verlag, New York.Google Scholar
  74. Vidal-Sanz, J. M. (1999).Universal Consistency of Delta Estimators: An Approximation Theory Based Approach, Ph.D. Dissertation, Universidad Carlos III de Madrid, Spain (in Spanish).Google Scholar
  75. Walter, G. and Blum, J. R. (1979). Probability density estimation using delta sequences,Annals of Statistics,7, 328–340.MathSciNetMATHGoogle Scholar
  76. Watson, G. S. and Leadbetter, M. R. (1963). On the estimation of probability density I,Annals of Statistics,34, 480–491.MathSciNetGoogle Scholar
  77. Watson, G. S. and Leadbetter, M. R. (1964). Hazard analysis II,Shankhyā, series A,26, 101–116.MathSciNetMATHGoogle Scholar
  78. Wheeden, R. and Zygmund, A. (1977).Measure and Integral, Marcel Dekker, New York.MATHGoogle Scholar
  79. Whittle, P. (1958). On the smoothing of probability density functions,Journal of the Royal Statistical Society Series B,20, 334–343.MathSciNetMATHGoogle Scholar
  80. Winter, B. B. (1973). Strong uniform consistency of integrals of density estimation,The Canadian Journal of Statistics,1, 247–253.MathSciNetMATHGoogle Scholar
  81. Winter, B. B. (1975). Rate of strong consistency of two nonparametric density estimators,Annals of Statistics,3, 759–766.MathSciNetMATHGoogle Scholar
  82. Woyczyński, W. A. (1978). Geometry and martingales in Banach spaces—Part II: Independent increments,Probability on Banach Spaces (ed. J. Kuebs),Advances in Probability and Related Topics, Vol. 4, 267–519, Marcel Dekker, New York.Google Scholar
  83. Xia, Dao-Xing (1972).Measure and Integration on Infinite-dimensional Spaces: Abstract Harmonic Analysis, Academic Press, New York.MATHGoogle Scholar

Copyright information

© The Institute of Statistical Mathematics 2004

Authors and Affiliations

  • Jose M. Vidal-Sanz
    • 1
  • Miguel A. Delgado
    • 2
  1. 1.Department of Business EconomicsUniversidad Carlos III de MadridGetafe, MadridSpain
  2. 2.Department of EconomicsUniversidad Carlos III de MadridGetafe, MadridSpain

Personalised recommendations