Skip to main content
SpringerLink
Log in

Universal consistency of delta estimators

  • Estimation
  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Abstract

This paper considers delta estimators of the Radon-Nikodym derivative of a probability function with respect to a σ-finite measure. We provide sufficient conditions for universal consistency, which are checked for some wide classes of nonparametric estimators.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abou-Jaoude, S. (1976a). Sur une condition nécessaire et suffisante deL 1 convergence presque complete de l'estimateur de la partition fixe pour une density,Comptes Rendus de l'Academie des Sciences de Paris, Serie A,283, 1107–1110.

    MathSciNet  MATH  Google Scholar 

  • Abou-Jaoude, S. (1976b). Sur la convergenceL 1 etL ∞ de l'estimateur de la partition aléatorie pour une densité,Annales de l'Institut Henry Poincairé,12, 299–317.

    MathSciNet  Google Scholar 

  • Abou-Jaoude, S. (1976c). Conditions nécessaries et suffisantes de convergenceL 1 in probabilité de l'histogram pour une densité,Annales de l'Institut Henry Poincairé,12, 213–231.

    MathSciNet  Google Scholar 

  • Araujo, A. and Giné, E. (1980).The Central Limit Theorem for Real and Banach Valued Random Variables, John Wiley & Sons, New York.

    MATH  Google Scholar 

  • Basawa, I. V. and Prakasa Rao, B. L. S. (1980).Statistical Inference for Stochastic Processes, Academic Press, New York.

    MATH  Google Scholar 

  • Billingsley, P. (1986).Probability and Measure, 2nd ed., John Wiley & Sons, New York.

    MATH  Google Scholar 

  • Bosq, D. (1969). Sur l'estimation d'une densité multivariée par une serie de fonctions orthogonales,Comptes Rendus de l'Academie des Sciences de Paris,268, 555–557.

    MathSciNet  MATH  Google Scholar 

  • Butzer, P. L. and Nessel, R. J. (1971).Fourier Analysis and Approximation, Vol. 1, Birkhäuser Verlag, Bassel and Stuttgart.

    MATH  Google Scholar 

  • Čencov, N. N. (1962). Evaluation of an unknown density by orthogonal series,Soviet Mathematics Doklady,3, 1559–1562.

    Google Scholar 

  • Cheney, E. W. (1982).Introduction to Approximation Theory, Chelsea Publishing Company, New York.

    MATH  Google Scholar 

  • Chung, K. L. (1974).A Course in Probability Theory, 2nd ed., Academic Press, San Diego, California.

    MATH  Google Scholar 

  • Davis, P. J. (1975).Interpolation and Approximation, Dover, New York.

    MATH  Google Scholar 

  • de Guzman, M. (1975). Differentiation of Integrals in ℝn, Lecture Notes in Mathematics,481, Springer Verlag, Berlin.

    Google Scholar 

  • DeVore, R. A. and Lorentz, G. G. (1993).Constructive Approximation.Grundlehren der mathemastischen Wissenschaften,303, Springer Verlag, Berlin.

    MATH  Google Scholar 

  • Devroye, L. (1983). The equivalence of weak, strong and complete convergence inL 1 for kernel density estimates,Annals of Statistics,11, 896–904.

    MathSciNet  MATH  Google Scholar 

  • Devroye, L. (1987). A course in density estimation,Progress in Probability and Statistics, Birkhäuser, Boston.

    MATH  Google Scholar 

  • Devroye, L. (1991). Exponential inequalities in nonparametric estimation,Nonparametric Functional Estimation and Related Topics (ed. G. Roussas), 31–44, Kluwer Academic Publishers, Dordrecht.

    Google Scholar 

  • Devroye, L. and Györfi, L. (1983). Distribution-free exponential bound on theL 1 error of partitioning estimates of a regression function,Proceedings of the Fourth Pannonian Symposium on Mathematical Statistics (eds. F. Konecny, J. Mogyorodi and W. Wertz), Akadémiai Kiadó, Budapest, Hungary.

    Google Scholar 

  • Devroye, L. and Györfi, L. (1985a).Nonparametric Density Estimation, The L 1 View, John Wiley & Sons, New York.

    MATH  Google Scholar 

  • Devroye, L. and Györfi, L. (1985b). Distribution free exponential bound for theL 1 error of the partitioning estimates of a regression function,Probability and Statistical Decision Theory, Proceedings of the Fourth Pannonian Symposium on Mathematica Statistics (eds. F. Konecny, J. Mogyorodi and W. Wertz), 67–76, Reidel, Dordrecht.

    Google Scholar 

  • Devroye, L. and Krzyżak, A. (1989). An equivalence theorem forL 1 convergence of nearest neighbor regression function estimates,Annals of Statistics,22, 1371–1385.

    Google Scholar 

  • Devroye, L. and Lugosi, G. (2001).Combinatorial Methods in Density Estimation, Springer Verlag, New York.

    MATH  Google Scholar 

  • Devroye, L. and Wagner, T. J. (1979). On theL 1 convergence of kernel density estimators,Annals of Statistics,7, 1136–1139.

    MathSciNet  MATH  Google Scholar 

  • Devroye, L. and Wagner, T. J. (1980a). Distribution free consistency results in nonparametric discrimination and regression function estimates,Annals of Statistics,8 231–239.

    MathSciNet  MATH  Google Scholar 

  • Devroye, L. and Wagner, T. J. (1980b). On theL 1 convergence of kernel estimators of regression functions with application in discrimination,Zeitschrift für Wahrsheinlichkeitstheorie und verwandte Gebiete,51, 15–25.

    Article  MathSciNet  MATH  Google Scholar 

  • Devroye, L. and Wagner, T. J. (1982). Nearest neighbor methods in discrimination,Handbook of Statistics (eds. P. Krishnaiah and L. Kanal), Vol. 2, 193–197, North Holland, Amsterdam.

    Google Scholar 

  • Devroye, L., Györfi, L., Krzyżak, A. and Lugosi, G. (1996a). On the strong universal consistency of nearest neighbor regression fucntion estimates,Annals of Statistics,22, 1371–1385.

    Google Scholar 

  • Devroye, L., Györfi, L. and Lugosi, G. (1996b). A probabilistic theory of pattern recognition,Applications of Mathematics, Stochastic Modelling and Applied Probability, Springer Verlag, New York.

    MATH  Google Scholar 

  • Dunford, N. and Schwartz, J. T. (1957).Linear Operators. Part I. General Theory, Wiley Classics Library Edition, 1988. John Wiley & Sons, New York.

    Google Scholar 

  • Edgar, G. A. and Sucheston, L. (1992). Stopping times and directed processes,Encyclopedia of Mathematics and Its Applications (ed. G. C. Rota),47, Cambridge University Press, Cambridge.

    Google Scholar 

  • Freedman, D. and Diaconis, P. (1981). On the histogram as a density estimator:L 2 theory,Zeitschrift fur Wahrsheinlichkeitstheorie und verwandte Gebiete,58, 139–157.

    Article  MathSciNet  MATH  Google Scholar 

  • Graunt, J. (1662).Natural and Political Observations Made upon the Bills of Mortality, Martyn, London.

    Google Scholar 

  • Györfi, L. (1981). The rate of convergence ofK n —NN regression estimation and classification,IEEE Transactions on Information Theory,IT-27, 500–509.

    Article  Google Scholar 

  • Györfi, L., Kohler, M., Krzyżak, A. and Walk, H. (2002).A Distribution-free Theory of Nonparametric Regression, Springer Verlag, New York.

    MATH  Google Scholar 

  • Hoffmann-Jørgensen, J. (1974). Sums of independent Banach space valued random variables,Studia Mathematica,52, 159–186.

    MathSciNet  Google Scholar 

  • Hoffmann-Jørgensen, J. (1976).Probability in Banach Spaces, Lecture Notes in Mathematics,598, Springer Verlag, New York.

    Google Scholar 

  • Hong, D. H., Ordoñez-Cabrera, M., Sung, S. H. and Volodin, A. I. (2000). On the weak law for randomly indexed partial sums for arrays of random elements in martingale typep Banach spaces,Statistics and Probability Letters,46, 177–185.

    Article  MathSciNet  MATH  Google Scholar 

  • Hu, T. C., Ordoñez-Cabrera, M. and Volodin, A. I. (2001). Convergence of randomly weighted sums of B-space valued random elements and uniform integrability concerning random weights,Statistics and Probability Letters,51, 155–164.

    Article  MathSciNet  MATH  Google Scholar 

  • Hu, T. H. and Chang, H. H. (1997). Complete convergence and the law of large numbers for arrays of random elements,Proceedings, 2nd World Congress of Nonlinear Analysis, Elsevier Science, U.K. (Nonlinear Analysis. Methods and Applications,30, 4257–4266).

    Google Scholar 

  • Kantorovich, L. V. and Akilov, G. P. (1982).Functional Analysis, 2nd ed., Pergamon Press, Oxford.

    MATH  Google Scholar 

  • Kreyszig, E. (1978).Introductory Functional Analysis with Applications, Wiley Classics Library, John Wiley & Sons, New York.

    MATH  Google Scholar 

  • Ledoux, M. and Talagrand, M. (1991).Probability in Banach Spaces: Isoperimetry and Processes, Springer Verlag, New York.

    MATH  Google Scholar 

  • Linde, W. (1986).Probability in Banach Spaces-Stable and Infinitely Divisible Distributions, John Wiley & Sons, New York.

    MATH  Google Scholar 

  • Lugosi, G. and Nobel, A. (1996). Consistency of data driven histogram methods for density estimation and classification,Annals of Statistics,24, 687–706.

    Article  MathSciNet  MATH  Google Scholar 

  • Lugosi, G. and Zeger, K. (1995). Nonparametric estimation via empirical risk minimization,IEEE Transactions on Information Theory,41, 677–678.

    Article  MathSciNet  MATH  Google Scholar 

  • McDiarmid, C. (1989). On the method of bounded differences,Surveys in Combinatorics, 148–188, Cambridge University Press, Cambridge.

    Google Scholar 

  • Ordoñez-Cabrera, M. (1994). Convergence of weighted sums of random variables and uniform integrability concerning the weights,Collectanea Mathematica,45, 121–132.

    MathSciNet  MATH  Google Scholar 

  • Patterson, R. F. and Taylor, R. L. (1997). Strong Laws of Large Numbers for negatively dependent random elements,Proceedings, 2nd World Congress of Nonlinear Analysis, Elsevier Science, U.K. (Nonlinear Analysis. Methods and Applications,30, 4229–4235).

    Google Scholar 

  • Pisier, G. (1986).Probabilistic Methods in the Geometry of Banach Spaces, Lecture Notes in Mathematics,1206, 167–241, Springer, Berlin.

    Google Scholar 

  • Pisier, G. (1989).The Volume of Convex Bodies and Banach Space Geometry, Cambridge University Press, Cambridge.

    MATH  Google Scholar 

  • Pollard, D. (1984).Convergence of Stochastic Processes, Springer Verlag, New York.

    MATH  Google Scholar 

  • Prakasa Rao, B. L. S. (1983).Nonparametric Functional Estimation, Academic Press, London.

    MATH  Google Scholar 

  • Révesz, P. (1971). Testing of density functions,Periodica Mathematica Hungarica,1, 35–44.

    Article  MathSciNet  MATH  Google Scholar 

  • Révesz, P. (1972). On empirical density function,Periodica Mathematica Hungarica,2, 85–110.

    Article  MathSciNet  MATH  Google Scholar 

  • Révesz, P. (1973). A strong law of the empirical density function,Transcations of the 6th Prague Conference on Information Theory, 469–472.

  • Révesz, P. (1974). On empirical density function,Probability and Statistical Methods—Summer School, Bulgarian Academy of Science, Varna, Bulgaria.

    Google Scholar 

  • Rudin, W. (1966).Real and Complex Analysis, 2nd ed., McGraw Hill, New York.

    MATH  Google Scholar 

  • Schwartz, L. (1981).Geometry and Probability in Banach Spaces, Lecture Notes in Mathematics,852, Springer Verlag, New York.

    MATH  Google Scholar 

  • Scott, D. W. (1979). On optimal data based histograms,Biometrica,66, 605–610.

    Article  MATH  Google Scholar 

  • Scott, D. W. (1992).Multivariate Density Estimation. Theory, Practice, and Visualization, John Wiley & Sons, New York.

    MATH  Google Scholar 

  • Shilov, G. E. and Gurevich, B. L. (1997).Integral, Measure & Derivative: A Unified Approach, Dover, New York.

    Google Scholar 

  • Silverman, A. N. (1978). Weak and strong uniform consistency of the kernel estimate of a density and its derivatives,Annals of Statistics,6, 177–184.

    MathSciNet  MATH  Google Scholar 

  • Stein, E. M. (1970).Singular Integrals and Differentiability Properties of Functions, Princeton University Press, Princeton, New Jersey.

    MATH  Google Scholar 

  • Stone, C. J. (1977). Consistent nonparametric regression,Annals of Statistics,5, 595–645.

    MathSciNet  MATH  Google Scholar 

  • Stute, W. (1982). A law of the logarithm for kernel density estimators,Annals of Probability,10, 414–422.

    MathSciNet  MATH  Google Scholar 

  • Taylor, R. L. and Hu, T. H. (1987). Strong Laws of Large Numbers for arrays of rowise independents random elements,International Journal of Mathematics and Mathematical Sciences,10, 804–814.

    Article  MathSciNet  Google Scholar 

  • Terrell, G. R. (1984). Efficiency of nonparametric density estimators, Tech. Report, Department of Mathematical Sciences, Rice University, Houston, Texas.

    Google Scholar 

  • Terrell, G. R. and Scott, D. W. (1992). Variable kernel density estimation,Annals of Statistics,20, 1236–1265.

    MathSciNet  MATH  Google Scholar 

  • Tukey, J. W. (1977).Exploratory Data Analysis, Addison-Wesley, Reading, Massachusetts.

    MATH  Google Scholar 

  • Vakhania, N. N. (1981).Probability Distributions on Linear Spaces, North Holland, New York.

    MATH  Google Scholar 

  • Vakhania, N. N., Tarieladze, V. I. and Chobanyan, S. A. (1987). Probability distributions on Banach spaces,Mathematics and Its Applications (Soviet Series), Reldel Publishing Company, Dordrecht (translated from Russian, 1985, Nauka, Moscow).

    MATH  Google Scholar 

  • van der Vaart, A. W. and Wellner, J. A. (1996).Weak Convergence and Empirical Processes with Applications to Statistics, Springer Verlag, New York.

    MATH  Google Scholar 

  • Vapnik, V. (1982).Estimation of Dependencies Based on Empirical Data, Springer Verlag, New York.

    Google Scholar 

  • Vidal-Sanz, J. M. (1999).Universal Consistency of Delta Estimators: An Approximation Theory Based Approach, Ph.D. Dissertation, Universidad Carlos III de Madrid, Spain (in Spanish).

    Google Scholar 

  • Walter, G. and Blum, J. R. (1979). Probability density estimation using delta sequences,Annals of Statistics,7, 328–340.

    MathSciNet  MATH  Google Scholar 

  • Watson, G. S. and Leadbetter, M. R. (1963). On the estimation of probability density I,Annals of Statistics,34, 480–491.

    MathSciNet  Google Scholar 

  • Watson, G. S. and Leadbetter, M. R. (1964). Hazard analysis II,Shankhyā, series A,26, 101–116.

    MathSciNet  MATH  Google Scholar 

  • Wheeden, R. and Zygmund, A. (1977).Measure and Integral, Marcel Dekker, New York.

    MATH  Google Scholar 

  • Whittle, P. (1958). On the smoothing of probability density functions,Journal of the Royal Statistical Society Series B,20, 334–343.

    MathSciNet  MATH  Google Scholar 

  • Winter, B. B. (1973). Strong uniform consistency of integrals of density estimation,The Canadian Journal of Statistics,1, 247–253.

    MathSciNet  MATH  Google Scholar 

  • Winter, B. B. (1975). Rate of strong consistency of two nonparametric density estimators,Annals of Statistics,3, 759–766.

    MathSciNet  MATH  Google Scholar 

  • Woyczyński, W. A. (1978). Geometry and martingales in Banach spaces—Part II: Independent increments,Probability on Banach Spaces (ed. J. Kuebs),Advances in Probability and Related Topics, Vol. 4, 267–519, Marcel Dekker, New York.

    Google Scholar 

  • Xia, Dao-Xing (1972).Measure and Integration on Infinite-dimensional Spaces: Abstract Harmonic Analysis, Academic Press, New York.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Business Economics, Universidad Carlos III de Madrid, C/Madrid, 126, 28903, Getafe, Madrid, Spain

    Jose M. Vidal-Sanz

  2. Department of Economics, Universidad Carlos III de Madrid, C/Madrid, 126, 28903, Getafe, Madrid, Spain

    Miguel A. Delgado

Authors
  1. Jose M. Vidal-Sanz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miguel A. Delgado
    View author publications

    You can also search for this author in PubMed Google Scholar

About this article

Cite this article

Vidal-Sanz, J.M., Delgado, M.A. Universal consistency of delta estimators. Ann Inst Stat Math 56, 791–818 (2004). https://doi.org/10.1007/BF02506490

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02506490

Key words and phrases

Search

Navigation