Skip to main content
SpringerLink
Log in

High order Parzen windows and randomized sampling

  • Published:
Advances in Computational Mathematics Aims and scope Submit manuscript

Abstract

In this paper high order Parzen windows stated by means of basic window functions are studied for understanding some algorithms in learning theory and randomized sampling in multivariate approximation. Learning rates are derived for the least-square regression and density estimation on bounded domains under some decay conditions on the marginal distributions near the boundary. These rates can be almost optimal when the marginal distributions decay fast and the order of the Parzen windows is large enough. For randomized sampling in shift-invariant spaces, we consider the situation when the sampling points are neither i.i.d. nor regular, but are noised from regular grids by probability density functions. The approximation orders are estimated by means of the regularity of the approximated function and the density function and the order of the Parzen windows.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aldroubi, A., Gröchenig, K.: Non-uniform sampling and reconstruction in shift-invariant spaces. SIAM Rev. 43, 585–620 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  2. Bartlett, M.S.: Statistical estimation of density functions. Sankhya Ser. A 25, 245–254 (1963)

    MATH  MathSciNet  Google Scholar 

  3. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003)

    Article  MATH  Google Scholar 

  4. de Boor, C.: A Practical Guide to Splines, Applied Mathematical Sciences, vol. 27. Springer, New York (1978)

    Google Scholar 

  5. Caponnetto, A., Smale, S.: Risk bounds for random regression graphs. Found. Comput. Math. 7, 495–528 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  6. Daubechies, I.: Ten Lectures on Wavelets. SIAM, Philadelphia (1992)

    MATH  Google Scholar 

  7. De Vito, E., Caponnetto, A., Rosasco, L.: Model selection for regularized least-squares algorithm in learning theory. Found. Comput. Math. 5, 59–85 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  8. Devroye, L., Lugosi, G.: Combinatorial Methods in Density Estimation. Springer, Heidelberg (2000)

    Google Scholar 

  9. Evgeniou, T., Pontil, M., Poggio, T.: Regularization networks and suport vector machines. Adv. Comput. Math. 13, 1–50 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  10. Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic, London (1990)

    MATH  Google Scholar 

  11. Hardin, D., Tsamardinos, I., Aliferis, C.F.: A theoretical characterization of linear SVM-based feature selection. In: Proc. of the 21st International Conference on Machine Learning, Banff, 4–8 July 2004

  12. Jia, R.Q.: Approximation with scaled shift-invariant spaces by means of quasi-projection operators. J. Approx. Theory 131, 30–46 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  13. Lei, J.J., Jia, R.Q., Cheney, E.W.: Approximation from shift-invariant spaces by integral operators. SIAM J. Math. Anal. 28, 481–498 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  14. Marron, J.S., Wand, M.P.: Exact mean integrated square error. Ann. Statist. 20, 712–736 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  15. Mukherjee, S., Zhou, D.X.: Learning coordinate covariances via gradients. J. Mach. Learn. Res. 7, 519–549 (2006)

    MathSciNet  Google Scholar 

  16. Parzen, E.: On the estimation of a probability density function and the mode. Ann. Math. Stat. 33, 1049–1051 (1962)

    Article  MathSciNet  Google Scholar 

  17. Pinelis, I.: Optimum bounds for the distributions of martingales in Banach spaces. Ann. Probab. 22, 1679–1706 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  18. Smale, S., Zhou, D.X.: Shannon sampling and function reconstruction from point values. Bull. Amer. Math. Soc. 41, 279–305 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  19. Smale, S., Zhou, D.X., Shannon sampling II. Connections to learning theory. Appl. Comput. Harmon. Anal. 19, 285–302 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  20. Smale, S., Zhou, D.X.: Learning theory estimates via integral operators and their approximations. Constr. Approx. 26, 153–172 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  21. Smale, S., Zhou, D.X.: Online learning with Markov sampling. Anal. Appl. (to appear)

  22. Stone, C.J.: Optimal rates of convergence for nonparametric estimators. Ann. Stat. 8, 1348–1360 (1980)

    Article  MATH  Google Scholar 

  23. Strang, G., Fix, G.: A Fourier analysis of the finite-element variational method. In: Geymonat, G. (ed.) Constructive Aspects of Functional Analysis, C. I. M. E., pp. 793–840 (1973)

  24. Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)

    MATH  Google Scholar 

  25. Wand, M.P., Jones, M.C.: Kernel Smoothing, Monographs on Statistics and Applied Probability, vol. 60. Chapman & Hall, London (1995)

    Google Scholar 

  26. Wu, Q., Ying, Y., Zhou, D.X.: Learning rates of least-square regularized regression. Found. Comput. Math. 6, 171–192 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  27. Ye, G.B., Zhou, D.X.: Learning and approximation by Gaussians on Riemannian manifolds. Adv. Comput. Math. doi:10.1007/s10444-007-9049-0

  28. Ying, Y., Zhou, D.X.: Learnability of Gaussians with flexible variances. J. Mach. Learn. Res. 8, 249–276 (2007)

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Joint Advanced Research Center in Suzhou, University of Science and Technology of China, City University of Hong Kong, Suzhou, Jiangshu, 215123, China

    Xiang-Jun Zhou

  2. Department of Mathematics, City University of Hong Kong, Kowloon, Hong Kong, China

    Ding-Xuan Zhou

Authors
  1. Xiang-Jun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ding-Xuan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ding-Xuan Zhou.

Additional information

Communicated by Juan Manuel Peña.

Supported partially by City University of Hong Kong [Project No. 7002126], National Science Fund for Distinguished Young Scholars of China [Project No. 10529101], and National Basic Research Program of China [Project No. 973-2006CB303102].

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhou, XJ., Zhou, DX. High order Parzen windows and randomized sampling. Adv Comput Math 31, 349 (2009). https://doi.org/10.1007/s10444-008-9073-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10444-008-9073-8

Keywords

Mathematics Subject Classifications (2000)

Search

Navigation