Skip to main content
Log in

Rates of convex approximation in non-hilbert spaces

  • Published:
Constructive Approximation Aims and scope

Abstract

This paper deals with sparse approximations by means of convex combinations of elements from a predetermined “basis” subsetS of a function space. Specifically, the focus is on therate at which the lowest achievable error can be reduced as larger subsets ofS are allowed when constructing an approximant. The new results extend those given for Hilbert spaces by Jones and Barron, including, in particular, a computationally attractive incremental approximation scheme. Bounds are derived for broad classes of Banach spaces; in particular, forL p spaces with 1<p<∞, theO (n −1/2) bounds of Barron and Jones are recovered whenp=2.

One motivation for the questions studied here arises from the area of “artificial neural networks,” where the problem can be stated in terms of the growth in the number of “neurons” (the elements ofS) needed in order to achieve a desired error rate. The focus on non-Hilbert spaces is due to the desire to understand approximation in the more “robust” (resistant to exemplar noise)L p, 1 ≤p<2, norms.

The techniques used borrow from results regarding moduli of smoothness in functional analysis as well as from the theory of stochastic processes on function spaces.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. S. Banach, S. Saks (1930):Sur la convergence forte dans les champs L P. Studia Math.,2:51–57.

    Google Scholar 

  2. A. R. Barron (1991):Approximation and estimation bounds for artificial neural networks. In: Proc. Fourth Annual Workshop on Computational Learning Theory. Morgan Kaufmann, pp. 243–249.

  3. A. R. Barron (1992):Neural net approximation. In: Proc. of the Seventh Yale Workshop on Adaptive and Learning Systems. pp. 69–72.

  4. C. Bessaga, A. Pelczynski (1958):A generalization of results of R. C. James concerning absolute bases in Banach spaces. Studia Math.,17:151–164.

    MATH  Google Scholar 

  5. J. A. Clarkson (1936):Uniformly convex spaces. Trans. Amer. Math. Soc.,40:396–414.

    Article  MATH  Google Scholar 

  6. C. Darken, M. Donahue, L. Gurvits, E. Sontag (1993):Rate of approximation results motivated by robust neural network learning. In: Proc. of the Sixth Annual ACM Conference on Computational Learning Theory. New York: The Association for Computing Machinery. pp. 303–309.

    Chapter  Google Scholar 

  7. R. Deville, G. Godefroy, V. Zizler (1993): Smoothness and Renormings in Banach Spaces. New York: Wiley.

    MATH  Google Scholar 

  8. J. Dieudonné (1960): Foundations of Modern Analysis. New York: Academic Press.

    MATH  Google Scholar 

  9. T. Figiel, G. Pisier (1974):Séries aléatoires dans les espaces uniformément convexes ou uniformément lisses. C. R. Acad. Sci. Paris,279:611–614.

    MATH  Google Scholar 

  10. W. T. Gowers (Preprint):A Banach space not containing c 0, l1, or a reflexive subspace.

  11. U. Haagerup (1982):The best constants in the Khintchine inequality. Studia Math.,70:231–283.

    MATH  Google Scholar 

  12. O. Hanner (1956):On the uniform convexity of L p and ℓp. Arkiv. Matematik,3:239–244.

    Article  MATH  Google Scholar 

  13. S. J. Hanson, D. J. Burr (1988):Minkowski-r back-propagation: learning in connectionist models with non-Euclidean error signals. In: Neural Information Processing Systems. New York: American Institute of Physics, p. 348.

    Google Scholar 

  14. R. C. James (1978):Nonreflexive spaces of type 2. Israel J. Math.,30:1–13.

    MATH  Google Scholar 

  15. L. K. Jones (1992):A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Ann. Statist.,20:608–613.

    MATH  Google Scholar 

  16. S. Kakutani (1938):Weak convergence in uniformly convex spaces. Tôhoku Math. J.,45:188–193.

    MATH  Google Scholar 

  17. J. Khintchine (1923):Über die diadischen Brüche. Math. Z.,18:109–116.

    Article  Google Scholar 

  18. M. Ledoux, M. Talagrand (1991): Probability in Banach Space. Berlin: Springer-Verlag

    Google Scholar 

  19. M. Leshno, V. Lin, A. Pinkus, S. Schocken (1992):Multilayer feedforward networks with a non-polynomial activation function can approximate any function. Preprint. Hebrew University.

  20. J. Lindenstrauss (1963):On the modulus of smoothness and divergent series in Banach spaces. Michigan Math. J.,10:241–252.

    Article  Google Scholar 

  21. J. Lindenstrauss, L. Tzafriri (1979): Classical Banach Spaces II: Function Spaces. Berlin: Springer-Verlag.

    MATH  Google Scholar 

  22. M. J. D. Powell (1981): Approximation Theory and Methods. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  23. W. J. Rey (1983): Introduction to Robust and Quasi-Robust Statistical Methods. Berlin: Springer-Verlag.

    MATH  Google Scholar 

  24. H. Rosenthal (1974):A characterization of Banach spaces containing l l. Proc. Nat. Acad. Sci. (USA),71:2411–2413.

    Article  MATH  Google Scholar 

  25. H. Rosenthal (1994):A subsequence principle characterizing Banach spaces containing c 0. Bull. Amer. Math. Soc.,30:227–233.

    MATH  Google Scholar 

  26. E. D. Sontag (1992):Feedback stabilization using two-hidden-layer nets. IEEE Trans. Neural Networks,3:981–990.

    Article  Google Scholar 

  27. K. R. Stromberg (1981): An Introduction to Classical Real Analysis. New York: Wadsworth.

    MATH  Google Scholar 

  28. J. Y. T. Woo (1973):On modular sequence spaces. Studia Math.,48:271–289.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Additional information

Communicated by Vladimir N. Temlyakov.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Donahue, M.J., Darken, C., Gurvits, L. et al. Rates of convex approximation in non-hilbert spaces. Constr. Approx 13, 187–220 (1997). https://doi.org/10.1007/BF02678464

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02678464

AMS classification

Key words and phrases

Navigation