Advances in Computational Mathematics

, Volume 1, Issue 1, pp 61–80 | Cite as

Approximation properties of a multilayered feedforward artificial neural network

  • H. N. Mhaskar
Articles

Abstract

We prove that an artificial neural network with multiple hidden layers and akth-order sigmoidal response function can be used to approximate any continuous function on any compact subset of a Euclidean space so as to achieve the Jackson rate of approximation. Moreover, if the function to be approximated has an analytic extension, then a nearly geometric rate of approximation can be achieved. We also discuss the problem of approximation of a compact subset of a Euclidean space with such networks with a classical sigmoidal response function.

Keywords

Neural networks uniform approximation multivariate splines analytic functions modulus of smoothness 

Subject classification

(AMS) 41A15 41A63 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    A.R. Barron, Universal approximation bounds for superposition of a sigmoidal function, Preprint (November 1990).Google Scholar
  2. [2]
    S.M. Caroll and S.M. Dickinson, Construction of neural nets using the radon transform, Preprint (1990).Google Scholar
  3. [3]
    T.P. Chen, H. Chen and R.W. Liu, A constructive proof of approximation by superposition of sigmoidal functions for neural networks, Preprint (1990).Google Scholar
  4. [4]
    C.K. Chui and X. Li, Approximation by ridge functions and neural networks with one hidden layer, CAT Report No. 222, Texas A&M University (1990).Google Scholar
  5. [5]
    C.K. Chui and X. Li, Realization of neural networks with one hidden layer, CAT Report No. 244, Texas A&M University (March 1991).Google Scholar
  6. [6]
    G. Cybenko, Approximation by superposition of sigmoidal functions, Math. Control, Signals and Systems 2(1989)303–314.Google Scholar
  7. [7]
    W. Dahmen and C.A. Micchelli, Some remarks on ridge functions, Approx. Theory Appl. 3(1987)139–143.Google Scholar
  8. [8]
    R. DeVore, R. Howard and C.A. Micchelli, Optimal nonlinear approximation, Manuscripta Mathematica 63(1989)469–478.Google Scholar
  9. [9]
    G. Freud and V.A. Popov, On approximation by spline functions,Proc. Conf. on Constructive Theory of Functions (Akadémiai Kiadó, 1972) pp. 163–173.Google Scholar
  10. [10]
    K.I. Funahashi, On the approximate realization of continuous mappings by neural networks, Neural Networks 2(1989)183–192.Google Scholar
  11. [11]
    R. Hecht-Nielsen,Neurocomputing (Addison-Wesley, New York, 1989).Google Scholar
  12. [12]
    K. Hornik, M. Stinchcombe and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2(1989)359–366.Google Scholar
  13. [13]
    B. Irie and S. Miyake, Capabilities of three layered perceptrons,IEEE Int. Conf. on Neural Networks, Vol. 1 (1988) pp. 641–648.Google Scholar
  14. [14]
    Leshno, Lin, Pinkus and Schocken, manuscript.Google Scholar
  15. [15]
    H.N. Mhaskar and C.A. Micchelli, Approximation by superposition of a sigmoidal function, Adv. Appl. Math. 13(1992)350–373.Google Scholar
  16. [16]
    T. Poggio and F. Girosi, Regularization algorithms for learning that are equivalent to multilayer networks, Science 247(1990)978–982.Google Scholar
  17. [17]
    V.A. Popov, Direct and converse theorems for spline approximation with free knots, Bulg. Math. Publ. 1(1975)218–224.Google Scholar
  18. [18]
    L.L. Schumaker,Spline Functions: Basic Theory (Wiley, New York, 1981).Google Scholar
  19. [19]
    J. Siciak, On some extremeal functions and their applications in the theory of analytic functions of several complex variables, Trans. Amer. Math. Soc. 105(1962)322–357.Google Scholar
  20. [20]
    M. Stinchcombe and H. White, Universal approximation using feedforward network with non-sigmoidal hidden layer activation functions, in:Proc. Int. Joint Conf. on Neural Networks, San Diego (SOS printing, 1989) pp. 613–618.Google Scholar
  21. [21]
    M. Stinchcombe and H. White, Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights,IEEE Int. Conf. on Neural Networks, Vol. 3 (1990) pp. III-7–III-16.Google Scholar
  22. [22]
    A.F. Timan,Theory of Approximation of Functions of a Real Variable (Macmillan, New York, 1963).Google Scholar

Copyright information

© J.C. Baltzer AG, Science Publishers 1993

Authors and Affiliations

  • H. N. Mhaskar
    • 1
  1. 1.Department of MathematicsCalifornia State UniversityLos AngelesUSA

Personalised recommendations