Advances in Computational Mathematics

, Volume 1, Issue 1, pp 61–80 | Cite as

Approximation properties of a multilayered feedforward artificial neural network

  • H. N. Mhaskar
Articles

Abstract

We prove that an artificial neural network with multiple hidden layers and akth-order sigmoidal response function can be used to approximate any continuous function on any compact subset of a Euclidean space so as to achieve the Jackson rate of approximation. Moreover, if the function to be approximated has an analytic extension, then a nearly geometric rate of approximation can be achieved. We also discuss the problem of approximation of a compact subset of a Euclidean space with such networks with a classical sigmoidal response function.

Keywords

Neural networks uniform approximation multivariate splines analytic functions modulus of smoothness 

Subject classification

(AMS) 41A15 41A63 

References

  1. [1]
    A.R. Barron, Universal approximation bounds for superposition of a sigmoidal function, Preprint (November 1990).Google Scholar
  2. [2]
    S.M. Caroll and S.M. Dickinson, Construction of neural nets using the radon transform, Preprint (1990).Google Scholar
  3. [3]
    T.P. Chen, H. Chen and R.W. Liu, A constructive proof of approximation by superposition of sigmoidal functions for neural networks, Preprint (1990).Google Scholar
  4. [4]
    C.K. Chui and X. Li, Approximation by ridge functions and neural networks with one hidden layer, CAT Report No. 222, Texas A&M University (1990).Google Scholar
  5. [5]
    C.K. Chui and X. Li, Realization of neural networks with one hidden layer, CAT Report No. 244, Texas A&M University (March 1991).Google Scholar
  6. [6]
    G. Cybenko, Approximation by superposition of sigmoidal functions, Math. Control, Signals and Systems 2(1989)303–314.Google Scholar
  7. [7]
    W. Dahmen and C.A. Micchelli, Some remarks on ridge functions, Approx. Theory Appl. 3(1987)139–143.Google Scholar
  8. [8]
    R. DeVore, R. Howard and C.A. Micchelli, Optimal nonlinear approximation, Manuscripta Mathematica 63(1989)469–478.Google Scholar
  9. [9]
    G. Freud and V.A. Popov, On approximation by spline functions,Proc. Conf. on Constructive Theory of Functions (Akadémiai Kiadó, 1972) pp. 163–173.Google Scholar
  10. [10]
    K.I. Funahashi, On the approximate realization of continuous mappings by neural networks, Neural Networks 2(1989)183–192.Google Scholar
  11. [11]
    R. Hecht-Nielsen,Neurocomputing (Addison-Wesley, New York, 1989).Google Scholar
  12. [12]
    K. Hornik, M. Stinchcombe and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2(1989)359–366.Google Scholar
  13. [13]
    B. Irie and S. Miyake, Capabilities of three layered perceptrons,IEEE Int. Conf. on Neural Networks, Vol. 1 (1988) pp. 641–648.Google Scholar
  14. [14]
    Leshno, Lin, Pinkus and Schocken, manuscript.Google Scholar
  15. [15]
    H.N. Mhaskar and C.A. Micchelli, Approximation by superposition of a sigmoidal function, Adv. Appl. Math. 13(1992)350–373.Google Scholar
  16. [16]
    T. Poggio and F. Girosi, Regularization algorithms for learning that are equivalent to multilayer networks, Science 247(1990)978–982.Google Scholar
  17. [17]
    V.A. Popov, Direct and converse theorems for spline approximation with free knots, Bulg. Math. Publ. 1(1975)218–224.Google Scholar
  18. [18]
    L.L. Schumaker,Spline Functions: Basic Theory (Wiley, New York, 1981).Google Scholar
  19. [19]
    J. Siciak, On some extremeal functions and their applications in the theory of analytic functions of several complex variables, Trans. Amer. Math. Soc. 105(1962)322–357.Google Scholar
  20. [20]
    M. Stinchcombe and H. White, Universal approximation using feedforward network with non-sigmoidal hidden layer activation functions, in:Proc. Int. Joint Conf. on Neural Networks, San Diego (SOS printing, 1989) pp. 613–618.Google Scholar
  21. [21]
    M. Stinchcombe and H. White, Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights,IEEE Int. Conf. on Neural Networks, Vol. 3 (1990) pp. III-7–III-16.Google Scholar
  22. [22]
    A.F. Timan,Theory of Approximation of Functions of a Real Variable (Macmillan, New York, 1963).Google Scholar

Copyright information

© J.C. Baltzer AG, Science Publishers 1993

Authors and Affiliations

  • H. N. Mhaskar
    • 1
  1. 1.Department of MathematicsCalifornia State UniversityLos AngelesUSA

Personalised recommendations