Constructive Approximation

, 30:653 | Cite as

On a Constructive Proof of Kolmogorov’s Superposition Theorem

Article

Abstract

Kolmogorov (Dokl. Akad. Nauk USSR, 14(5):953–956, 1957) showed that any multivariate continuous function can be represented as a superposition of one-dimensional functions, i.e.,
$$f(x_{1},\ldots,x_{n})=\sum_{q=0}^{2n}\varPhi _{q}\Biggl(\sum_{p=1}^{n}\psi_{q,p}(x_{p})\Biggr).$$
The proof of this fact, however, was not constructive, and it was not clear how to choose the outer and inner functions Φq and ψq,p, respectively. Sprecher (Neural Netw. 9(5):765–772, 1996; Neural Netw. 10(3):447–457, 1997) gave a constructive proof of Kolmogorov’s superposition theorem in the form of a convergent algorithm which defines the inner functions explicitly via one inner function ψ by ψp,q:=λpψ(xp+qa) with appropriate values λp,a∈ℝ. Basic features of this function such as monotonicity and continuity were supposed to be true but were not explicitly proved and turned out to be not valid. Köppen (ICANN 2002, Lecture Notes in Computer Science, vol. 2415, pp. 474–479, 2002) suggested a corrected definition of the inner function ψ and claimed, without proof, its continuity and monotonicity. In this paper we now show that these properties indeed hold for Köppen’s ψ, and we present a correct constructive proof of Kolmogorov’s superposition theorem for continuous inner functions ψ similar to Sprecher’s approach.

Keywords

Kolmogorov’s superposition theorem Superposition of functions Representation of functions 

Mathematics Subject Classification (2000)

26B40 

References

  1. 1.
    Arnold, V.: On the representation of functions of several variables by superpositions of functions of fewer variables. Mat. Prosvesh. 3, 41–61 (1958) Google Scholar
  2. 2.
    Arnold, V.: On functions of three variables. Dokl. Akad. Nauk SSSR 114, 679–681 (1957). English translation: Am. Math. Soc. Transl. (2), 28, 51–54 (1963) MathSciNetGoogle Scholar
  3. 3.
    Arnold, V.: On the representation of continuous functions of three variables by superpositions of continuous functions of two variables. Mat. Sb. 48, 3–74 (1959). English translation: Am. Math. Soc. Transl. (2), 28, 61–147 (1963) MathSciNetGoogle Scholar
  4. 4.
    de Figueiredo, R.J.P.: Implications and applications of Kolmogorov’s superposition theorem. IEEE Trans. Autom. Control AC-25(6), (1980) Google Scholar
  5. 5.
    Fridman, B.: An improvement on the smoothness of the functions in Kolmogorov’s theorem on superpositions. Dokl. Akad. Nauk SSSR 177, 1019–1022 (1967). English translation: Soviet Math. Dokl. (8), 1550–1553 (1967) MathSciNetGoogle Scholar
  6. 6.
    Girosi, F., Poggio, T.: Representation properties of networks: Kolmogorov’s theorem is irrelevant. Neural Comput. 1, 465–469 (1989) CrossRefGoogle Scholar
  7. 7.
    Hecht-Nielsen, R.: Counter propagation networks. In: Proceedings of the International Conference on Neural Networks II, pp. 19–32 (1987) Google Scholar
  8. 8.
    Hecht-Nielsen, R.: Kolmogorov’s mapping neural network existence theorem. In: Proceedings of the International Conference on Neural Networks III, pp. 11–14 (1987) Google Scholar
  9. 9.
    Hedberg, T.: The Kolmogorov superposition theorem, Appendix II to H.S. Shapiro, Topics in Approximation Theory. Lecture Notes in Math., vol. 187, pp. 267–275 (1971) Google Scholar
  10. 10.
    Hilbert, D.: Mathematical problems. Bull. Am. Math. Soc. 8, 461–462 (1902) CrossRefMathSciNetGoogle Scholar
  11. 11.
    Hurewicz, W., Wallman, H.: Dimension Theory. Princeton University Press, Princeton (1948) MATHGoogle Scholar
  12. 12.
    Igelnik, B., Parikh, N.: Kolmogorov’s spline network. IEEE Trans. Neural Netw. 14, 725–733 (2003) CrossRefGoogle Scholar
  13. 13.
    Khavinson, S.: Best approximation by linear superpositions. Transl. Math. Monogr. 159 (1997). AMS Google Scholar
  14. 14.
    Kolmogorov, A.N.: On the representation of continuous functions of many variables by superpositions of continuous functions of one variable and addition. Dokl. Akad. Nauk USSR 14(5), 953–956 (1957) MathSciNetGoogle Scholar
  15. 15.
    Kolmogorov, A.N., Tikhomirov, V.M.: ε-entropy and ε-capacity of sets in function spaces. Usp. Mat. Nauk 13(2), 3–86 (1959). English translation: Am. Math. Soc. Transl. 17(2), 277–364 (1961) MathSciNetGoogle Scholar
  16. 16.
    Köppen, M.: On the training of a Kolmogorov network. In: ICANN 2002, Lecture Notes in Computer Science, vol. 2415, pp. 474–479 (2002) Google Scholar
  17. 17.
    Kurkova, V.: Kolmogorov’s theorem is relevant. Neural Comput. 3, 617–622 (1991) CrossRefGoogle Scholar
  18. 18.
    Kurkova, V.: Kolmogorov’s theorem and multilayer neural networks. Neural Netw. 5, 501–506 (1992) CrossRefGoogle Scholar
  19. 19.
    Lorentz, G.: Approximation of functions. Holt, Rinehart & Winston (1966) Google Scholar
  20. 20.
    Lorentz, G., Golitschek, M., Makovoz, Y.: Constructive Approximation (1996) Google Scholar
  21. 21.
    Nakamura, M., Mines, R., Kreinovich, V.: Guaranteed intervals for Kolmogorov’s theorem (and their possible relation to neural networks). Interval Comput. 3, 183–199 (1993) MathSciNetGoogle Scholar
  22. 22.
    Nees, M.: Approximative versions of Kolmogorov’s superposition theorem, proved constructively. J. Comput. Appl. Math. 54, 239–250 (1994) MATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Ostrand, P.A.: Dimension of metric spaces and Hilbert’s problem 13. Bull. Am. Math. Soc. 71, 619–622 (1965) MATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    Rassias, T., Simsa, J.: Finite sum decompositions in mathematical analysis. Pure Appl. Math. (1995) Google Scholar
  25. 25.
    Sprecher, D.A.: On the structure of continuous functions of several variables. Trans. Am. Math. Soc. 115(3), 340–355 (1965) MATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Sprecher, D.A.: An improvement in the superposition theorem of Kolmogorov. J. Math. Anal. Appl. 38, 208–213 (1972) MATHCrossRefMathSciNetGoogle Scholar
  27. 27.
    Sprecher, D.A.: A numerical implementation of Kolmogorov’s superpositions. Neural Netw. 9(5), 765–772 (1996) CrossRefGoogle Scholar
  28. 28.
    Sprecher, D.A.: A numerical implementation of Kolmogorov’s superpositions II. Neural Netw. 10(3), 447–457 (1997) CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Institute for Numerical SimulationUniversity of BonnBonnGermany

Personalised recommendations