Constructive Approximation

, 30:653

On a Constructive Proof of Kolmogorov’s Superposition Theorem

Article

Abstract

Kolmogorov (Dokl. Akad. Nauk USSR, 14(5):953–956, 1957) showed that any multivariate continuous function can be represented as a superposition of one-dimensional functions, i.e.,
$$f(x_{1},\ldots,x_{n})=\sum_{q=0}^{2n}\varPhi _{q}\Biggl(\sum_{p=1}^{n}\psi_{q,p}(x_{p})\Biggr).$$
The proof of this fact, however, was not constructive, and it was not clear how to choose the outer and inner functions Φq and ψq,p, respectively. Sprecher (Neural Netw. 9(5):765–772, 1996; Neural Netw. 10(3):447–457, 1997) gave a constructive proof of Kolmogorov’s superposition theorem in the form of a convergent algorithm which defines the inner functions explicitly via one inner function ψ by ψp,q:=λpψ(xp+qa) with appropriate values λp,a∈ℝ. Basic features of this function such as monotonicity and continuity were supposed to be true but were not explicitly proved and turned out to be not valid. Köppen (ICANN 2002, Lecture Notes in Computer Science, vol. 2415, pp. 474–479, 2002) suggested a corrected definition of the inner function ψ and claimed, without proof, its continuity and monotonicity. In this paper we now show that these properties indeed hold for Köppen’s ψ, and we present a correct constructive proof of Kolmogorov’s superposition theorem for continuous inner functions ψ similar to Sprecher’s approach.

Keywords

Kolmogorov’s superposition theorem Superposition of functions Representation of functions

26B40

References

1. 1.
Arnold, V.: On the representation of functions of several variables by superpositions of functions of fewer variables. Mat. Prosvesh. 3, 41–61 (1958) Google Scholar
2. 2.
Arnold, V.: On functions of three variables. Dokl. Akad. Nauk SSSR 114, 679–681 (1957). English translation: Am. Math. Soc. Transl. (2), 28, 51–54 (1963)
3. 3.
Arnold, V.: On the representation of continuous functions of three variables by superpositions of continuous functions of two variables. Mat. Sb. 48, 3–74 (1959). English translation: Am. Math. Soc. Transl. (2), 28, 61–147 (1963)
4. 4.
de Figueiredo, R.J.P.: Implications and applications of Kolmogorov’s superposition theorem. IEEE Trans. Autom. Control AC-25(6), (1980) Google Scholar
5. 5.
Fridman, B.: An improvement on the smoothness of the functions in Kolmogorov’s theorem on superpositions. Dokl. Akad. Nauk SSSR 177, 1019–1022 (1967). English translation: Soviet Math. Dokl. (8), 1550–1553 (1967)
6. 6.
Girosi, F., Poggio, T.: Representation properties of networks: Kolmogorov’s theorem is irrelevant. Neural Comput. 1, 465–469 (1989)
7. 7.
Hecht-Nielsen, R.: Counter propagation networks. In: Proceedings of the International Conference on Neural Networks II, pp. 19–32 (1987) Google Scholar
8. 8.
Hecht-Nielsen, R.: Kolmogorov’s mapping neural network existence theorem. In: Proceedings of the International Conference on Neural Networks III, pp. 11–14 (1987) Google Scholar
9. 9.
Hedberg, T.: The Kolmogorov superposition theorem, Appendix II to H.S. Shapiro, Topics in Approximation Theory. Lecture Notes in Math., vol. 187, pp. 267–275 (1971) Google Scholar
10. 10.
Hilbert, D.: Mathematical problems. Bull. Am. Math. Soc. 8, 461–462 (1902)
11. 11.
Hurewicz, W., Wallman, H.: Dimension Theory. Princeton University Press, Princeton (1948)
12. 12.
Igelnik, B., Parikh, N.: Kolmogorov’s spline network. IEEE Trans. Neural Netw. 14, 725–733 (2003)
13. 13.
Khavinson, S.: Best approximation by linear superpositions. Transl. Math. Monogr. 159 (1997). AMS Google Scholar
14. 14.
Kolmogorov, A.N.: On the representation of continuous functions of many variables by superpositions of continuous functions of one variable and addition. Dokl. Akad. Nauk USSR 14(5), 953–956 (1957)
15. 15.
Kolmogorov, A.N., Tikhomirov, V.M.: ε-entropy and ε-capacity of sets in function spaces. Usp. Mat. Nauk 13(2), 3–86 (1959). English translation: Am. Math. Soc. Transl. 17(2), 277–364 (1961)
16. 16.
Köppen, M.: On the training of a Kolmogorov network. In: ICANN 2002, Lecture Notes in Computer Science, vol. 2415, pp. 474–479 (2002) Google Scholar
17. 17.
Kurkova, V.: Kolmogorov’s theorem is relevant. Neural Comput. 3, 617–622 (1991)
18. 18.
Kurkova, V.: Kolmogorov’s theorem and multilayer neural networks. Neural Netw. 5, 501–506 (1992)
19. 19.
Lorentz, G.: Approximation of functions. Holt, Rinehart & Winston (1966) Google Scholar
20. 20.
Lorentz, G., Golitschek, M., Makovoz, Y.: Constructive Approximation (1996) Google Scholar
21. 21.
Nakamura, M., Mines, R., Kreinovich, V.: Guaranteed intervals for Kolmogorov’s theorem (and their possible relation to neural networks). Interval Comput. 3, 183–199 (1993)
22. 22.
Nees, M.: Approximative versions of Kolmogorov’s superposition theorem, proved constructively. J. Comput. Appl. Math. 54, 239–250 (1994)
23. 23.
Ostrand, P.A.: Dimension of metric spaces and Hilbert’s problem 13. Bull. Am. Math. Soc. 71, 619–622 (1965)
24. 24.
Rassias, T., Simsa, J.: Finite sum decompositions in mathematical analysis. Pure Appl. Math. (1995) Google Scholar
25. 25.
Sprecher, D.A.: On the structure of continuous functions of several variables. Trans. Am. Math. Soc. 115(3), 340–355 (1965)
26. 26.
Sprecher, D.A.: An improvement in the superposition theorem of Kolmogorov. J. Math. Anal. Appl. 38, 208–213 (1972)
27. 27.
Sprecher, D.A.: A numerical implementation of Kolmogorov’s superpositions. Neural Netw. 9(5), 765–772 (1996)
28. 28.
Sprecher, D.A.: A numerical implementation of Kolmogorov’s superpositions II. Neural Netw. 10(3), 447–457 (1997)