ICAISC 2008: Artificial Intelligence and Soft Computing – ICAISC 2008 pp 101-110 | Cite as
Nonlinear Function Learning Using Radial Basis Function Networks: Convergence and Rates
Conference paper
Abstract
We apply normalized RBF networks to the problem of learning nonlinear regression functions. The parameters of the networks are learned by empirical risk minimization and complexity regularization. We study convergence of the RBF networks for various radial kernels as the number of training samples increases. The rates of convergence are also examined.
Keywords
Normalized radial basis function networks convergence rates of convergencePreview
Unable to display preview. Download preview PDF.
References
- 1.Anthony, M., Bartlett, P.L.: Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge (1999)MATHGoogle Scholar
- 2.Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. on Information Theory 39, 930–945 (1993)CrossRefMathSciNetMATHGoogle Scholar
- 3.Barron, A.R.: Approximation and estimation bounds for artificial neural networks. Machine Learning 14, 115–133 (1994)MATHGoogle Scholar
- 4.Cybenko, G.: Approximations by superpositions of sigmoidal functions. Mathematics of Control, Signals, and Systems 2, 303–314 (1989)CrossRefMathSciNetMATHGoogle Scholar
- 5.Devroye, L., Györfi, L., Lugosi, G.: Probabilistic Theory of Pattern Recognition. Springer, New York (1996)MATHGoogle Scholar
- 6.Duchon, J.: Sur l’erreur d’interpolation des fonctions de plusieurs variables par les Dm-splines. RAIRO Anal. Numér. 12(4), 325–334 (1978)MathSciNetMATHGoogle Scholar
- 7.Faragó, A., Lugosi, G.: Strong universal consistency of neural network classifiers. IEEE Trans. on Information Theory 39, 1146–1151 (1993)CrossRefMATHGoogle Scholar
- 8.Girosi, F.: Regularization theory, radial basis functions and networks. In: Cherkassky, V., Friedman, J.H., Wechsler, H. (eds.) From Statistics to Neural Networks. Theory and Pattern recognition Applications, pp. 166–187. Springer, Berlin (1992)Google Scholar
- 9.Girosi, F., Anzellotti, G.: Rates of convergence for radial basis functions and neural networks. In: Mammone, R.J. (ed.) Artificial Neural Networks for Speech and Vision, pp. 97–113. Chapman and Hall, London (1993)Google Scholar
- 10.Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural network architectures. Neural Computation 7, 219–267 (1995)CrossRefGoogle Scholar
- 11.Grenander, U.: Abstract Inference. Wiley, New York (1981)MATHGoogle Scholar
- 12.Györfi, L., Kohler, M., Krzyżak, A., Walk, H.: A DistributionFree Theory of Nonparametric Regression. Springer, New York (2002)Google Scholar
- 13.Haussler, D.: Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation 100, 78–150 (1992)CrossRefMathSciNetMATHGoogle Scholar
- 14.Hornik, K., Stinchocombe, S., White, H.: Multilayer feed-forward networks are universal approximators. Neural Networks 2, 359–366 (1989)CrossRefGoogle Scholar
- 15.Kolmogorov, A.N., Tihomirov, V.M.: ε-entropy and ε-capacity of sets in function spaces. Translations of the American Mathematical Society 17, 277–364 (1961)MathSciNetGoogle Scholar
- 16.Krzyżak, A., Linder, T., Lugosi, G.: Nonparametric estimation and classification using radial basis function nets and empirical risk minimization. IEEE Trans. Neural Networks 7(2), 475–487 (1996)CrossRefGoogle Scholar
- 17.Krzyżak, A., Linder, T.: Radial basis function networks and complexity regularization in function learning. IEEE Trans. Neural Networks 9(2), 247–256 (1998)CrossRefGoogle Scholar
- 18.Krzyżak, A., Niemann, H.: Convergence and rates of convergence of radial basis functions networks in function learning. Nonlinear Analysis 47, 281–292 (2001)CrossRefMathSciNetMATHGoogle Scholar
- 19.Krzyżak, A., Schäfer, D.: Nonparametric regression estimation by normalized radial basis function networks. IEEE Transactions on Information Theory 51(3), 1003–1010 (2005)CrossRefGoogle Scholar
- 20.Lugosi, G., Zeger, K.: Nonparametric estimation via empirical risk minimization. IEEE Trans. on Information Theory 41, 677–687 (1995)CrossRefMathSciNetMATHGoogle Scholar
- 21.Moody, J., Darken, J.: Fast learning in networks of locally-tuned processing units. Neural Computation 1, 281–294 (1989)CrossRefGoogle Scholar
- 22.Park, J., Sandberg, I.W.: Universal approximation using Radial-Basis-Function networks. Neural Computation 3, 246–257 (1991)CrossRefGoogle Scholar
- 23.Park, J., Sandberg, I.W.: Approximation and Radial-Basis-Function networks. Neural Computation 5, 305–316 (1993)CrossRefGoogle Scholar
- 24.Pollard, D.: Convergence of Stochastic Processes. Springer, New York (1984)MATHGoogle Scholar
- 25.Rissanen, J.: A universal prior for integers and estimation by minimum description length. Annals of Statistics 11, 416–431 (1983)CrossRefMathSciNetMATHGoogle Scholar
- 26.Ripley, B.D.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (1996)MATHGoogle Scholar
- 27.Shorten, R., Murray-Smith, R.: Side effects of normalising radial basis function networks. International Journal of Neural Systems 7, 167–179 (1996)CrossRefGoogle Scholar
- 28.Specht, D.F.: Probabilistic neural networks. Neural Networks 3, 109–118 (1990)CrossRefGoogle Scholar
- 29.Vapnik, V.N.: Estimation of Dependences Based on Empirical Data, 2nd edn. Springer, New York (1999)Google Scholar
- 30.Xu, L., Krzyżak, A., Yuille, A.L.: On radial basis function nets and kernel regression: approximation ability, convergence rate and receptive field size. Neural Networks 7, 609–628 (1994)CrossRefMATHGoogle Scholar
Copyright information
© Springer-Verlag Berlin Heidelberg 2008