Skip to main content
SpringerLink
Log in
Book cover

International Conference on Artificial Intelligence and Soft Computing

ICAISC 2018: Artificial Intelligence and Soft Computing pp 118–129Cite as

Learning and Convergence of the Normalized Radial Basis Functions Networks

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10841))

Abstract

In the paper we analyze convergence and rates of convergence of the normalized radial basis function networks by relating their \(L_2\) error to the \(L_2\) error of the Wolverton-Wagner regression estimate. The network parameters are learned by minimizing the empirical risk and are applied in function learning and classification.

A. Krzyżak—Research of the first author was supported by the Natural Sciences and Engineering Research Council of Canada under Grant RGPIN-2015-06412. He carried out this research at WUT during his sabbatical leave from Concordia University.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Anthony, M., Bartlett, P.L.: Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge (1999)

    Book  MATH  Google Scholar 

  2. Azuma, K.: Weighted sums of certain dependent random variables. Tohoku Math. J. 19(3), 357–367 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  3. Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inf. Theory 39, 930–945 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  4. Beirlant, J., Györfi, L.: On the asymptotic \({L}_2\)-error in partitioning regression estimation. J. Stat. Plan. Inference 71, 93–107 (1998)

    MATH  Google Scholar 

  5. Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)

    Article  MATH  Google Scholar 

  6. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth Advanced Books and Software, Belmont, CA (1984)

    Google Scholar 

  7. Broomhead, D.S., Lowe, D.: Multivariable functional interpolation and adaptive networks. Complex Syst. 2, 321–323 (1988)

    MathSciNet  MATH  Google Scholar 

  8. Cybenko, G.: Approximations by superpositions of sigmoidal functions. Math. Control Sig. Syst. 2, 303–314 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  9. Devroye, L.: Any discrimination rule can have arbitrary bad probability of error for finite sample size. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-4, 154–157 (1982)

    Google Scholar 

  10. Devroye, L.P., Wagner, T.J.: On the L1 convergence of the kernel estimators of regression functions with applications in discrimination. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 51(1), 15–25 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  11. Devroye, L., Györfi, L., Lugosi, G.: Probabilistic Theory of Pattern Recognition. Springer, New York (1996). https://doi.org/10.1007/978-1-4612-0711-5

    Book  MATH  Google Scholar 

  12. Devroye, L., Györfi, L., Krzyżak, A., Lugosi, G.: On the strong universal consistency of nearest neighbor regression function estimates. Ann. Stat. 22, 1371–1385 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  13. Devroye, L., Krzyżak, A.: An equivalence theorem for \(L_1\) convergence of the kernel regression estimate. J. Stat. Plan. Inference 23, 71–82 (1989)

    MathSciNet  MATH  Google Scholar 

  14. Devroye, L., Biau, G.: Lectures on the Nearest Neighbor Method. Springer, New York (2015). https://doi.org/10.1007/978-3-319-25388-6

    Book  MATH  Google Scholar 

  15. Duchon, J.: Sur l’erreur d’interpolation des fonctions de plusieurs variables par les \(D^{m}\)-splines. RAIRO-Analyse Numèrique 12(4), 325–334 (1978)

    MathSciNet  MATH  Google Scholar 

  16. Faragó, A., Lugosi, G.: Strong universal consistency of neural network classifiers. IEEE Trans. Inf. Theory 39, 1146–1151 (1993)

    Article  MATH  Google Scholar 

  17. Girosi, F., Anzellotti, G.: Rates of convergence for radial basis functions and neural networks. In: Mammone, R.J. (ed.) Artificial Neural Networks for Speech and Vision, pp. 97–113. Chapman and Hall, London (1993)

    Google Scholar 

  18. Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural network architectures. Neural Comput. 7, 219–267 (1995)

    Article  Google Scholar 

  19. Greblicki, W.: Asymptotically Optimal Probabilistic Algorithms for Pattern Recognition and Identification. Monografie No. 3. Prace Naukowe Instytutu Cybernetyki Technicznej Politechniki Wroclawskiej, Nr. 18, Wroclaw, Poland (1974)

    Google Scholar 

  20. Greblicki, W., Pawlak, M.: Fourier and Hermite series estimates of regression functions. Ann. Inst. Stat. Math. 37, 443–454 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  21. Greblicki, W., Pawlak, M.: Necessary and sufficient conditions for Bayes risk consistency of a recursive kernel classification rule. IEEE Trans. Inf. Theory, IT-33, 408–412 (1987)

    Google Scholar 

  22. Györfi, L., Kohler, M., Krzyżak, A., Walk, H.: A Distribution-Free Theory of Nonparametric Regression. Springer, New York (2002). https://doi.org/10.1007/b97848

    Book  MATH  Google Scholar 

  23. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning; Data Mining, Inference and Prediction, 2nd edn. Springer, New York (2009). https://doi.org/10.1007/978-0-387-84858-7

    Book  MATH  Google Scholar 

  24. Haykin, S.O.: Neural Networks and Learning Machines, 3rd edn. Prentice-Hall, New York (2008)

    Google Scholar 

  25. Hornik, K., Stinchocombe, S., White, H.: Multilayer feed-forward networks are universal approximators. Neural Netw. 2, 359–366 (1989)

    Article  MATH  Google Scholar 

  26. Kohler, M., Krzyżak, A.: Nonparametric regression based on hierarchical interaction models. IEEE Trans. Inf. Theory 63, 1620–1630 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  27. Krzyżak, A.: The rates of convergence of kernel regression estimates and classification rules. IEEE Trans. Inf. Theory, IT-32, 668–679 (1986)

    Google Scholar 

  28. Krzyżak, A.: Global convergence of recursive kernel regression estimates with applications in classification and nonlinear system estimation. IEEE Trans. Inf. Theory IT-38, 1323–1338 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  29. Krzyżak, A., Linder, T., Lugosi, G.: Nonparametric estimation and classification using radial basis function nets and empirical risk minimization. IEEE Trans. Neural Netw. 7(2), 475–487 (1996)

    Article  Google Scholar 

  30. Krzyżak, A., Linder, T.: Radial basis function networks and complexity regularization in function learning. IEEE Trans. Neural Netw. 9(2), 247–256 (1998)

    Article  Google Scholar 

  31. Krzyżak, A., Niemann, H.: Convergence and rates of convergence of radial basis functions networks in function learning. Nonlinear Anal. 47, 281–292 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  32. Krzyżak, A., Partyka, M.: Convergence and rates of convergence of recursive radial basis functions networks in function learning and classification. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2017. LNCS (LNAI), vol. 10245, pp. 107–117. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59063-9_10

    Chapter  Google Scholar 

  33. Krzyżak, A., Pawlak, M.: Universal consistency results for the Wolverton-Wagner regression estimate with application in discrimination. Probl. Control Inf. Theory 12, 33–42 (1983)

    MathSciNet  MATH  Google Scholar 

  34. Krzyżak, A., Pawlak, M.: Distribution-free consistency of a nonparametric kernel regression estimate and classification. IEEE Trans. Inf. Theory IT-30, 78–81 (1984)

    Google Scholar 

  35. Krzyżak, A., Schäfer, D.: Nonparametric regression estimation by normalized radial basis function networks. IEEE Trans. Inf. Theory 51, 1003–1010 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  36. Lugosi, G., Zeger, K.: Nonparametric estimation via empirical risk minimization. IEEE Trans. Inf. Theory 41, 677–687 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  37. McDiarmid, C.: On the method of bounded differences. Surv. Comb. 141, 148–188 (1989)

    MathSciNet  MATH  Google Scholar 

  38. Moody, J., Darken, J.: Fast learning in networks of locally-tuned processing units. Neural Comput. 1, 281–294 (1989)

    Article  Google Scholar 

  39. Park, J., Sandberg, I.W.: Universal approximation using Radial-Basis-Function networks. Neural Comput. 3, 246–257 (1991)

    Article  Google Scholar 

  40. Park, J., Sandberg, I.W.: Approximation and Radial-Basis-Function networks. Neural Comput. 5, 305–316 (1993)

    Article  Google Scholar 

  41. Ripley, B.D.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (2008)

    MATH  Google Scholar 

  42. Scornet, E., Biau, G., Vert, J.-P.: Consistency of random forest. Ann. Stat. 43(4), 1716–1741 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  43. Shorten, R., Murray-Smith, R.: Side effects of normalising radial basis function networks. Int. J. Neural Syst. 7, 167–179 (1996)

    Article  Google Scholar 

  44. Specht, D.F.: Probabilistic neural networks. Neural Netw. 3, 109–118 (1990)

    Article  Google Scholar 

  45. Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16, 264–280 (1971)

    Article  MATH  Google Scholar 

  46. Vapnik, V.N.: Estimation of Dependences Based on Empirical Data. Springer, New York (1999). https://doi.org/10.1007/0-387-34239-7

    Book  MATH  Google Scholar 

  47. White, H.: Connectionist nonparametric regression: multilayer feedforward networks that can learn arbitrary mappings. Neural Netw. 3, 535–549 (1990)

    Article  Google Scholar 

  48. Wolverton, C.T., Wagner, T.J.: Asymptotically optimal discriminant functions for pattern classification. IEEE Trans. Inf. Theory IT-15, 258–265 (1969)

    Google Scholar 

  49. Xu, L., Krzyżak, A., Yuille, A.L.: On radial basis function nets and kernel regression: approximation ability, convergence rate and receptive field size. Neural Netw. 7, 609–628 (1994)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Software Engineering, Concordia University, 1455 de Maisonneuve Blvd. West, Montreal, H3G 1M8, Canada

    Adam Krzyżak

  2. Department of Electrical Engineering, Westpomeranian University of Technology, 70-313, Szczecin, Poland

    Adam Krzyżak

  3. Department of Knowledge Engineering, Faculty of Production Engineering and Logistics, Opole University of Technology, ul. Ozimska 75, 45-370, Opole, Poland

    Marian Partyka

Authors
  1. Adam Krzyżak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marian Partyka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adam Krzyżak .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  3. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  4. University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  5. AGH University of Science and Technology, Kraków, Poland

    Ryszard Tadeusiewicz

  6. University of Louisville, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Krzyżak, A., Partyka, M. (2018). Learning and Convergence of the Normalized Radial Basis Functions Networks. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2018. Lecture Notes in Computer Science(), vol 10841. Springer, Cham. https://doi.org/10.1007/978-3-319-91253-0_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91253-0_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91252-3

  • Online ISBN: 978-3-319-91253-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation