Advertisement

A New Method for Learning RBF Networks by Utilizing Singular Regions

  • Seiya Satoh
  • Ryohei NakanoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10841)

Abstract

The usual way to learn radial basis function (RBF) networks consists of two stages: first, select reasonable weights between input and hidden layers, and then optimize weights between hidden and output layers. When we learn multilayer perceptrons (MLPs), we usually employ the stochastic descent called backpropagation (BP) algorithm or 2nd-order methods such as pseudo-Newton method and conjugate gradient method. Recently new learning methods called singularity stairs following (SSF) methods have been proposed for learning real-valued or complex-valued MLPs by making good use of singular regions. SSF can monotonically decrease training error along with the increase of hidden units, and stably find a series of excellent solutions. This paper proposes a completely new method for learning RBF networks by introducing the SSF paradigm, and compares its performance with those of existing learning methods.

Keywords

Neural networks RBF networks Learning method Singular region Reducibility mapping 

Notes

Acknowledgment

This work was supported by Grants-in-Aid for Scientific Research (C) 16K00342.

References

  1. 1.
    UCI Machine Learning Repository (1996). http://archive.ics.uci.edu/ml/
  2. 2.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Clarendon Press, Oxford (1995)zbMATHGoogle Scholar
  3. 3.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum-likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B 39, 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, Hoboken (1987)zbMATHGoogle Scholar
  5. 5.
    Fukumizu, K., Amari, S.: Local minima and plateaus in hierarchical structure of multilayer perceptrons. Neural Netw. 13(3), 317–327 (2000)CrossRefGoogle Scholar
  6. 6.
    Làzaro, M., Santamarìa, I., Pantaleòn, C.: A new EM-based training algorithm for RBF networks. Neural Netw. 16, 69–77 (2003)CrossRefGoogle Scholar
  7. 7.
    Little, M.A., McSharry, P.E., Roberts, S.J., Costello, D.A.E., Moroz, I.M.: Exploiting nonlinear recurrence and fractal scaling properties for voice disorder detection. Biomed. Eng. OnLine 6, 23 (2007)CrossRefGoogle Scholar
  8. 8.
    Nitta, T.: Local minima in hierarchical structures of complex-valued neural networks. Neural Netw. 43, 1–7 (2013)zbMATHCrossRefGoogle Scholar
  9. 9.
    Satoh, S., Nakano, R.: Multilayer perceptron learning utilizing singular regions and search pruning. In: Proceedings of International Conference on Machine Learning and Data Analysis, pp. 790–795 (2013)Google Scholar
  10. 10.
    Satoh, S., Nakano, R.: A yet faster version of complex-valued multilayer perceptron learning using singular regions and search pruning. In: Proceedings of 7th International Joint Conference on Computational Intelligence (IJCCI), NCTA, vol. 3, pp. 122–129 (2015)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.National Institute of Advanced Industrial Science and TechnologyKoto-kuJapan
  2. 2.Chubu UniversityKasugaiJapan

Personalised recommendations