Convergence rate of minimization learning for neural networks

  • Marghny H. Mohamed
  • Teruya Minamoto
  • Koichi Niijima
Neural Networks
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1398)


In this paper, we present the convergence rate of the error in a neural network which was learnt by a constructive method. The constructive mechanism is used to learn the neural network by adding hidden units to this neural network. The main idea of this work is to find the eigenvalues of the transformation matrix concerning the error before and after adding hidden units in the neural network. By using the eigenvalues, we show the relation between the convergence rate in neural networks without and with thresholds in the output layer.


Neural Network Hide Layer Convergence Rate Output Layer Connection Weight 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. [1]
    Giles, C.J., Chen, D., Sun, G., Chen, H., Lee, Y., and Goudreau, M.W.: Constructive learning of recurrent neural networks: Limitations of recurrent cascade correlation and a simple solution. IEEE Trans. Neural Networks. 6(1) (1995) 829–836CrossRefGoogle Scholar
  2. [2]
    Hassibi, B., Stork, D. G.: Second order derivatives for network pruning: Optimal brain surgeon. In S.J. Hanson, J.D.Cowan, C.L. Giles (Eds.). Advances in Neural Information Processing Systems. 5 (1993) 164–171, San Mateo, CA: Morgan KaufmannGoogle Scholar
  3. [3]
    Mohamed, H.M., Minamoto, T., and Niijima, K.: Convergence rate of minimization learning for neural networks. DOI Technical Report, DOI-TR-141, Kyushu University, (1997)Google Scholar
  4. [4]
    Niijima, K., Yamada, M., Mohamed,H.M., Akanuma, T., Minamoto, T., and Ohkubo, A.: Minimization learning of neural networks by adding hidden units. Research Reports on Information Science and Electrical Engineering of Kyushu University. 2(2) (1997) 173–178Google Scholar
  5. [5]
    Parekh, R., Yang, J., Honavar, V.: Constructive Neural Networks Learning Algorithms for Multi-Category Pattern Classification. Technical Report TR95-15, AI Research Group, Dept. of Computer Science, Iowa State UniversityGoogle Scholar
  6. [6]
    Weymaere, N., and Martens, J.: On the initialization and optimization of multilayer perceptrons. IEEE Trans. Neural Networks. 5(5) (1994) 738–751CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Marghny H. Mohamed
    • 1
  • Teruya Minamoto
    • 1
  • Koichi Niijima
    • 1
  1. 1.Department of InformaticsKyushu UniversityFukuoka, KasukaJapan

Personalised recommendations