A Note on Learning Automata Based Schemes for Adaptation of BP Parameters

  • M. R. Meybodi
  • H. Beigy
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1983)

Abstract

Backpropagation is often used as the learning algorithm in layered-structure neural networks, because of its efficiency. However, backpropagation is not free from problems. The learning process sometimes gets trapped in a local minimum and the network cannot produce the required response. In addition, The algorithm has number of parameters such learning rate (µ), momentum factor (α) and steepness parameter (λ). whose values are not known in advance, and must be determined by trail and error. The appropriate selection of these parameters have large effect on the convergence of the algorithm. Many techniques that adaptively adjust these parameters have been developed to increase speed of convergence. A class of algorithms which are developed recently uses learning automata (LA) for adjusting the parameters µ,α, and λ based on the observation of random response of the neural networks. One of the important aspects of learning automata based schemes is its remarkable effectiveness as a solution for increasing the speed of convergence. Another important aspect of learning automata based schemes which has not been pointed out earlier is its ability to escape from local minima with high possibility during the training period. In this report we study the ability of LA based schemes in escaping from local minma when standard BP fails to find the global minima. It is demonstrated through simulation that LA based schemes comparing to other schemes such as SAB, Super SAB, Fuzzy BP, ASBP method, and VLR method have higher ability in escaping from local minima.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Rumelhart, D. E., Hinton, G. E. and Williams, R. J.: Learning Internal Representations by Error Propagation. In Parallel Distributed Processing, Cambridge, MA: MIT Press (1986)Google Scholar
  2. 2.
    Arabshahi, P. et al: Fuzzy Parameter Adaptation in Optimazation: Some Neural Net Training Examples. IEEE Computational Science and Engineering (1996)57–65Google Scholar
  3. 3.
    Sarkar, D.: Methods to Speedup Error Backpropagation Learning Algorithm. ACM Computing Surveys 27 (1995) 519–542CrossRefGoogle Scholar
  4. 4.
    Sperduti, A. and Starita, A.: Speed Up Learning and Network Optimization with Extended Backpropagation., Neural Networks, 6(1993) 365–383CrossRefGoogle Scholar
  5. 5.
    Beigy, H., Meybodi, M. R. and Menhaj, M. B.: Adaptation of Learning Rate in Backpropagation Algorithm Using Fixed Structure Learning Automata. Proc. of Iranian Conf. on Electerical Enginerring, ICEE-98 (1998) 117–123Google Scholar
  6. 6.
    Meybodi, M. R. and Beigy, H.: New Class of Learning Automata Based Schemes for Adaptation of Bachpropagation Algorithm Parameters. Iranian Journal of Science and technology (to appear)Google Scholar
  7. 7.
    Beigy, H. and Meybodi, M. R.: Adaptation of Momentum Factor and Steepness Parameter in Backpropagation Algorithm Using Fixed Structure Learning Automata. International Journal of Science and technology (to appear)Google Scholar
  8. 8.
    Menhaj, M. B. and Hagen, M. H.: Rapid Learning Using Modified BP Algorithms for Multi-Layer Feedforward Neural Nets. Proc. of ICEE-95 Conference, University of Science and Technology, Tehran, Iran (1995)Google Scholar
  9. 9.
    Narendra, K. S. and Thathachar, M. A. L.: Learning Automata: An Introduction. Prentice-hall, Englewood cliffs (1989)Google Scholar
  10. 10.
    Gori, M. and Tesi, A.: On the Problem of Local Minima in Backpropagation. IEEE Trans. on Pattern Analysis and Machine Intelligence PAMI-14 (1992) 76–86CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • M. R. Meybodi
    • 1
  • H. Beigy
    • 1
  1. 1.Computer Engineering DepartmentAmirkabir University of TechnologyTehranIran

Personalised recommendations