An Improved Algorithm for Eleman Neural Network by Adding a Modified Error Function

  • Zhiqiang Zhang
  • Zheng Tang
  • GuoFeng Tang
  • Vairappan Catherine
  • XuGang Wang
  • RunQun Xiong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4492)


The Eleman Neural Network has been widely used in various fields ranging from temporal version of the Exclusive-OR function to the discovery of syntactic categories in natural language date. However, one of the problems often associated with this type of network is the local minima problem which usually occurs in the process of the learning. To solve this problem, we have proposed an error function which can harmonize the update weights connected to the hidden layer and those connected to the output layer by adding one term to the conventional error function. It can avoid the local minima problem caused by this disharmony. We applied this method to the Boolean Series Prediction Questions problems to demonstrate its validity. The result shows that the proposed method can avoid the local minima problem and largely accelerate the speed of the convergence and get good results for the prediction tasks.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Elman, J.L.: Finding Structure in Time. Cognitive Science 14, 179–211 (1990)CrossRefGoogle Scholar
  2. 2.
    Jordan, M.I.: Attractor Dynamics and Parallelism in a Connectionsist Sequential Machine. In: Proceedings of the 8th Conference on Cognitive Science, pp. 531–546 (1986)Google Scholar
  3. 3.
    Omlin, C.W., Giles, C.L.: Extraction of Rules from Dicrete-Time Recurrent Neural Networks. Neural Networks 9(1), 41–52 (1996)CrossRefGoogle Scholar
  4. 4.
    Stagge, P., Sendhoff, B.: Organisation of Past States in Recurrent Neural Networks: Implicit Embedding. In: Mohammadian, M. (ed.) Computational Intelligence for Modelling, Control & Automation, pp. 21–27. IOS Press, Amsterdam (1999)Google Scholar
  5. 5.
    Pham, D.T., Liu, X.: Identification of Linear and Nonlinear Dynamic Systems Using Recurrent Neural Networks. Artificial Intelligence in Engineering 8, 90–97 (1993)Google Scholar
  6. 6.
    Smith, A.: Branch Prediction with Neural Networks: Hidden Layers and Recurrent Connections. Department of Computer Science University of California, San Diego La Jolla, CA 92307 (2004)Google Scholar
  7. 7.
    Cybenko, G.: Approximation by Superposition of a Sigmoid Function. Mathematics of Control, Signals, and Systems 2, 303–314 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Kwok, D.P., Wang, P., Zhou, K.: Process Identification Using a Modified Eleman Neural Network. In: International Symposium on Speech, Image Processing and Neural Networks, pp. 499–502 (1994)Google Scholar
  9. 9.
    Gao, X.Z., Gao, X.M., Ovaska, S.J.: A Modified Eleman Neural Network Model with Application to Dynamical Systems Identification. In: Proceedings of the IEEE International Conference on System, Man and Cybernetics, vol. 2, pp. 1376–1381 (1996)Google Scholar
  10. 10.
    Chagra, W., Abdennour, R.B., Bouani, F., Ksouri, M., Favier, G.: A Comparative Study on the Channel Modeling Using Feedforward and Recurrent Neural Network Structures. In: Proceedings of the IEEE International Conference on System, Man and Cybernetics, vol. 4, pp. 3759–3763 (1998)Google Scholar
  11. 11.
    Kalinli, A., Sagiroglu, S.: Eleman Network with Embedded Memory for System Identification. Journal of Informaiton Science and Engineering 22, 1555–1668 (2006)Google Scholar
  12. 12.
    Servan-Schreiber, C., Printz, H., Cohen, J.: A Network Model of Neuromodulatory Effects: Gain, Signal- to-Noise Ratio and Behavior. Science 249, 892–895 (1990)CrossRefGoogle Scholar
  13. 13.
    Cybenko, G.: Approximation by Superposition of a Sigmoid Function. Mathematics of Control, Signals, and System 2, 303–314 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Wang, X., Tang, Z.: An Improved Backpropagation Algorithm to Avoid the Local Minima Problem. Neurocomputing 56, 455–460 (2004)CrossRefGoogle Scholar
  15. 15.

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Zhiqiang Zhang
    • 1
  • Zheng Tang
    • 1
  • GuoFeng Tang
    • 1
  • Vairappan Catherine
    • 1
  • XuGang Wang
    • 2
  • RunQun Xiong
    • 3
  1. 1.Faculty of Engineering, Toyama University, Gofuku 3190, Toyama shi, 930-8555, dalaosha@hotmail.comJapan
  2. 2.Institute of Software, Chinese Academy of Sciences, BeiJing 100080China
  3. 3.Key Lab of Computer Network and Information Integration, Southeast University, Nanjing 210096China

Personalised recommendations