Neural Computing and Applications

, Volume 18, Issue 2, pp 127–133 | Cite as

A fuzzy neighborhood-based training algorithm for feedforward neural networks

Original Article

Abstract

In this work we present a new hybrid algorithm for feedforward neural networks, which combines unsupervised and supervised learning. In this approach, we use a Kohonen algorithm with a fuzzy neighborhood for training the weights of the hidden layers and gradient descent method for training the weights of the output layer. The goal of this method is to assist the existing variable learning rate algorithms. Simulation results show the effectiveness of the proposed algorithm compared with other well-known learning methods.

Keywords

Feedforward neural network Gradient descent algorithm Supervised and unsupervised learning Fuzzy self-organizing feature map Hybrid training 

References

  1. 1.
    Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. Parallel distributed processing: explorations in the microstructures of cognition. MIT Press, 1:318–362Google Scholar
  2. 2.
    Jacobs RA (1988) Increased rates of convergence through learning rate adaptation. Neural Netw 1:295–307CrossRefGoogle Scholar
  3. 3.
    Silva F, Almeida L (1990) Speeding-up backpropagation. In: Eckmiller R (ed) Advanced neural computers. North-Holland, Amsterdam, pp 151–156Google Scholar
  4. 4.
    Najim K, Chtourou M, Thibault J (1992) Neural network synthesis using learning automata. J Syst Eng 2(4):192–197Google Scholar
  5. 5.
    Zhang N, Wu W, Zheng G (2006) Convergence of gradient method with momentum for two-layer feedforward neural networks. IEEE Trans Neural Netw 17(2):522–525CrossRefGoogle Scholar
  6. 6.
    Bortoletti A, Di FIore C, Fanelli S, Zellini P (2003) A new class of quasi-newtonian methods for optimal learning in MLP-networks. IEEE Trans Neural Netw 14(2):263–273CrossRefGoogle Scholar
  7. 7.
    Lera G, Pinzolas M (2002) Neighborhood based Levenberg-Marquardt algorithm for neural network training. IEEE Trans Neural Netw 13(5):1200–1203CrossRefGoogle Scholar
  8. 8.
    Abid S, Fnaiech F, Najim M (2001) A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm. IEEE Trans Neural Netw 12(2):424–430CrossRefGoogle Scholar
  9. 9.
    Nickolai SR (2000) The layer-wise method and the backpropagation hybrid approch to learning a feedforward neural network. IEEE Trans Neural Netw 11(2):295–305CrossRefGoogle Scholar
  10. 10.
    Oscar FR, Deniz E, Jose CP (2003) Accelerating the convergence speed of neural networks learning methods using least squares. In: Proceedings of ESANN’2003, Belgium, 23–25 April, pp 255–260Google Scholar
  11. 11.
    Bilski J (2005) The UD RLS algorithm for training feedforward neural networks. Int J Appl Math Comput Sci 15(1):115–123MATHGoogle Scholar
  12. 12.
    Leung CS, Tsoi AC, Chan LW (2001) Two regularizers for recursive least squared algorithm in feedforward multilayered neural networks. IEEE Trans Neural Netw 12(6):1314–1332CrossRefGoogle Scholar
  13. 13.
    Huntsberger TL, Ajjimarangsee P (1989) Parallel self organizing feature maps for unsupervised pattern recognition. Int J Gen Syst 16:357–372CrossRefGoogle Scholar
  14. 14.
    Bezdek JC, Tsao EC, Pal NR (1992) Fuzzy Kohonen clustering networks. In: Proceedings of IEEE international conference on fuzzy systems, March 1992, San Diego, pp 1035–1041Google Scholar
  15. 15.
    Zweiri YH, Whidborne JF, Seneviratne LD (2003) Three-term backpropagation algorithm. Neurocomputing 50:305–318MATHCrossRefGoogle Scholar
  16. 16.
    Sha D, Bajie BV (2002) An on line hybrid learning algorithm for multilayer perceptron in identification problems. Comput Electr Eng 28:587–598MATHCrossRefGoogle Scholar
  17. 17.
    Liu P, Li H (2004) Efficient learning algorithms for three-layer regular feedforward fuzzy neural networks. IEEE Trans Neural Netw 15(3):545–558MATHCrossRefGoogle Scholar
  18. 18.
    Ben Nasr M, Chtourou M (2006) A hybrid training algorithm for feedforward neural networks. Neural Process Lett 24(2):107–117CrossRefGoogle Scholar
  19. 19.
    Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2:359–366CrossRefGoogle Scholar
  20. 20.
    Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480CrossRefGoogle Scholar
  21. 21.
    Alexander GP, Benito F, Amir FA, Jayakumar M, Wei KT (1994) An accelerated learning algorithm for multilayer networks. IEEE Trans Neural Netw 5(3):493–497CrossRefGoogle Scholar
  22. 22.
    Mackey MC, Glass L (1977) Oscillation and chaos in physiological control system. Science 197:287–289CrossRefGoogle Scholar
  23. 23.
    Box GE, Jenkins GM (1970) Time series analysis, forecasting and control. Holden Day, San FranciscoMATHGoogle Scholar
  24. 24.
    Kim J (1999) Adaptive neuro-fuzzy inference system and their application to non linear dynamical system. Neural Netw 12:1301–1319CrossRefGoogle Scholar
  25. 25.
    Ben Nasr M, Chtourou M, A constructive hybrid training algorithm for feedforward neural networks. Submitted to Int J Syst SciGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2007

Authors and Affiliations

  1. 1.Department of Electrical EngineeringResearch Unit on Intelligent Control, Design and Optimization of Complex Systems (ICOS), ENISSfaxTunisia

Personalised recommendations