Neural Computing and Applications

, Volume 18, Issue 2, pp 157–174 | Cite as

Identification using ANFIS with intelligent hybrid stable learning algorithm approaches

  • Mahdi Aliyari Shoorehdeli
  • Mohammad Teshnehlab
  • Ali Khaki Sedigh
Original Article


This paper suggests novel hybrid learning algorithm with stable learning laws for adaptive network based fuzzy inference system (ANFIS) as a system identifier and studies the stability of this algorithm. The new hybrid learning algorithm is based on particle swarm optimization (PSO) for training the antecedent part and gradient descent (GD) for training the conclusion part. Lyapunov stability theory is used to study the stability of the proposed algorithm. This paper, studies the stability of PSO as an optimizer in training the identifier, for the first time. Stable learning algorithms for the antecedent and consequent parts of fuzzy rules are proposed. Some constraints are obtained and simulation results are given to validate the results. It is shown that instability will not occur for the leaning rate and PSO factors in the presence of constraints. The learning rate can be calculated on-line and will provide an adaptive learning rate for the ANFIS structure. This new learning scheme employs adaptive learning rate that is determined by input–output data.


Learning rate Hybrid learning algorithm Intelligent optimization Fuzzy systems TSK system ANFIS Lyapunov theory Identification Stability analysis Fuzzy neural networks Gradient based Recursive least square and particle swarm optimization 


  1. 1.
    Zhou Y, Li S, Jin R (2002) A new fuzzy neural network with fast learning algorithm and guaranteed stability for manufacturing process control. Fuzzy Sets Syst 132(2):201–216MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Chen MS (1999) A comparative study of learning methods in tuning parameters of fuzzy membership functions. IEEE Conf on Syst Man Cybern 3:40–44Google Scholar
  3. 3.
    Yu W, Li X (2004) Fuzzy identification using fuzzy neural networks with stable learning algorithms. IEEE Trans Fuzzy Syst 12(3):411–420CrossRefGoogle Scholar
  4. 4.
    Polycarpou MM, Ioannou PA (1992) Learning and convergence analysis of neural-type structured networks. IEEE Trans Neural Netw 3(1):39–50CrossRefGoogle Scholar
  5. 5.
    Kim WC, Ahn SC, Kwon WH (1995) Stability analysis and stabilization of fuzzy state space models. Fuzzy Sets Syst 71(1):131–142MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Lee C-H, Teng C-C (2000) Identification and control of dynamic systems using recurrent fuzzy neural networks. IEEE Trans Fuzzy Syst 8(4):349–366CrossRefGoogle Scholar
  7. 7.
    Yu W, Li X (2003) Fuzzy neural modeling using stable learning algorithm. In: Proceedings of American control conference, pp 4542–4547Google Scholar
  8. 8.
    Jang J-SR (1993) ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23(3)Google Scholar
  9. 9.
    Yager RR, Zadeh LA (1994) Fuzzy sets neural networks, and soft computing. Thomson LearningGoogle Scholar
  10. 10.
    Kumar M, Garg DP (2004) Intelligent learning of fuzzy logic controllers via neural network and genetic algorithm. In: Proceedings of Japan–USA symposium on flexible automation, Denver CoGoogle Scholar
  11. 11.
    Mascioli FM, Varazi GM, Martinelli G (1997) Constructive algorithm for neuro-fuzzy networks. In: Proceedings of the sixth IEEE international conference on fuzzy systems, vol 1, pp 459–464Google Scholar
  12. 12.
    Jang J-SR, Mizutani E (1996) Levenberg–Marquardt method for ANFIS learning. In: Proceedings of Biennial conference of the North American fuzzy information processing society, pp 87–91Google Scholar
  13. 13.
    Jang J-SR (1996) Input selection for ANFIS learning. In: Proceedings of the fifth IEEE international conference on fuzzy systems, vol 2, pp 1493–1499Google Scholar
  14. 14.
    Aliyari M Sh, Teshnehlab M, Sedigh AK (2006) A novel training algorithm in ANFIS structure. In: Proceedings of the American control conferencesGoogle Scholar
  15. 15.
    Eberhart RC, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of 6th international symposium micromachine human sci, vol 1, pp 39–43Google Scholar
  16. 16.
    Shi Y, Eberhart RC (1999) Empirical study of particle swarm optimization. In: Proceedings of IEEE congress on evolution of computation, pp 1945–1950Google Scholar
  17. 17.
    Mendes R, Kennedy J, Neves J (2004) The fully informed particle swarm: simpler, maybe better. IEEE Trans Evol Comput 8(3):204–210CrossRefGoogle Scholar
  18. 18.
    Eberhart RC, Hu X (1999) Human tremor analysis using particle swarm optimization. In: Proceedings of IEEE congress on evolution of computation, pp 1927–1930Google Scholar
  19. 19.
    Yoshida H, Kawata K, Fukuyama Y, Takayama S, Naknishi Y (2000) A particle swarm optimization for reactive power and voltage control considering voltage security assessment. IEEE Trans Power Syst 15(4):1232–1239CrossRefGoogle Scholar
  20. 20.
    Ciuprina G, Ioan D, Munteanu I (2002) Use of intelligent-particle swarm optimization in electromagnetics. IEEE Trans Magn 38(2):1037–1040CrossRefGoogle Scholar
  21. 21.
    M.Wachowiak, Smolikova R, Zheng Y, Zurada J, Elmaghraby A (2004) An approach to multimodal biomedical image registration utilizing particle swarm optimization. IEEE Trans Evol Comput 8(3):289–301CrossRefGoogle Scholar
  22. 22.
    Messerschmidt L, Engelbrecht A (2004) Learning to play games using a PSO-based competitive learning approach. IEEE Trans Evol Comput 8(3):280–288CrossRefGoogle Scholar
  23. 23.
    Ratnaweera A, Halgamuge S, Watson H (2004) Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans Evol Comput 8(3):240–255CrossRefGoogle Scholar
  24. 24.
    Coello CAC, Pulido G, Lechuga M (2004) Handling multiple objectives with particle swarm optimization. IEEE Trans Evol Comput 8(3):256–279CrossRefGoogle Scholar
  25. 25.
    Eberhart RC, Hu X (2002) Multiobjective optimization using dynamic neighborhood particle swarm optimization. In: Proceedings of IEEE congress on evolution of computation, pp 1677–1681Google Scholar
  26. 26.
    Parsopouls KE, Vrahatis MN (2002) Particle swarm optimization method in multiobjective problems. In: Proceedings of ACMSymp appl comput evol comput, pp 603–607Google Scholar
  27. 27.
    Hu X, Eberhart RC, Shi Y (2003) Particle swarm with extended memory for multiobjective optimization. In: Proceedings of IEEE swarm Intell symp, Indianapolis, pp 193–197Google Scholar
  28. 28.
    Angeline PJ (1998) Using selection to improve particle swami optimization. In: Proceedings of IEEE congress on evolution of computation, pp 84–89Google Scholar
  29. 29.
    Suganthan PN (1999) Particle swarm optimizer with neighborhood operator. In: Proceedings of IEEE congress on evolution of computation, pp 1958–1962Google Scholar
  30. 30.
    Kennedy J (1998) The behavior of particle. In: Proceedings of 7th Annu Conf Evol Program, pp 581–589Google Scholar
  31. 31.
    Kadirkamanathan V, Selvarajah K, Fleming PJ (2006) Stability analysis of the particle dynamics in particle swarm optimizer. IEEE Trans Evol Comput 10(3):245–255CrossRefGoogle Scholar
  32. 32.
    Ozcan E, Mohan CK (1999) Particle swami optimization: surfing the waves. In: Proceedings of IEEE congress on evolution of computation, pp 1939–1944Google Scholar
  33. 33.
    Mahfouf M, Chen M-Y, Linkens DA (2004) Adaptive weighted particle swarm optimization for multi-objective optimal design of alloy steels. In: Proceedings of parallel problem solving from nature VIII conference, pp 761–771Google Scholar
  34. 34.
    Ho SL, Shiyou Yang, Guangzheng Ni, Edward WC Lo, Wong HC (2005) A particle swarm optimization-based method for multiobjective design optimizations. IEEE Trans Magn 41(5):1756–1759CrossRefGoogle Scholar
  35. 35.
    Naka S, Genji T, Yura T, Fukuyama Y (2002) Hybrid particle swarm optimization based distribution state estimation using constriction factor approach. In: Proceedings of Int Conf SCIS & ISIS, vol 2, pp 1083–1088Google Scholar
  36. 36.
    Miranda V, Fonseca N (2002) EPSO—evolutionary particle swarm optimization, a new algorithm with applications in power systems. In: Proceedings of IEEE Trans distribution Conf exhibition, vol 2, pp 6–10Google Scholar
  37. 37.
    Miranda V, Fonseca N (2002) EPSO-best-of-two-worlds meta-heuristic applied to power system problems. In: Proceedings of IEEE congress on evolution of computation, vol 2, pp 12–17Google Scholar
  38. 38.
    Clerc M (1999) The swarm and the queen: towards a deterministic and adaptive particle swarm optimization. In: Proceedings of IEEE Int Conf Evol Comput, pp 1951–1957Google Scholar
  39. 39.
    Eberhart R, Shi Y (2000) Comparing inertia weights and constriction factors in particle swarm optimization. In: Proceedings of IEEE congress on evolution of computation, pp 84–88Google Scholar
  40. 40.
    Heo JS, Lee KY, Garduno-Ramirez R (2006) Multiobjective control of power plants using particle swarm optimization techniques. IEEE Trans Energy Conversion 21(2):552–561CrossRefGoogle Scholar
  41. 41.
    Narendra KS, Parthasarathy K (1990) Identification and control of dynamical system using neural networks. IEEE Trans Neural Netw 1(1):4–27CrossRefGoogle Scholar
  42. 42.
    Chatterjee A, Watanabe K (2006) An optimized Takagi-Sugeno type neuro-fuzzy system for modeling robot manipulators. Neural Comput Appl 15(1):55–61CrossRefGoogle Scholar
  43. 43.
    Aliyari M Sh, Teshnehlab M, Sedigh AK (2007) A novel hybrid learning algorithm for tuning ANFIS parameters using adaptive weighted PSO. In: Proceedings of IEEE Int Fuzzy Sys Conf, July 2007Google Scholar
  44. 44.
    Takagi H, Hayashi I (1991) NN- driven fuzzy reasoning. Int J Approx Reason 5(3):191–212MATHCrossRefGoogle Scholar
  45. 45.
    Mackey MC, Glass L (1977) Oscillation and chaos in physical control system. Science 197:287–289CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2008

Authors and Affiliations

  • Mahdi Aliyari Shoorehdeli
    • 1
  • Mohammad Teshnehlab
    • 1
  • Ali Khaki Sedigh
    • 1
  1. 1.K. N. Toosi University of TechnologyTehranIran

Personalised recommendations