Self-organized direction aware for regularized fuzzy neural networks

  • Paulo Vitor de Campos SouzaEmail author
  • Cristiano Fraga Guimaraes Nunes
  • Augusto Junio Guimares
  • Thiago Silva Rezende
  • Vanessa Souza Araujo
  • Vincius Jonathan Silva Arajuo
Original Paper


The fuzzy neural networks are efficient hybrid structures to perform tasks of regression, patterns classification and time series prediction. To define its architecture, some models use techniques that fuzzification of data that can divide the sample space in grid format through membership functions. The models that use such techniques achieve results with a high degree of accuracy in their activities, but their structures can vary greatly when the number of features of the problem is high, making of fuzzy neurons an exponential relationship between the number of inputs and the membership functions numbers used in the model of the input space. A multi-neuron structure can make the training and update of parameters damaging to the model’s computational performance, making it impossible to work with problems of high dimensions or even with a high number of samples. To solve the problem of the creation of structures of hybrid models based on neural networks and fuzzy systems this paper proposes the use of a novel fully data-driven algorithm. This algorithm uses an extra cosine similarity-based directional component to work together with a traditional distance metric and nonparametric Empirical Data Analytics to data partitioning and forming data clouds in the first layer of the model. Another problem that exists in fuzzy neural network models is that some of their parameters are defined at random, so they challenging to interpret and can introduce casual situations that may impair model responses. In this paper we also propose the definition of bias and weights of the neurons of the first layer using the concepts of the wavelet transform, allowing the parameters of the neurons also to be directly related to the input data submitted to the model. In the second layer, the unineurons aggregate the neurons generated in the first layer and a regularization function is activated to determine the most significant unineurons. The weights used in the third layer, represented by an artificial neural network with an activation function of type ReLU, are generated using the concepts of the extreme learning machine. To verify the new training approach for fuzzy neural networks, tests with real and synthetic databases were performed for pattern classification, which led to the conclusion that the cloud-based approach and neuron weights generation based on the data frequency of training proves that the accuracy of the model is adequate to perform binary classification problems.


Fuzzy neural networks SODA Wavelets ReLU Pattern classification 



The thanks of this work are destined to CEFET-MG, University Center UNA and University Center UNIBH.


  1. Aliev RA, Guirimov B, Fazlollahi B, Aliev RR (2009) Evolutionary algorithm-based learning of fuzzy neural networks. Part 2: recurrent fuzzy neural networks. Fuzzy Sets Syst 160(17):2553–2566MathSciNetzbMATHGoogle Scholar
  2. Amari S-I (1993) Backpropagation and stochastic gradient descent method. Neurocomputing 5(4–5):185–196zbMATHGoogle Scholar
  3. Angelov PP, Filev DP (2004) An approach to online identification of takagi-sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B (Cybern) 34(1):484–498Google Scholar
  4. Angelov PP, Gu X, Príncipe JC (2017) A generalized methodology for data analysis. IEEE Trans CybernGoogle Scholar
  5. Angelov PP, Zhou X (2008) Evolving fuzzy-rule-based classifiers from data streams. IEEE Trans Fuzzy Syst 16(6):1462–1475Google Scholar
  6. Angelov P, Filev D, Kasabov N (2008) Guest editorial: evolving fuzzy systems: preface to the special section. IEEE Trans Fuzzy Syst 16(6):1390–1392Google Scholar
  7. Angelov P, Lughofer E, Zhou X (2008) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159(23):3160–3182MathSciNetzbMATHGoogle Scholar
  8. Angelov P, Gu X, Kangin D (2017) Empirical data analytics. Int J Intell Syst 32(12):1261–1284Google Scholar
  9. Angelov P, Filev DP, Kasabov N (2010) Evolving intelligent systems: methodology and applications, vol 12. Wiley, AmsterdamGoogle Scholar
  10. Angelov P, Kasabov N (2005) Evolving computational intelligence systems. In: Proceedings of the 1st international workshop on genetic fuzzy systems, pp 76–82Google Scholar
  11. Angelov P, Kasabov N (2006) Evolving intelligent systems, eis. IEEE SMC eNewsLetter 15:1–13Google Scholar
  12. Angelov P, Victor J, Dourado A, Filev D (2004) On-line evolution of takagi-sugeno fuzzy models. In: IFAC Proceedings Vol 37(16):67–72Google Scholar
  13. Bach FR (2008) Bolasso: model consistent lasso estimation through the bootstrap. In: Proceedings of the 25th international conference on machine learning, ACM, pp 33–40Google Scholar
  14. Bache K, Lichman M (2013) Uci machine learning repositoryGoogle Scholar
  15. Ballini R, Gomide F (2002) Learning in recurrent, hybrid neurofuzzy networks. In: Fuzzy Systems, 2002. FUZZ-IEEE’02. Proceedings of the 2002 IEEE international conference on, vol 1, IEEE, pp 785–790Google Scholar
  16. Baruah RD, Angelov P (2012) Evolving local means method for clustering of streaming data. In: 2012 IEEE international conference on fuzzy systems, IEEE, pp 1–8Google Scholar
  17. Bezdek JC, Ehrlich R, Full W (1984) FCM: the fuzzy c-means clustering algorithm. Comput Geosci 10(2–3):191–203Google Scholar
  18. Bordignon F, Gomide F (2014) Uninorm based evolving neural networks and approximation capabilities. Neurocomputing 127:13–20Google Scholar
  19. Buckley JJ, Hayashi Y (1994) Fuzzy neural networks: a survey. Fuzzy Sets Syst 66(1):1–13MathSciNetGoogle Scholar
  20. Clevert D.-A, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (elus). arXiv:1511.07289
  21. Daubechies I (1990) The wavelet transform, time-frequency localization and signal analysis. IEEE Trans Inf Theory 36(5):961–1005MathSciNetzbMATHGoogle Scholar
  22. de Campos Souza PV, de Oliveira PFA (2018) Regularized fuzzy neural networks based on nullneurons for problems of classification of patterns. In: 2018 IEEE symposium on computer applications industrial electronics (ISCAIE), pp 25–30Google Scholar
  23. de Campos Souza PV, Guimaraes AJ, Araújo VS, Rezende TS, Araújo VJS (2018) Fuzzy neural networks based on fuzzy logic neurons regularized by resampling techniques and regularization theory for regression problems. Intell Artif 21(62):114–133Google Scholar
  24. de Campos Souza PV, Silva GRL, Torres LCB (2018) Uninorm based regularized fuzzy neural networks. In: 2018 IEEE conference on evolving and adaptive intelligent systems (EAIS), pp 1–8Google Scholar
  25. de Campos Souza PV, Torres LCB (2018) Regularized fuzzy neural network based on or neuron for time series forecasting. In: Barreto GA, Coelho R, (eds), Fuzzy information processing, Springer, Cham, pp 13–23Google Scholar
  26. de Campos Souza PV, Torres LCB, Guimaraes AJ, Araujo VS, Araujo VJS, Rezende TS (2019) Data density-based clustering for regularized fuzzy neural networks based on nullneurons and robust activation function. Soft Comput.
  27. de Jesús Rubio J (2017) Usnfis: uniform stable neuro fuzzy inference system. Neurocomputing 262:57–66Google Scholar
  28. de Jesús Rubio J (2018) Error convergence analysis of the sufin and csufin. Appl Soft ComputGoogle Scholar
  29. de Jesus Rubio J, Garcia E, Aquino G, Aguilar-Ibañez C, Pacheco J, Meda-Campaña JA (2018) Mínimos cuadrados recursivos para un manipulador que aprende por demostración. Revista Iberoamericana de Automática e Informática industrialGoogle Scholar
  30. de Jesús Rubio J, Garcia E, Aquino G, Aguilar-Ibañez C, Pacheco J, Zacarias A (2018) Learning of operator hand movements via least angle regression to be teached in a manipulator. Evol Syst pp 1–16Google Scholar
  31. Ebadzadeh MM, Salimi-Badr A (2018) Ic-fnn: A novel fuzzy neural network with interpretable, intuitive, and correlated-contours fuzzy rules for function approximation. IEEE Trans Fuzzy Syst 26(3):1288–1302Google Scholar
  32. Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32(2):407–499MathSciNetzbMATHGoogle Scholar
  33. Fei J, Lu C (2018) Adaptive sliding mode control of dynamic systems using double loop recurrent neural network structure. IEEE Trans Neural Netw Learn Syst 29(4):1275–1286Google Scholar
  34. Goldberg DE, Holland JH (1988) Genetic algorithms and machine learning. Mach Learn 3(2):95–99Google Scholar
  35. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press
  36. Gu X, Angelov P, Kangin D, Principe J (2018) Self-organised direction aware data partitioning algorithm. Inf Sci 423:80–95MathSciNetGoogle Scholar
  37. Guimaraes AJ, Araújo VJ, de Oliveira Batista L, Souza PVC, Araújo V, Rezende TS (2018) Using fuzzy neural networks to improve prediction of expert systems for detection of breast cancer. In: XV Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), no 1, pp 799–810Google Scholar
  38. Guimarães AJ, Araujo VJS, de Campos Souza PV, Araujo VS, Rezende TS (2018) Using fuzzy neural networks to the prediction of improvement in expert systems for treatment of immunotherapy. In: Ibero-American conference on artificial intelligence, Springer, pp 229–240Google Scholar
  39. Hansen LP (1982) Large sample properties of generalized method of moments estimators. Econometrica pp 1029–1054Google Scholar
  40. Han M, Zhong K, Qiu T, Han B (2018) Interval type-2 fuzzy neural networks for chaotic time series prediction: a concise overview. IEEE Trans Cybern pp 1–12Google Scholar
  41. Hell M, Costa P, Gomide F (2008) Hybrid neurofuzzy computing with nullneurons. In: Neural networks, IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, IEEE 2008:3653–3659Google Scholar
  42. Hell M, Costa P, Gomide F (2014) Participatory learning in the neurofuzzy short-term load forecasting. In: Computational intelligence for engineering solutions (CIES), (2014) IEEE symposium on, IEEE, pp 176–182Google Scholar
  43. Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501Google Scholar
  44. Jang J-S (1993) Anfis: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23(3):665–685Google Scholar
  45. Juang C-F, Huang R-B, Cheng W-Y (2010) An interval type-2 fuzzy-neural network with support-vector regression for noisy regression problems. IEEE Trans Fuzzy Syst 18(4):686–699Google Scholar
  46. Karlik B, Olgac AV (2011) Performance analysis of various activation functions in generalized mlp architectures of neural networks. Int J Artif Intell Expert Syst 1(4):111–122Google Scholar
  47. Kasabov N (2001) Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning. IEEE Trans Syst Man Cybern Part B (Cybern) 31(6):902–918Google Scholar
  48. Kasabov NK, Song Q (1999) Dynamic evolving fuzzy neural networks with“ m-out-of-n” activation nodes for on-line adaptive systems. Department of Information Science, University of OtagoGoogle Scholar
  49. Kasabov NK (2001) On-line learning, reasoning, rule extraction and aggregation in locally optimized evolving fuzzy neural networks. Neurocomputing 41(1–4):25–45zbMATHGoogle Scholar
  50. Kasabov NK, Song Q (2002) Denfis: dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154Google Scholar
  51. Kasabov N, Filev D (2006) Evolving intelligent systems: methods, learning, and applications. In: Evolving fuzzy systems, (2006) international symposium on, IEEE, pp 8–18Google Scholar
  52. Koutrika G, Zadeh ZM, Garcia-Molina H (2009) Data clouds: summarizing keyword search results over structured data. In: Proceedings of the 12th international conference on extending database technology: advances in database technology. ACM, pp 391–402Google Scholar
  53. Lemos AP, Caminhas W, Gomide F (2012) A fast learning algorithm for uninorm-based fuzzy neural networks. In: Fuzzy information processing society (NAFIPS) (2012) Annual Meeting of the North American, IEEE pp 1–6Google Scholar
  54. Lemos A, Caminhas W, Gomide F (2010) New uninorm-based neuron model and fuzzy neural networks. In: Fuzzy information processing society (NAFIPS) (2010) annual meeting of the North American, IEEE 1–6Google Scholar
  55. Liao G-C, Tsao T-P (2004) Application of fuzzy neural networks and artificial intelligence for load forecasting. Electr Power Syst Res 70(3):237–244Google Scholar
  56. Lin Y.-C, Wu Z.-Y, Lee S.-J, Ouyang C.-S (2018) Neuro-fuzzy network for pm2. 5 prediction. In: International conference on smart vehicular technology, transportation, communication and applications, Springer, New York, pp 269–276Google Scholar
  57. Lughofer E (2011) Evolving fuzzy systems-methodologies, advanced concepts and applications, vol 53. Springer, New YorkzbMATHGoogle Scholar
  58. Lughofer E (2012) Single-pass active learning with conflict and ignorance. Evol Syst 3(4):251–271Google Scholar
  59. Lughofer E, Pratama M, Skrjanc I (2018) Incremental rule splitting in generalized evolving fuzzy systems for autonomous drift compensation. IEEE Trans Fuzzy Syst 26(4):1854–1865Google Scholar
  60. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol 30(1), p 3Google Scholar
  61. Maciel L, Lemos A, Gomide F, Ballini R (2012) Evolving fuzzy systems for pricing fixed income options. Evol Syst 3(1):5–18Google Scholar
  62. Okabe A, Boots B, Sugihara K, Chiu S N (2009) Spatial tessellations: concepts and applications of Voronoi diagrams, vol 501. Wiley, AmsterdamzbMATHGoogle Scholar
  63. Pedrycz W (1991) Neurocomputations in relational systems. IEEE Trans Pattern Anal Mach Intell 3:289–297Google Scholar
  64. Pedrycz W (2006) Logic-based fuzzy neurocomputing with unineurons. IEEE Trans Fuzzy Syst 14(6):860–873Google Scholar
  65. Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley, AmsterdamGoogle Scholar
  66. Perova I, Bodyanskiy Y (2017) Fast medical diagnostics using autoassociative neuro-fuzzy memory. Int J Comput 16(1):34–40Google Scholar
  67. Pratama M, Zhang G, Er MJ, Anavatti S (2017) An incremental type-2 meta-cognitive extreme learning machine. IEEE Trans Cybern 47(2):339–353Google Scholar
  68. Rong H-J, Huang G-B, Sundararajan N, Saratchandran P (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Trans Syst Man Cybern Part B (Cybern) 39(4):1067–1072Google Scholar
  69. Rong H-J, Sundararajan N, Huang G-B, Saratchandran P (2006) Sequential adaptive fuzzy inference system (safis) for nonlinear system identification and prediction. Fuzzy Sets Syst 157(9):1260–1275MathSciNetzbMATHGoogle Scholar
  70. Rong H-J, Sundararajan N, Huang G-B, Zhao G-S (2011) Extended sequential adaptive fuzzy inference system for classification problems. Evol Syst 2(2):71–82Google Scholar
  71. Rosa R, Gomide F, Ballini R (2013) Evolving hybrid neural fuzzy network for system modeling and time series forecasting. In: Machine learning and applications (ICMLA), 2013 12th international conference on, vol 2, IEEE, pp 78–383Google Scholar
  72. Rosa R, Maciel L, Gomide F, Ballini R (2014) Evolving hybrid neural fuzzy network for realized volatility forecasting with jumps. In: Computational intelligence for financial engineering and economics (CIFEr), 2104 IEEE Conference on, IEEE, pp 481–488Google Scholar
  73. Sharifian A, Ghadi MJ, Ghavidel S, Li L, Zhang J (2018) A new method based on type-2 fuzzy neural network for accurate wind power forecasting under uncertain data. Renew Energy 120:220–230Google Scholar
  74. Silva AM, Caminhas W, Lemos A, Gomide F (2014) A fast learning algorithm for evolving neo-fuzzy neuron. Appl Soft Comput 14:194–209Google Scholar
  75. Song L-K, Wen J, Fei C-W, Bai G-C (2018) Distributed collaborative probabilistic design of multi-failure structure with fluid–structure interaction using fuzzy neural network of regression. Mech Syst Signal Process 104:72–86Google Scholar
  76. Souza PVDC, Guimaraes AJ, Araujo VS, Rezende TS, Araujo VJS (2018) Regularized fuzzy neural networks to aid effort forecasting in the construction and software development. Int J Artif Intell Appl 9(6):13–26Google Scholar
  77. Souza PVC (2018) Regularized fuzzy neural networks for pattern classification problems. Int J Appl Eng Res 13(5):2985–2991Google Scholar
  78. Subramanian K, Suresh S (2012) A meta-cognitive sequential learning algorithm for neuro-fuzzy inference system. Appl Soft Comput 12(11):3603–3614Google Scholar
  79. Tang J, Liu F, Zou Y, Zhang W, Wang Y (2017) An improved fuzzy neural network for traffic speed prediction considering periodic characteristic. IEEE Trans Intell Transp Syst 18(9):2340–2350Google Scholar
  80. Tikhonov AN, Goncharsky A, Stepanov V, Yagola AG (2013) Numerical methods for the solution of ill-posed problems. Springer, New York, vol 328Google Scholar
  81. Wang W-Y, Li Y-H (2003) Evolutionary learning of bmf fuzzy-neural networks using a reduced-form genetic algorithm. IEEE Trans Syst Man Cybern Part B (Cybern) 33(6):966–976Google Scholar
  82. Yager RR, Rybalov A (1996) Uninorm aggregation operators. Fuzzy Sets Syst 80(1):111–120MathSciNetzbMATHGoogle Scholar
  83. Yen VT, Nan WY, Van Cuong P (2018) Recurrent fuzzy wavelet neural networks based on robust adaptive sliding mode control for industrial robot manipulators. Neural Comput Appl pp 1–14Google Scholar
  84. Yu X, Fu Y, Li P, Zhang Y (2018) Fault-tolerant aircraft control based on self-constructing fuzzy neural networks and multivariable smc under actuator faults. IEEE Trans Fuzzy Syst 26(4):2324–2335Google Scholar
  85. Yu L, Zhang Y-Q (2005) Evolutionary fuzzy neural networks for hybrid financial prediction. IEEE Trans Syst Man Cybern Part C (Appl Rev) 35(2):244–249Google Scholar
  86. Zhang Q-Z, Gan W-S, Zhou Y-L (2006) Adaptive recurrent fuzzy neural networks for active noise control. J Sound Vib 296(4–5):935–948Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Paulo Vitor de Campos Souza
    • 1
    Email author
  • Cristiano Fraga Guimaraes Nunes
    • 2
  • Augusto Junio Guimares
    • 3
  • Thiago Silva Rezende
    • 3
  • Vanessa Souza Araujo
    • 3
  • Vincius Jonathan Silva Arajuo
    • 3
  1. 1.Faculty Una of Betim CEFET-MGBelo HorizonteBrazil
  2. 2.Postgraduate Program in Mathematical and Computational Modeling CEFET-MGBelo HorizonteBrazil
  3. 3.Faculty Una of BetimBetimBrazil

Personalised recommendations