Data density-based clustering for regularized fuzzy neural networks based on nullneurons and robust activation function

  • Paulo Vitor de Campos SouzaEmail author
  • Luiz Carlos Bambirra Torres
  • Augusto Junio Guimaraes
  • Vanessa Souza Araujo
  • Vincius Jonathan Silva Araujo
  • Thiago Silva Rezende
Methodologies and Application


This paper proposes the use of fuzzification functions based on clustering of data based on their density to perform the granularization of the input space. The neurons formed in this layer are built through the density centers obtained with the input data of the model. In the second layer, the nullneurons aggregate the generated neurons in the first layer and allow the creation of if/then fuzzy rules. Even in the second layer, a regularization function is activated to determine the essential nullneurons. The concepts of extreme learning machine generate the weights used in the third layer, but with a regularizing factor. Finally, in the third layer, represented by an artificial neural network, it has a single neuron that the activation function uses robust functions to carry out the model. To verify the new training approach for fuzzy neural networks, we performed real and synthetic database tests for the pattern classification, which led to the conclusion that the data density-based approach the use of regularization factors in the second model layer and neurons with more robust activation functions allowed better results compared to other classifiers that use the concepts of extreme learning machine.


DDC Fuzzy neural networks Pattern classification RAF 



The thanks of this work are destined to CEFET-MG, University Center UNA.

Compliance with ethical standards

Conflict of interest

All authors declare no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors


  1. Aliev RA, Fazlollahi B, Aliev RR, Guirimov B (2008) Linguistic time series forecasting using fuzzy recurrent neural network. Soft Comput 12(2):183–190CrossRefGoogle Scholar
  2. Angelov P (2014) Anomaly detection based on eccentricity analysis. In: 2014 IEEE symposium on evolving and autonomous learning systems (EALS). IEEE, pp 1–8Google Scholar
  3. Bach FR (2008) Bolasso: model consistent lasso estimation through the bootstrap. In: Proceedings of the 25th international conference on machine learning. ACM, pp 33–40Google Scholar
  4. Bache K, Lichman M (2013) Uci machine learning repositoryGoogle Scholar
  5. Ballini R, Gomide F (2002) Learning in recurrent, hybrid neurofuzzy networks. In: Proceedings of the 2002 IEEE international conference on Fuzzy systems, 2002. FUZZ-IEEE’02, vol 1. IEEE, pp 785–790Google Scholar
  6. Bordignon F, Gomide F (2014) Uninorm based evolving neural networks and approximation capabilities. Neurocomputing 127:13–20CrossRefGoogle Scholar
  7. Buckley JJ, Hayashi Y (1994) Fuzzy neural networks: a survey. Fuzzy Sets Syst 66(1):1–13MathSciNetCrossRefGoogle Scholar
  8. Calvo T, De Baets B, Fodor J (2001) The functional equations of frank and alsina for uninorms and nullnorms. Fuzzy Sets Systems 120(3):385–394MathSciNetCrossRefzbMATHGoogle Scholar
  9. de Campos Souza VP (2018) Pruning fuzzy neural networks based on unineuron for problems of classification of patterns. J Intell Fuzzy Syst (Preprint), 1–9Google Scholar
  10. de Campos Souza PV, de Oliveira PFA (2018) Regularized fuzzy neural networks based on nullneurons for problems of classification of patterns, pp 25–30Google Scholar
  11. de Campos Souza PV, Guimaraes AJ (2018) Using fuzzy neural networks for improving the prediction of children with autism through mobile devices. In: 2018 IEEE symposium on computers and communications (ISCC). IEEE, pp 01086–01089Google Scholar
  12. de Campos Souza PV, Torres LCB (2018) Regularized fuzzy neural network based on or neuron for time series forecasting. In: Barreto GA, Coelho R (eds) Fuzzy information processing. Springer, Cham, pp 13–23CrossRefGoogle Scholar
  13. de Campos Souza PV, Guimaraes AJ, Araújo VS, Rezende TS, Araújo VJS (2018a) Fuzzy neural networks based on fuzzy logic neurons regularized by resampling techniques and regularization theory for regression problems. Inteligencia Artif 21(62):114–133CrossRefGoogle Scholar
  14. de Campos Souza PV, Silva GRL, Torres LCB (2018b) Uninorm based regularized fuzzy neural networks. In: 2018 IEEE conference on evolving and adaptive intelligent systems (EAIS), pp 1–8.
  15. Ebadzadeh MM, Salimi-Badr A (2018) Ic-fnn: a novel fuzzy neural network with interpretable, intuitive, and correlated-contours fuzzy rules for function approximation. IEEE Trans Fuzzy Syst 26(3):1288–1302. CrossRefGoogle Scholar
  16. Efron B, Hastie T, Johnstone I, Tibshirani R et al (2004) Least angle regression. Ann Stat 32(2):407–499MathSciNetCrossRefzbMATHGoogle Scholar
  17. Guimarães AJ, Araujo VJS, de Campos Souza PV, Araujo VS, Rezende TS (2018) Using fuzzy neural networks to the prediction of improvement in expert systems for treatment of immunotherapy. In: Ibero-American conference on artificial intelligence. Springer, Cham, pp 229–240Google Scholar
  18. Hansen LP (1982) Large sample properties of generalized method of moments estimators. Econometrica: J Econom Soc 1029–1054Google Scholar
  19. Han M, Zhong K, Qiu T, Han B (2018) Interval type-2 fuzzy neural networks for chaotic time series prediction: a concise overview. IEEE Trans Cybern.
  20. He C, Liu Y, Yao T, Xu F, Hu Y, Zheng J (2018) A fast learning algorithm based on extreme learning machine for regular fuzzy neural network. J Intell Fuzzy Syst (Preprint), 1–7Google Scholar
  21. Hell M, Costa P, Gomide F Hybrid (2008) neurofuzzy computing with nullneurons. In: Neural Networks, 2008. IEEE international joint conference on IJCNN 2008 (IEEE World Congress on Computational Intelligence). IEEE, pp 3653–3659Google Scholar
  22. Hsu J (1996) Multiple comparisons: theory and methods. Chapman and Hall/CRC, Boca RatonCrossRefzbMATHGoogle Scholar
  23. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRefGoogle Scholar
  24. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B (Cybern) 42(2):513–529CrossRefGoogle Scholar
  25. Hyde R, Angelov P (2014) Data density based clustering. In: 14th UK workshop on computational intelligence (UKCI), 2014. IEEE, pp 1–7Google Scholar
  26. Javanmardi S, Shojafar M, Shariatmadari S, Ahrabi SS (2014) Frtrust: a fuzzy reputation based model for trust management in semantic p2p grids. arXiv preprint arXiv:1404.2632
  27. Juang CF, Huang RB, Cheng WY (2010) An interval type-2 fuzzy-neural network with support-vector regression for noisy regression problems. IEEE Trans Fuzzy Syst 18(4):686–699CrossRefGoogle Scholar
  28. Karlik B, Olgac AV (2011) Performance analysis of various activation functions in generalized mlp architectures of neural networks. Int J Artif Intell Expert Syst 1(4):111–122Google Scholar
  29. Lemos A, Caminhas W, Gomide F (2010) New uninorm-based neuron model and fuzzy neural networks. In: Fuzzy Information Processing Society (NAFIPS), 2010 Annual Meeting of the North American. IEEE, pp 1–6Google Scholar
  30. Lemos AP, Caminhas W, Gomide F (2012) A fast learning algorithm for uninorm-based fuzzy neural networks. In: Fuzzy Information Processing Society (NAFIPS), 2012 Annual Meeting of the North American. IEEE, pp 1–6Google Scholar
  31. Liu S, Feng L, Xiao Y, Wang H (2014) Robust activation function and its application: semi-supervised kernel extreme learning method. Neurocomputing 144:318–328CrossRefGoogle Scholar
  32. MartíNez-MartíNez JM, Escandell-Montero P, Soria-Olivas E, MartíN-Guerrero JD, Magdalena-Benedito R, GóMez-Sanchis J (2011) Regularized extreme learning machine for regression problems. Neurocomputing 74(17):3716–3721CrossRefGoogle Scholar
  33. Pedrycz W (1991) Neurocomputations in relational systems. IEEE Trans Pattern Anal Mach Intell 3:289–297CrossRefGoogle Scholar
  34. Pedrycz W (1993) Fuzzy neural networks and neurocomputations. Fuzzy Sets Syst 56(1):1–28MathSciNetCrossRefGoogle Scholar
  35. Pedrycz W (2006) Logic-based fuzzy neurocomputing with unineurons. IEEE Trans Fuzzy Syst 14(6):860–873CrossRefGoogle Scholar
  36. Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley, HobokenCrossRefGoogle Scholar
  37. Ravi V, Zimmermann HJ (2001) A neural network and fuzzy rule base hybrid for pattern classification. Soft Comput 5(2):152–159CrossRefzbMATHGoogle Scholar
  38. Rosa R, Gomide F, Ballini R (2013) Evolving hybrid neural fuzzy network for system modeling and time series forecasting. In: 2013 12th international conference on machine learning and applications (ICMLA), vol 2. IEEE, pp 378–383Google Scholar
  39. Rosa R, Maciel L, Gomide F, Ballini R (2014) Evolving hybrid neural fuzzy network for realized volatility forecasting with jumps. In: 2014 IEEE conference on computational intelligence for financial engineering & economics (CIFEr). IEEE, pp 481–488Google Scholar
  40. Sharifian A, Ghadi MJ, Ghavidel S, Li L, Zhang J (2018) A new method based on type-2 fuzzy neural network for accurate wind power forecasting under uncertain data. Renew Energy 120:220–230CrossRefGoogle Scholar
  41. Shi Y, Jian S (2018) Permeability estimation of rock reservoir based on pca and elman neural networks. In: IOP conference series: earth and environmental science, vol 128. IOP Publishing, p 012001Google Scholar
  42. Silva AM, Caminhas W, Lemos A, Gomide F (2014) A fast learning algorithm for evolving neo-fuzzy neuron. Appl Soft Comput 14:194–209CrossRefGoogle Scholar
  43. Song LK, Wen J, Fei CW, Bai GC (2018) Distributed collaborative probabilistic design of multi-failure structure with fluid-structure interaction using fuzzy neural network of regression. Mech Syst Signal Process 104:72–86CrossRefGoogle Scholar
  44. Souza PVC (2018) Regularized fuzzy neural networks for pattern classification problems. Int J Appl Eng Res 13(5):2985–2991Google Scholar
  45. Souza PVdC, Guimaraes AJ, Araujo VS, Rezende TS, Araujo VJS (2018) Regularized fuzzy neural networks to aid effort forecasting in the construction and software development. arXiv preprint arXiv:1812.01351
  46. Szabo A, de França FO (2015) The proposal of dynamic thresholds in an immune algorithm for fuzzy clustering. In: 2015 IEEE international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 1–8Google Scholar
  47. Tang J, Liu F, Zou Y, Zhang W, Wang Y (2017) An improved fuzzy neural network for traffic speed prediction considering periodic characteristic. IEEE Trans Intell Transp Syst 18(9):2340–2350CrossRefGoogle Scholar
  48. Tikhonov A (2013) Numerical methods for the solution of ill-posed problemsGoogle Scholar
  49. Trevor H, Robert T, JH F (2009) The elements of statistical learning: data mining, inference, and predictionGoogle Scholar
  50. Vinicius JA, de Oliveira Batista L, Campos Souza PV, Vanessa A, Thiago SR, AJG (2018) Using fuzzy neural networks to improve prediction of expert systems for detection of breast cancer. Anais do Encontro Nacional de Inteligncia Artificial e Computacional (ENIAC), pp 799–810.
  51. Zhao T, Li P, Cao J (2018) Self-organising interval type-2 fuzzy neural network with asymmetric membership functions and its application. Soft Comput 1–14Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Paulo Vitor de Campos Souza
    • 1
    • 2
    Email author
  • Luiz Carlos Bambirra Torres
    • 2
  • Augusto Junio Guimaraes
    • 1
  • Vanessa Souza Araujo
    • 1
  • Vincius Jonathan Silva Araujo
    • 1
  • Thiago Silva Rezende
    • 1
  1. 1.Faculty UNA of BetimBetimBrazil
  2. 2.Federal University of Minas GeraisBelo HorizonteBrazil

Personalised recommendations