Data density-based clustering for regularized fuzzy neural networks based on nullneurons and robust activation function

Abstract

This paper proposes the use of fuzzification functions based on clustering of data based on their density to perform the granularization of the input space. The neurons formed in this layer are built through the density centers obtained with the input data of the model. In the second layer, the nullneurons aggregate the generated neurons in the first layer and allow the creation of if/then fuzzy rules. Even in the second layer, a regularization function is activated to determine the essential nullneurons. The concepts of extreme learning machine generate the weights used in the third layer, but with a regularizing factor. Finally, in the third layer, represented by an artificial neural network, it has a single neuron that the activation function uses robust functions to carry out the model. To verify the new training approach for fuzzy neural networks, we performed real and synthetic database tests for the pattern classification, which led to the conclusion that the data density-based approach the use of regularization factors in the second model layer and neurons with more robust activation functions allowed better results compared to other classifiers that use the concepts of extreme learning machine.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Notes

  1. 1.

    For the interval of bt = [8, 16, 32, 64] and \(\lambda \) = [0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1].

References

  1. Aliev RA, Fazlollahi B, Aliev RR, Guirimov B (2008) Linguistic time series forecasting using fuzzy recurrent neural network. Soft Comput 12(2):183–190

    Article  Google Scholar 

  2. Angelov P (2014) Anomaly detection based on eccentricity analysis. In: 2014 IEEE symposium on evolving and autonomous learning systems (EALS). IEEE, pp 1–8

  3. Bach FR (2008) Bolasso: model consistent lasso estimation through the bootstrap. In: Proceedings of the 25th international conference on machine learning. ACM, pp 33–40

  4. Bache K, Lichman M (2013) Uci machine learning repository

  5. Ballini R, Gomide F (2002) Learning in recurrent, hybrid neurofuzzy networks. In: Proceedings of the 2002 IEEE international conference on Fuzzy systems, 2002. FUZZ-IEEE’02, vol 1. IEEE, pp 785–790

  6. Bordignon F, Gomide F (2014) Uninorm based evolving neural networks and approximation capabilities. Neurocomputing 127:13–20

    Article  Google Scholar 

  7. Buckley JJ, Hayashi Y (1994) Fuzzy neural networks: a survey. Fuzzy Sets Syst 66(1):1–13

    MathSciNet  Article  Google Scholar 

  8. Calvo T, De Baets B, Fodor J (2001) The functional equations of frank and alsina for uninorms and nullnorms. Fuzzy Sets Systems 120(3):385–394

    MathSciNet  MATH  Article  Google Scholar 

  9. de Campos Souza VP (2018) Pruning fuzzy neural networks based on unineuron for problems of classification of patterns. J Intell Fuzzy Syst (Preprint), 1–9

  10. de Campos Souza PV, de Oliveira PFA (2018) Regularized fuzzy neural networks based on nullneurons for problems of classification of patterns, pp 25–30

  11. de Campos Souza PV, Guimaraes AJ (2018) Using fuzzy neural networks for improving the prediction of children with autism through mobile devices. In: 2018 IEEE symposium on computers and communications (ISCC). IEEE, pp 01086–01089

  12. de Campos Souza PV, Torres LCB (2018) Regularized fuzzy neural network based on or neuron for time series forecasting. In: Barreto GA, Coelho R (eds) Fuzzy information processing. Springer, Cham, pp 13–23

    Google Scholar 

  13. de Campos Souza PV, Guimaraes AJ, Araújo VS, Rezende TS, Araújo VJS (2018a) Fuzzy neural networks based on fuzzy logic neurons regularized by resampling techniques and regularization theory for regression problems. Inteligencia Artif 21(62):114–133

    Article  Google Scholar 

  14. de Campos Souza PV, Silva GRL, Torres LCB (2018b) Uninorm based regularized fuzzy neural networks. In: 2018 IEEE conference on evolving and adaptive intelligent systems (EAIS), pp 1–8. https://doi.org/10.1109/EAIS.2018.8397176

  15. Ebadzadeh MM, Salimi-Badr A (2018) Ic-fnn: a novel fuzzy neural network with interpretable, intuitive, and correlated-contours fuzzy rules for function approximation. IEEE Trans Fuzzy Syst 26(3):1288–1302. https://doi.org/10.1109/TFUZZ.2017.2718497

    Article  Google Scholar 

  16. Efron B, Hastie T, Johnstone I, Tibshirani R et al (2004) Least angle regression. Ann Stat 32(2):407–499

    MathSciNet  MATH  Article  Google Scholar 

  17. Guimarães AJ, Araujo VJS, de Campos Souza PV, Araujo VS, Rezende TS (2018) Using fuzzy neural networks to the prediction of improvement in expert systems for treatment of immunotherapy. In: Ibero-American conference on artificial intelligence. Springer, Cham, pp 229–240

  18. Hansen LP (1982) Large sample properties of generalized method of moments estimators. Econometrica: J Econom Soc 1029–1054

  19. Han M, Zhong K, Qiu T, Han B (2018) Interval type-2 fuzzy neural networks for chaotic time series prediction: a concise overview. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2018.2834356

  20. He C, Liu Y, Yao T, Xu F, Hu Y, Zheng J (2018) A fast learning algorithm based on extreme learning machine for regular fuzzy neural network. J Intell Fuzzy Syst (Preprint), 1–7

  21. Hell M, Costa P, Gomide F Hybrid (2008) neurofuzzy computing with nullneurons. In: Neural Networks, 2008. IEEE international joint conference on IJCNN 2008 (IEEE World Congress on Computational Intelligence). IEEE, pp 3653–3659

  22. Hsu J (1996) Multiple comparisons: theory and methods. Chapman and Hall/CRC, Boca Raton

    Google Scholar 

  23. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  24. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B (Cybern) 42(2):513–529

    Article  Google Scholar 

  25. Hyde R, Angelov P (2014) Data density based clustering. In: 14th UK workshop on computational intelligence (UKCI), 2014. IEEE, pp 1–7

  26. Javanmardi S, Shojafar M, Shariatmadari S, Ahrabi SS (2014) Frtrust: a fuzzy reputation based model for trust management in semantic p2p grids. arXiv preprint arXiv:1404.2632

  27. Juang CF, Huang RB, Cheng WY (2010) An interval type-2 fuzzy-neural network with support-vector regression for noisy regression problems. IEEE Trans Fuzzy Syst 18(4):686–699

    Article  Google Scholar 

  28. Karlik B, Olgac AV (2011) Performance analysis of various activation functions in generalized mlp architectures of neural networks. Int J Artif Intell Expert Syst 1(4):111–122

    Google Scholar 

  29. Lemos A, Caminhas W, Gomide F (2010) New uninorm-based neuron model and fuzzy neural networks. In: Fuzzy Information Processing Society (NAFIPS), 2010 Annual Meeting of the North American. IEEE, pp 1–6

  30. Lemos AP, Caminhas W, Gomide F (2012) A fast learning algorithm for uninorm-based fuzzy neural networks. In: Fuzzy Information Processing Society (NAFIPS), 2012 Annual Meeting of the North American. IEEE, pp 1–6

  31. Liu S, Feng L, Xiao Y, Wang H (2014) Robust activation function and its application: semi-supervised kernel extreme learning method. Neurocomputing 144:318–328

    Article  Google Scholar 

  32. MartíNez-MartíNez JM, Escandell-Montero P, Soria-Olivas E, MartíN-Guerrero JD, Magdalena-Benedito R, GóMez-Sanchis J (2011) Regularized extreme learning machine for regression problems. Neurocomputing 74(17):3716–3721

    Article  Google Scholar 

  33. Pedrycz W (1991) Neurocomputations in relational systems. IEEE Trans Pattern Anal Mach Intell 3:289–297

    Article  Google Scholar 

  34. Pedrycz W (1993) Fuzzy neural networks and neurocomputations. Fuzzy Sets Syst 56(1):1–28

    Article  Google Scholar 

  35. Pedrycz W (2006) Logic-based fuzzy neurocomputing with unineurons. IEEE Trans Fuzzy Syst 14(6):860–873

    Article  Google Scholar 

  36. Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley, Hoboken

    Google Scholar 

  37. Ravi V, Zimmermann HJ (2001) A neural network and fuzzy rule base hybrid for pattern classification. Soft Comput 5(2):152–159

    MATH  Article  Google Scholar 

  38. Rosa R, Gomide F, Ballini R (2013) Evolving hybrid neural fuzzy network for system modeling and time series forecasting. In: 2013 12th international conference on machine learning and applications (ICMLA), vol 2. IEEE, pp 378–383

  39. Rosa R, Maciel L, Gomide F, Ballini R (2014) Evolving hybrid neural fuzzy network for realized volatility forecasting with jumps. In: 2014 IEEE conference on computational intelligence for financial engineering & economics (CIFEr). IEEE, pp 481–488

  40. Sharifian A, Ghadi MJ, Ghavidel S, Li L, Zhang J (2018) A new method based on type-2 fuzzy neural network for accurate wind power forecasting under uncertain data. Renew Energy 120:220–230

    Article  Google Scholar 

  41. Shi Y, Jian S (2018) Permeability estimation of rock reservoir based on pca and elman neural networks. In: IOP conference series: earth and environmental science, vol 128. IOP Publishing, p 012001

  42. Silva AM, Caminhas W, Lemos A, Gomide F (2014) A fast learning algorithm for evolving neo-fuzzy neuron. Appl Soft Comput 14:194–209

    Article  Google Scholar 

  43. Song LK, Wen J, Fei CW, Bai GC (2018) Distributed collaborative probabilistic design of multi-failure structure with fluid-structure interaction using fuzzy neural network of regression. Mech Syst Signal Process 104:72–86

    Article  Google Scholar 

  44. Souza PVC (2018) Regularized fuzzy neural networks for pattern classification problems. Int J Appl Eng Res 13(5):2985–2991

    Google Scholar 

  45. Souza PVdC, Guimaraes AJ, Araujo VS, Rezende TS, Araujo VJS (2018) Regularized fuzzy neural networks to aid effort forecasting in the construction and software development. arXiv preprint arXiv:1812.01351

  46. Szabo A, de França FO (2015) The proposal of dynamic thresholds in an immune algorithm for fuzzy clustering. In: 2015 IEEE international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 1–8

  47. Tang J, Liu F, Zou Y, Zhang W, Wang Y (2017) An improved fuzzy neural network for traffic speed prediction considering periodic characteristic. IEEE Trans Intell Transp Syst 18(9):2340–2350

    Article  Google Scholar 

  48. Tikhonov A (2013) Numerical methods for the solution of ill-posed problems

  49. Trevor H, Robert T, JH F (2009) The elements of statistical learning: data mining, inference, and prediction

  50. Vinicius JA, de Oliveira Batista L, Campos Souza PV, Vanessa A, Thiago SR, AJG (2018) Using fuzzy neural networks to improve prediction of expert systems for detection of breast cancer. Anais do Encontro Nacional de Inteligncia Artificial e Computacional (ENIAC), pp 799–810. https://doi.org/10.5753/eniac.2018.4468. http://portaldeconteudo.sbc.org.br/index.php/eniac/article/view/4468

  51. Zhao T, Li P, Cao J (2018) Self-organising interval type-2 fuzzy neural network with asymmetric membership functions and its application. Soft Comput 1–14

Download references

Acknowledgements

The thanks of this work are destined to CEFET-MG, University Center UNA.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Paulo Vitor de Campos Souza.

Ethics declarations

Conflict of interest

All authors declare no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by V. Loia.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

de Campos Souza, P.V., Torres, L.C.B., Guimaraes, A.J. et al. Data density-based clustering for regularized fuzzy neural networks based on nullneurons and robust activation function. Soft Comput 23, 12475–12489 (2019). https://doi.org/10.1007/s00500-019-03792-z

Download citation

Keywords

  • DDC
  • Fuzzy neural networks
  • Pattern classification
  • RAF