Skip to main content
SpringerLink
Log in

Self-organized direction aware for regularized fuzzy neural networks

  • Original Paper
  • Published:
Evolving Systems Aims and scope Submit manuscript

Abstract

The fuzzy neural networks are efficient hybrid structures to perform tasks of regression, patterns classification and time series prediction. To define its architecture, some models use techniques that fuzzification of data that can divide the sample space in grid format through membership functions. The models that use such techniques achieve results with a high degree of accuracy in their activities, but their structures can vary greatly when the number of features of the problem is high, making of fuzzy neurons an exponential relationship between the number of inputs and the membership functions numbers used in the model of the input space. A multi-neuron structure can make the training and update of parameters damaging to the model’s computational performance, making it impossible to work with problems of high dimensions or even with a high number of samples. To solve the problem of the creation of structures of hybrid models based on neural networks and fuzzy systems this paper proposes the use of a novel fully data-driven algorithm. This algorithm uses an extra cosine similarity-based directional component to work together with a traditional distance metric and nonparametric Empirical Data Analytics to data partitioning and forming data clouds in the first layer of the model. Another problem that exists in fuzzy neural network models is that some of their parameters are defined at random, so they challenging to interpret and can introduce casual situations that may impair model responses. In this paper we also propose the definition of bias and weights of the neurons of the first layer using the concepts of the wavelet transform, allowing the parameters of the neurons also to be directly related to the input data submitted to the model. In the second layer, the unineurons aggregate the neurons generated in the first layer and a regularization function is activated to determine the most significant unineurons. The weights used in the third layer, represented by an artificial neural network with an activation function of type ReLU, are generated using the concepts of the extreme learning machine. To verify the new training approach for fuzzy neural networks, tests with real and synthetic databases were performed for pattern classification, which led to the conclusion that the cloud-based approach and neuron weights generation based on the data frequency of training proves that the accuracy of the model is adequate to perform binary classification problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Aliev RA, Guirimov B, Fazlollahi B, Aliev RR (2009) Evolutionary algorithm-based learning of fuzzy neural networks. Part 2: recurrent fuzzy neural networks. Fuzzy Sets Syst 160(17):2553–2566

    MathSciNet  MATH  Google Scholar 

  • Amari S-I (1993) Backpropagation and stochastic gradient descent method. Neurocomputing 5(4–5):185–196

    MATH  Google Scholar 

  • Angelov PP, Filev DP (2004) An approach to online identification of takagi-sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B (Cybern) 34(1):484–498

  • Angelov PP, Gu X, Príncipe JC (2017) A generalized methodology for data analysis. IEEE Trans Cybern

  • Angelov PP, Zhou X (2008) Evolving fuzzy-rule-based classifiers from data streams. IEEE Trans Fuzzy Syst 16(6):1462–1475

    Google Scholar 

  • Angelov P, Filev D, Kasabov N (2008) Guest editorial: evolving fuzzy systems: preface to the special section. IEEE Trans Fuzzy Syst 16(6):1390–1392

    Google Scholar 

  • Angelov P, Lughofer E, Zhou X (2008) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159(23):3160–3182

    MathSciNet  MATH  Google Scholar 

  • Angelov P, Gu X, Kangin D (2017) Empirical data analytics. Int J Intell Syst 32(12):1261–1284

    Google Scholar 

  • Angelov P, Filev DP, Kasabov N (2010) Evolving intelligent systems: methodology and applications, vol 12. Wiley, Amsterdam

  • Angelov P, Kasabov N (2005) Evolving computational intelligence systems. In: Proceedings of the 1st international workshop on genetic fuzzy systems, pp 76–82

  • Angelov P, Kasabov N (2006) Evolving intelligent systems, eis. IEEE SMC eNewsLetter 15:1–13

  • Angelov P, Victor J, Dourado A, Filev D (2004) On-line evolution of takagi-sugeno fuzzy models. In: IFAC Proceedings Vol 37(16):67–72

  • Bach FR (2008) Bolasso: model consistent lasso estimation through the bootstrap. In: Proceedings of the 25th international conference on machine learning, ACM, pp 33–40

  • Bache K, Lichman M (2013) Uci machine learning repository

  • Ballini R, Gomide F (2002) Learning in recurrent, hybrid neurofuzzy networks. In: Fuzzy Systems, 2002. FUZZ-IEEE’02. Proceedings of the 2002 IEEE international conference on, vol 1, IEEE, pp 785–790

  • Baruah RD, Angelov P (2012) Evolving local means method for clustering of streaming data. In: 2012 IEEE international conference on fuzzy systems, IEEE, pp 1–8

  • Bezdek JC, Ehrlich R, Full W (1984) FCM: the fuzzy c-means clustering algorithm. Comput Geosci 10(2–3):191–203

    Google Scholar 

  • Bordignon F, Gomide F (2014) Uninorm based evolving neural networks and approximation capabilities. Neurocomputing 127:13–20

    Google Scholar 

  • Buckley JJ, Hayashi Y (1994) Fuzzy neural networks: a survey. Fuzzy Sets Syst 66(1):1–13

    MathSciNet  Google Scholar 

  • Clevert D.-A, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (elus). arXiv:1511.07289

  • Daubechies I (1990) The wavelet transform, time-frequency localization and signal analysis. IEEE Trans Inf Theory 36(5):961–1005

    MathSciNet  MATH  Google Scholar 

  • de Campos Souza PV, de Oliveira PFA (2018) Regularized fuzzy neural networks based on nullneurons for problems of classification of patterns. In: 2018 IEEE symposium on computer applications industrial electronics (ISCAIE), pp 25–30

  • de Campos Souza PV, Guimaraes AJ, Araújo VS, Rezende TS, Araújo VJS (2018) Fuzzy neural networks based on fuzzy logic neurons regularized by resampling techniques and regularization theory for regression problems. Intell Artif 21(62):114–133

  • de Campos Souza PV, Silva GRL, Torres LCB (2018) Uninorm based regularized fuzzy neural networks. In: 2018 IEEE conference on evolving and adaptive intelligent systems (EAIS), pp 1–8

  • de Campos Souza PV, Torres LCB (2018) Regularized fuzzy neural network based on or neuron for time series forecasting. In: Barreto GA, Coelho R, (eds), Fuzzy information processing, Springer, Cham, pp 13–23

  • de Campos Souza PV, Torres LCB, Guimaraes AJ, Araujo VS, Araujo VJS, Rezende TS (2019) Data density-based clustering for regularized fuzzy neural networks based on nullneurons and robust activation function. Soft Comput. https://doi.org/10.1007/s00500-019-03792-z

  • de Jesús Rubio J (2017) Usnfis: uniform stable neuro fuzzy inference system. Neurocomputing 262:57–66

  • de Jesús Rubio J (2018) Error convergence analysis of the sufin and csufin. Appl Soft Comput

  • de Jesus Rubio J, Garcia E, Aquino G, Aguilar-Ibañez C, Pacheco J, Meda-Campaña JA (2018) Mínimos cuadrados recursivos para un manipulador que aprende por demostración. Revista Iberoamericana de Automática e Informática industrial

  • de Jesús Rubio J, Garcia E, Aquino G, Aguilar-Ibañez C, Pacheco J, Zacarias A (2018) Learning of operator hand movements via least angle regression to be teached in a manipulator. Evol Syst pp 1–16

  • Ebadzadeh MM, Salimi-Badr A (2018) Ic-fnn: A novel fuzzy neural network with interpretable, intuitive, and correlated-contours fuzzy rules for function approximation. IEEE Trans Fuzzy Syst 26(3):1288–1302

    Google Scholar 

  • Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32(2):407–499

    MathSciNet  MATH  Google Scholar 

  • Fei J, Lu C (2018) Adaptive sliding mode control of dynamic systems using double loop recurrent neural network structure. IEEE Trans Neural Netw Learn Syst 29(4):1275–1286

    Google Scholar 

  • Goldberg DE, Holland JH (1988) Genetic algorithms and machine learning. Mach Learn 3(2):95–99

    Google Scholar 

  • Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press http://www.deeplearningbook.org

  • Gu X, Angelov P, Kangin D, Principe J (2018) Self-organised direction aware data partitioning algorithm. Inf Sci 423:80–95

    MathSciNet  MATH  Google Scholar 

  • Guimaraes AJ, Araújo VJ, de Oliveira Batista L, Souza PVC, Araújo V, Rezende TS (2018) Using fuzzy neural networks to improve prediction of expert systems for detection of breast cancer. In: XV Encontro Nacional de Inteligência Artificial e Computacional (ENIAC), no 1, pp 799–810

  • Guimarães AJ, Araujo VJS, de Campos Souza PV, Araujo VS, Rezende TS (2018) Using fuzzy neural networks to the prediction of improvement in expert systems for treatment of immunotherapy. In: Ibero-American conference on artificial intelligence, Springer, pp 229–240

  • Hansen LP (1982) Large sample properties of generalized method of moments estimators. Econometrica pp 1029–1054

  • Han M, Zhong K, Qiu T, Han B (2018) Interval type-2 fuzzy neural networks for chaotic time series prediction: a concise overview. IEEE Trans Cybern pp 1–12

  • Hell M, Costa P, Gomide F (2008) Hybrid neurofuzzy computing with nullneurons. In: Neural networks, IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, IEEE 2008:3653–3659

  • Hell M, Costa P, Gomide F (2014) Participatory learning in the neurofuzzy short-term load forecasting. In: Computational intelligence for engineering solutions (CIES), (2014) IEEE symposium on, IEEE, pp 176–182

  • Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Google Scholar 

  • Jang J-S (1993) Anfis: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23(3):665–685

    Google Scholar 

  • Juang C-F, Huang R-B, Cheng W-Y (2010) An interval type-2 fuzzy-neural network with support-vector regression for noisy regression problems. IEEE Trans Fuzzy Syst 18(4):686–699

    Google Scholar 

  • Karlik B, Olgac AV (2011) Performance analysis of various activation functions in generalized mlp architectures of neural networks. Int J Artif Intell Expert Syst 1(4):111–122

    Google Scholar 

  • Kasabov N (2001) Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning. IEEE Trans Syst Man Cybern Part B (Cybern) 31(6):902–918

  • Kasabov NK, Song Q (1999) Dynamic evolving fuzzy neural networks with“ m-out-of-n” activation nodes for on-line adaptive systems. Department of Information Science, University of Otago

  • Kasabov NK (2001) On-line learning, reasoning, rule extraction and aggregation in locally optimized evolving fuzzy neural networks. Neurocomputing 41(1–4):25–45

    MATH  Google Scholar 

  • Kasabov NK, Song Q (2002) Denfis: dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154

    Google Scholar 

  • Kasabov N, Filev D (2006) Evolving intelligent systems: methods, learning, and applications. In: Evolving fuzzy systems, (2006) international symposium on, IEEE, pp 8–18

  • Koutrika G, Zadeh ZM, Garcia-Molina H (2009) Data clouds: summarizing keyword search results over structured data. In: Proceedings of the 12th international conference on extending database technology: advances in database technology. ACM, pp 391–402

  • Lemos AP, Caminhas W, Gomide F (2012) A fast learning algorithm for uninorm-based fuzzy neural networks. In: Fuzzy information processing society (NAFIPS) (2012) Annual Meeting of the North American, IEEE pp 1–6

  • Lemos A, Caminhas W, Gomide F (2010) New uninorm-based neuron model and fuzzy neural networks. In: Fuzzy information processing society (NAFIPS) (2010) annual meeting of the North American, IEEE 1–6

  • Liao G-C, Tsao T-P (2004) Application of fuzzy neural networks and artificial intelligence for load forecasting. Electr Power Syst Res 70(3):237–244

    Google Scholar 

  • Lin Y.-C, Wu Z.-Y, Lee S.-J, Ouyang C.-S (2018) Neuro-fuzzy network for pm2. 5 prediction. In: International conference on smart vehicular technology, transportation, communication and applications, Springer, New York, pp 269–276

  • Lughofer E (2011) Evolving fuzzy systems-methodologies, advanced concepts and applications, vol 53. Springer, New York

    MATH  Google Scholar 

  • Lughofer E (2012) Single-pass active learning with conflict and ignorance. Evol Syst 3(4):251–271

    Google Scholar 

  • Lughofer E, Pratama M, Skrjanc I (2018) Incremental rule splitting in generalized evolving fuzzy systems for autonomous drift compensation. IEEE Trans Fuzzy Syst 26(4):1854–1865

    Google Scholar 

  • Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol 30(1), p 3

  • Maciel L, Lemos A, Gomide F, Ballini R (2012) Evolving fuzzy systems for pricing fixed income options. Evol Syst 3(1):5–18

    Google Scholar 

  • Okabe A, Boots B, Sugihara K, Chiu S N (2009) Spatial tessellations: concepts and applications of Voronoi diagrams, vol 501. Wiley, Amsterdam

    MATH  Google Scholar 

  • Pedrycz W (1991) Neurocomputations in relational systems. IEEE Trans Pattern Anal Mach Intell 3:289–297

    Google Scholar 

  • Pedrycz W (2006) Logic-based fuzzy neurocomputing with unineurons. IEEE Trans Fuzzy Syst 14(6):860–873

    Google Scholar 

  • Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley, Amsterdam

    Google Scholar 

  • Perova I, Bodyanskiy Y (2017) Fast medical diagnostics using autoassociative neuro-fuzzy memory. Int J Comput 16(1):34–40

    Google Scholar 

  • Pratama M, Zhang G, Er MJ, Anavatti S (2017) An incremental type-2 meta-cognitive extreme learning machine. IEEE Trans Cybern 47(2):339–353

    Google Scholar 

  • Rong H-J, Huang G-B, Sundararajan N, Saratchandran P (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Trans Syst Man Cybern Part B (Cybern) 39(4):1067–1072

  • Rong H-J, Sundararajan N, Huang G-B, Saratchandran P (2006) Sequential adaptive fuzzy inference system (safis) for nonlinear system identification and prediction. Fuzzy Sets Syst 157(9):1260–1275

    MathSciNet  MATH  Google Scholar 

  • Rong H-J, Sundararajan N, Huang G-B, Zhao G-S (2011) Extended sequential adaptive fuzzy inference system for classification problems. Evol Syst 2(2):71–82

    Google Scholar 

  • Rosa R, Gomide F, Ballini R (2013) Evolving hybrid neural fuzzy network for system modeling and time series forecasting. In: Machine learning and applications (ICMLA), 2013 12th international conference on, vol 2, IEEE, pp 78–383

  • Rosa R, Maciel L, Gomide F, Ballini R (2014) Evolving hybrid neural fuzzy network for realized volatility forecasting with jumps. In: Computational intelligence for financial engineering and economics (CIFEr), 2104 IEEE Conference on, IEEE, pp 481–488

  • Sharifian A, Ghadi MJ, Ghavidel S, Li L, Zhang J (2018) A new method based on type-2 fuzzy neural network for accurate wind power forecasting under uncertain data. Renew Energy 120:220–230

    Google Scholar 

  • Silva AM, Caminhas W, Lemos A, Gomide F (2014) A fast learning algorithm for evolving neo-fuzzy neuron. Appl Soft Comput 14:194–209

    Google Scholar 

  • Song L-K, Wen J, Fei C-W, Bai G-C (2018) Distributed collaborative probabilistic design of multi-failure structure with fluid–structure interaction using fuzzy neural network of regression. Mech Syst Signal Process 104:72–86

    Google Scholar 

  • Souza PVDC, Guimaraes AJ, Araujo VS, Rezende TS, Araujo VJS (2018) Regularized fuzzy neural networks to aid effort forecasting in the construction and software development. Int J Artif Intell Appl 9(6):13–26

  • Souza PVC (2018) Regularized fuzzy neural networks for pattern classification problems. Int J Appl Eng Res 13(5):2985–2991

    Google Scholar 

  • Subramanian K, Suresh S (2012) A meta-cognitive sequential learning algorithm for neuro-fuzzy inference system. Appl Soft Comput 12(11):3603–3614

    Google Scholar 

  • Tang J, Liu F, Zou Y, Zhang W, Wang Y (2017) An improved fuzzy neural network for traffic speed prediction considering periodic characteristic. IEEE Trans Intell Transp Syst 18(9):2340–2350

    Google Scholar 

  • Tikhonov AN, Goncharsky A, Stepanov V, Yagola AG (2013) Numerical methods for the solution of ill-posed problems. Springer, New York, vol 328

  • Wang W-Y, Li Y-H (2003) Evolutionary learning of bmf fuzzy-neural networks using a reduced-form genetic algorithm. IEEE Trans Syst Man Cybern Part B (Cybern) 33(6):966–976

  • Yager RR, Rybalov A (1996) Uninorm aggregation operators. Fuzzy Sets Syst 80(1):111–120

    MathSciNet  MATH  Google Scholar 

  • Yen VT, Nan WY, Van Cuong P (2018) Recurrent fuzzy wavelet neural networks based on robust adaptive sliding mode control for industrial robot manipulators. Neural Comput Appl pp 1–14

  • Yu X, Fu Y, Li P, Zhang Y (2018) Fault-tolerant aircraft control based on self-constructing fuzzy neural networks and multivariable smc under actuator faults. IEEE Trans Fuzzy Syst 26(4):2324–2335

    Google Scholar 

  • Yu L, Zhang Y-Q (2005) Evolutionary fuzzy neural networks for hybrid financial prediction. IEEE Trans Syst Man Cybern Part C (Appl Rev) 35(2):244–249

  • Zhang Q-Z, Gan W-S, Zhou Y-L (2006) Adaptive recurrent fuzzy neural networks for active noise control. J Sound Vib 296(4–5):935–948

    Google Scholar 

Download references

Acknowledgements

The thanks of this work are destined to CEFET-MG, University Center UNA and University Center UNIBH.

Author information

Authors and Affiliations

  1. Faculty Una of Betim CEFET-MG, Belo Horizonte, Brazil

    Paulo Vitor de Campos Souza

  2. Postgraduate Program in Mathematical and Computational Modeling CEFET-MG, Belo Horizonte, Brazil

    Cristiano Fraga Guimaraes Nunes

  3. Faculty Una of Betim, Betim, Brazil

    Augusto Junio Guimares, Thiago Silva Rezende, Vanessa Souza Araujo & Vincius Jonathan Silva Arajuo

Authors
  1. Paulo Vitor de Campos Souza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cristiano Fraga Guimaraes Nunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Augusto Junio Guimares
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thiago Silva Rezende
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vanessa Souza Araujo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vincius Jonathan Silva Arajuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paulo Vitor de Campos Souza.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Campos Souza, P.V., Guimaraes Nunes, C.F., Guimares, A.J. et al. Self-organized direction aware for regularized fuzzy neural networks. Evolving Systems 12, 303–317 (2021). https://doi.org/10.1007/s12530-019-09278-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12530-019-09278-5

Keywords

Search

Navigation