Skip to main content

Advertisement

SpringerLink
Log in

Fuzzy neural network with support vector-based learning for classification and regression

  • Foundations
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Fuzzy neural network (FNN) and support vector machine (SVM) are two prominent and powerful learning models broadly used for classification and regression. FNN has a significant local representation and human reasoning advantage. However, the drawback of such a network is that the focal point of the learning algorithms is minimizing empirical risk. In contrary to FNN, SVM emphasizes simultaneously on minimizing empirical and expected risks, which theoretically leads to an excellent generalization performance power. In this paper, we show that a Takagi–Sugeno–Kang (TSK)-type-based fuzzy neural network is, in fact, equivalent to an SVM with an adaptive kernel based on fuzzy rules generated in this FNN. Consequently, it is possible to learn the last layer of the FNN using the concepts of SVM and thus taking the advantage of SVM in generalization. In fact, on the one side, the proposed method is an SVM with an adaptive kernel based on fuzzy rules and, on the other side, it is a TSK–FNN with SVM-based learning. As a matter of fact, the defined kernel in FNN is an adaptive kernel based on data characteristics, which is derived from the fuzzy rules generated by the FNN itself. The results obtained from the proposed method for classification and regression outperform the results of SVM with the conventional kernels and fuzzy neural network trained in the traditional way.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Aeberhard S, Coomans D, de Vel O (1992) The classification performance of RDA. Dept of Computer Science and Dept of Mathematics and Statistics, James Cook University of North Queensland, Tech Rep, pp 92–01

  • Aiolli F, Donini M (2015) Easymkl: a scalable multiple kernel learning algorithm. Neurocomputing 169:215–224

    Google Scholar 

  • Al-Janabi S (2017) Pragmatic miner to risk analysis for intrusion detection (pmra-id). In: International conference on soft computing in data science, Springer, pp 263–277

  • Al-Janabi S (2018) Smart system to create an optimal higher education environment using IDA and IOTs. Int J Comput Appl. https://doi.org/10.1080/1206212X.2018.1512460

    Article  Google Scholar 

  • Al-Janabi S, Abaid Mahdi M (2019) Evaluation prediction techniques to achieve optimal biomedical analysis. Int J Grid Util Comput. https://doi.org/10.1504/IJGUC.2019.10020511

    Google Scholar 

  • Al-Janabi S, Alkaim AF (2019) A nifty collaborative analysis to predicting a novel tool (DRFLLS) for missing values estimation. Soft Comput 1–15

  • Al-Janabi S, Razaq F (2018) Intelligent big data analysis to design smart predictor for customer churn in telecommunication industry. In: International conference on big data and smart digital environment, Springer, pp 246–272

  • Al-Janabi S, Al-Shourbaji I, Salman MA (2018) Assessing the suitability of soft computing approaches for forest fires prediction. Appl Comput Inform 14(2):214–224

    Google Scholar 

  • Al-Janabi S, Rawat S, Patel A, Al-Shourbaji I (2015) Design and evaluation of a hybrid system for detection and prediction of faults in electrical transformers. Int J Electr Power Energy Syst 67:324–335

    Google Scholar 

  • Argyriou A, Micchelli CA, Pontil M (2005) Learning convex combinations of continuously parameterized basic kernels. In: International conference on computational learning theory, Springer, pp 338–352

  • Batuwita R, Palade V (2010) FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(3):558–571

    Google Scholar 

  • Berthold MR, Diamond J (1998) Constructive training of probabilistic neural networks. Neurocomputing 19(1–3):167–183

    Google Scholar 

  • Blake C, Merz C (1998) UCI machine learning repository. http://archive.ics.uci.edu/ml/datasets/. Accessed 19 July 2018

  • Chai Y, Jia L, Zhang Z (2009) Mamdani model based adaptive neural fuzzy inference system and its application. Int J Comput Intell 5(1):22–29

    Google Scholar 

  • Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159

    MATH  Google Scholar 

  • Chen Y, Wang JZ (2003) Support vector learning for fuzzy rule-based classification systems. IEEE Trans Fuzzy Syst 11(6):716–728

    Google Scholar 

  • Chiang JH, Hao PY (2004) Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Trans Fuzzy Syst 12(1):1–12

    Google Scholar 

  • Chuang CC, Su SF, Chen SS (2001) Robust tsk fuzzy modeling for function approximation with outliers. IEEE Trans Fuzzy Syst 9(6):810–821

    Google Scholar 

  • Cortes C, Mohri M, Rostamizadeh A (2010) Two-stage learning kernel algorithms. In: ICML, pp 239–246

  • Cristianini N, Shawe-Taylor J et al (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Cristianini N, Shawe-Taylor J, Elisseeff A, Kandola JS (2002) On kernel-target alignment. Advances in neural information processing systems. MIT press, Cambridge, pp 367–373

    Google Scholar 

  • De Vito S, Massera E, Piga M, Martinotto L, Di Francia G (2008) On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens Actuator B Chem 129(2):750–757

    Google Scholar 

  • Dickerson JA, Kosko B (1996) Fuzzy function approximation with ellipsoidal rules. IEEE Trans Syst Man Cybern Part B Cybern 26(4):542–560

    Google Scholar 

  • Ebadzadeh MM, Salimi-Badr A (2015) Cfnn: correlated fuzzy neural network. Neurocomputing 148:430–444

    Google Scholar 

  • Ekong U, Lam H, Xiao B, Ouyang G, Liu H, Chan KY, Ling SH (2016) Classification of epilepsy seizure phase using interval type-2 fuzzy support vector machines. Neurocomputing 199:66–76

    Google Scholar 

  • Esme E, Karlik B (2016) Fuzzy c-means based support vector machines classifier for perfume recognition. Appl Soft Comput 46:452–458

    Google Scholar 

  • Ester M, Kriegel HP, Sander J, Xu X et al (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. Kdd 96:226–231

    Google Scholar 

  • Fan Q, Wang Z, Li D, Gao D, Zha H (2017) Entropy-based fuzzy support vector machine for imbalanced datasets. Knowl Based Syst 115:87–99

    Google Scholar 

  • Ferreira RP, Martiniano A, Ferreira A, Ferreira A, Sassi RJ (2016) Study on daily demand forecasting orders using artificial neural network. IEEE Latin Am Trans 14(3):1519–1525

    Google Scholar 

  • Fraley C, Raftery AE (1998) How many clusters? Which clustering method? Answers via model-based cluster analysis. Comput J 41(8):578–588

    MATH  Google Scholar 

  • Fraley C, Raftery AE (2002) Model-based clustering, discriminant analysis, and density estimation. J Am Stat Assoc 97(458):611–631

    MathSciNet  MATH  Google Scholar 

  • Gabrys B, Bargiela A (2000) General fuzzy min-max neural network for clustering and classification. IEEE Trans Neural Netw 11(3):769–783

    Google Scholar 

  • Ganji H, Khadivi S, Ebadzadeh MM (2017) Support vector-based fuzzy classifier with adaptive kernel. Neural Comput Appl. https://doi.org/10.1007/s00521-017-3170-x

    Article  Google Scholar 

  • González J, Rojas H, Ortega J, Prieto A (2002) A new clustering technique for function approximation. IEEE Trans Neural Netw 13(1):132–142

    Google Scholar 

  • Guevara J (2016) Supervised machine learning with kernel embeddings of fuzzy sets and probability measures. PhD Thesis, IME USP

  • Guevara J, Hirata Jr R, Canu S (2013) Kernel functions in takagi-sugeno-kang fuzzy system with nonsingleton fuzzy input. In: 2013 IEEE international conference on fuzzy systems, pp 1–8

  • Guevara J, Hirata R, Canu S (2014) Positive definite kernel functions on fuzzy sets. In: 2014 ieee international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 439–446

  • Guevara J, Hirata R, Canu S (2017) Cross product kernels for fuzzy set similarity. In: 2017 IEEE international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 1–6

  • Heo G, Gader P (2011) Robust kernel discriminant analysis using fuzzy memberships. Pattern Recogn 44(3):716–723

    MATH  Google Scholar 

  • Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425

    Google Scholar 

  • Huang HC, Chuang YY, Chen CS (2012) Multiple kernel fuzzy clustering. IEEE Trans Fuzzy Syst 20(1):120–134

    Google Scholar 

  • Jang JS (1993) Anfis: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23(3):665–685

    Google Scholar 

  • Jawanpuria P, Nath JS, Ramakrishnan G (2015) Generalized hierarchical kernel learning. J Mach Learn Res 16(1):617–652

    MathSciNet  MATH  Google Scholar 

  • Juang CF (2002) A tsk-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms. IEEE Trans Fuzzy Syst 10(2):155–170

    Google Scholar 

  • Juang CF, Chen GC (2012) A ts fuzzy system learned through a support vector machine in principal component space for real-time object detection. IEEE Trans Ind Electron 59(8):3309–3320

    Google Scholar 

  • Juang CF, Lin CT (1998) An online self-constructing neural fuzzy inference network and its applications. IEEE Trans Fuzzy Syst 6(1):12–32

    Google Scholar 

  • Juang CF, Chiu SH, Chang SW (2007) A self-organizing ts-type fuzzy network with support vector learning and its application to classification problems. IEEE Trans Fuzzy Syst 15(5):998–1008

    Google Scholar 

  • Kanzawa Y, Endo Y, Miyamoto S (2010) On kernel fuzzy c-means for data with tolerance using explicit mapping for kernel data analysis. In: 2010 IEEE international conference on fuzzy systems (FUZZ). IEEE, pp 1–6

  • Karypis G, Han EH, Kumar V (1999) Chameleon: Hierarchical clustering using dynamic modeling. Computer 32(8):68–75

    Google Scholar 

  • Khan NM, Ksantini R, Ahmad IS, Guan L (2014) Sn-svm: a sparse nonparametric support vector machine classifier. Signal Image Video Process 8(8):1625–1637

    Google Scholar 

  • Khayat O, Ebadzadeh MM, Shahdoosti HR, Rajaei R, Khajehnasiri I (2009) A novel hybrid algorithm for creating self-organizing fuzzy neural networks. Neurocomputing 73(1–3):517–524

    Google Scholar 

  • Klawonn F, Kruse R (1997) Constructing a fuzzy controller from data. Fuzzy Sets Syst 85(2):177–193

    MathSciNet  Google Scholar 

  • Kloft M, Brefeld U, Sonnenburg S, Zien A (2011) Lp-norm multiple kernel learning. J Mach Learn Res 12(Mar):953–997

    MathSciNet  MATH  Google Scholar 

  • Kosko B (1994) Fuzzy systems as universal approximators. IEEE Trans Comput 43(11):1329–1333

    MATH  Google Scholar 

  • Kroll A (1996) Identification of functional fuzzy models using multidimensional reference fuzzy sets. Fuzzy Sets Syst 80(2):149–158

    MathSciNet  Google Scholar 

  • Lanckriet GR, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5(Jan):27–72

    MathSciNet  MATH  Google Scholar 

  • Lee MM, Keerthi SS, Ong CJ, DeCoste D (2004) An efficient method for computing leave-one-out error in support vector machines with Gaussian kernels. IEEE Trans Neural Netw 15(3):750–757

    Google Scholar 

  • Leng G, Prasad G, McGinnity TM (2004) An on-line algorithm for creating self-organizing fuzzy neural networks. Neural Netw 17(10):1477–1493

    MATH  Google Scholar 

  • Leng G, McGinnity TM, Prasad G (2006) Design for self-organizing fuzzy neural networks based on genetic algorithms. IEEE Trans Fuzzy Syst 14(6):755–766

    Google Scholar 

  • Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(2):464–471

    Google Scholar 

  • Lin CT, Lee CG, Lin CT, Lin C (1996) Neural fuzzy systems: a neuro-fuzzy synergism to intelligent systems, vol 205. Prentice hall PTR, Upper Saddle River, NJ

    Google Scholar 

  • Lin CT, Yeh CM, Hsu CF et al (2004) Fuzzy neural network classification design using support vector machine. In: ISCAS, vol 5. pp 724–727

  • Lin CT, Yeh CM, Liang SF, Chung JF, Kumar N (2006) Support-vector-based fuzzy neural network for pattern classification. IEEE Trans Fuzzy Syst 14(1):31–41

    Google Scholar 

  • Malek H, Ebadzadeh MM, Rahmati M (2012) Three new fuzzy neural networks learning algorithms based on clustering, training error and genetic algorithm. Appl Intell 37(2):280–289

    Google Scholar 

  • Michie D, Spiegelhalter DJ, Taylor CC (1994) Machine learning, neural and statistical classification. http://ftp.stams.strath.ac.uk/pub/Statlog/pub//. Accessed 19 July 2018

  • Nauck D, Kruse R (1999) Neuro-fuzzy systems for function approximation. Fuzzy Sets Syst 101(2):261–271

    MathSciNet  MATH  Google Scholar 

  • Nauck D, Kruse R (1997) Function approximation by nefprox. In: Proceedings of second European workshop on fuzzy decision analysis and neural networks for management, planning, and optimization (EFDAN’97), pp 160–169

  • Oh SK, Kim WD, Pedrycz W, Park BJ (2011) Polynomial-based radial basis function neural networks (p-rbf nns) realized with the aid of particle swarm optimization. Fuzzy Sets Syst 163(1):54–77

    MathSciNet  MATH  Google Scholar 

  • Ong CS, Smola AJ, Williamson RC (2005) Learning the kernel with hyperkernels. J Mach Learn Res 6:1043–1071

    MathSciNet  MATH  Google Scholar 

  • Patel A, Al-Janabi S, AlShourbaji I, Pedersen J (2015) A novel methodology towards a trusted environment in mashup web applications. Comput Secur 49:107–122

    Google Scholar 

  • Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9(Nov):2491–2521

    MathSciNet  MATH  Google Scholar 

  • Schilling RJ, Carroll JJ, Al-Ajlouni AF (2001) Approximation of nonlinear systems with radial basis function neural networks. IEEE Trans Neural Netw 12(1):1–15

    Google Scholar 

  • Sollich P (2002) Bayesian methods for support vector machines: evidence and predictive class probabilities. Mach Learn 46(1–3):21–52

    MATH  Google Scholar 

  • Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern 1:116–132

    MATH  Google Scholar 

  • Tsanas A, Xifara A (2012) Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build 49:560–567

    Google Scholar 

  • Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines for multiclass problems. Neural Netw 16(5–6):785–792

    Google Scholar 

  • Vapnik V (2013) The nature of statistical learning theory. Springer, New York

    MATH  Google Scholar 

  • Vlassis N, Likas A (2002) A greedy em algorithm for Gaussian mixture learning. Neural Process Lett 15(1):77–87

    MATH  Google Scholar 

  • Wang JS, Lee CG (2001) Efficient neuro-fuzzy control systems for autonomous underwater vehicle control. In: Proceedings 2001 ICRA. IEEE international conference on robotics and automation, 2001. IEEE, vol 3, pp 2986–2991

  • Wang Y, Wang S, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Syst 13(6):820–831

    Google Scholar 

  • Wang S, Chung KF, Zhaohong D, Dewen H (2007) Robust fuzzy clustering neural network based on e-insensitive loss function. Appl Soft Comput 7(2):577–584

    Google Scholar 

  • Wang J, Hua J, Guo J (2010) Fuzzy maximum scatter discriminant analysis with kernel methods. In: 2010 Seventh international conference on fuzzy systems and knowledge discovery (FSKD). IEEE, vol 2, pp 560–564

  • Wu S, Er MJ (2000) Dynamic fuzzy neural networks-a novel approach to function approximation. IEEE Trans Syst Man Cybern Part B (Cybern) 30(2):358–364

    Google Scholar 

  • Wu S, Er MJ, Gao Y (2001) A fast approach for automatic generation of fuzzy rules by generalized dynamic fuzzy neural networks. IEEE Trans Fuzzy Syst 9(4):578–594

    Google Scholar 

  • Wu Z, Zhang H, Liu J (2014) A fuzzy support vector machine algorithm for classification based on a novel pim fuzzy clustering method. Neurocomputing 125:119–124

    Google Scholar 

  • Xiong H, Swamy M, Ahmad MO (2005) Optimizing the kernel in the empirical feature space. IEEE Trans Neural Netw 16(2):460–474

    Google Scholar 

  • Zhang D, Kamel M, Elmasry MI (1994) Fuzzy clustering neural network (FCNN): competitive learning and parallel architecture. J Intell Fuzzy Syst 2(4):289–298

    Google Scholar 

  • Zhong S, Chen D, Xu Q, Chen T (2013) Optimizing the Gaussian kernel function with the formulated kernel target alignment criterion for two-class pattern classification. Pattern Recogn 46(7):2045–2054

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Theoretical Physics of Condensed Matter (LPTMC), Sorbonne Universités, UPMC, 4 place Jussieu, 75005, Paris, France

    Ghazaleh Khodabandelou

  2. Computer Engineering Department, Amirkabir University of Technology, Hafez Street, Tehran, Iran

    Mohammad Mehdi Ebadzadeh

Authors
  1. Ghazaleh Khodabandelou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Mehdi Ebadzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ghazaleh Khodabandelou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and Animal Rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by A. Di Nola.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khodabandelou, G., Ebadzadeh, M.M. Fuzzy neural network with support vector-based learning for classification and regression. Soft Comput 23, 12153–12168 (2019). https://doi.org/10.1007/s00500-019-04116-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04116-x

Keywords

Search

Navigation