Advertisement

Soft Computing

, Volume 23, Issue 23, pp 12153–12168 | Cite as

Fuzzy neural network with support vector-based learning for classification and regression

  • Ghazaleh KhodabandelouEmail author
  • Mohammad Mehdi Ebadzadeh
Foundations

Abstract

Fuzzy neural network (FNN) and support vector machine (SVM) are two prominent and powerful learning models broadly used for classification and regression. FNN has a significant local representation and human reasoning advantage. However, the drawback of such a network is that the focal point of the learning algorithms is minimizing empirical risk. In contrary to FNN, SVM emphasizes simultaneously on minimizing empirical and expected risks, which theoretically leads to an excellent generalization performance power. In this paper, we show that a Takagi–Sugeno–Kang (TSK)-type-based fuzzy neural network is, in fact, equivalent to an SVM with an adaptive kernel based on fuzzy rules generated in this FNN. Consequently, it is possible to learn the last layer of the FNN using the concepts of SVM and thus taking the advantage of SVM in generalization. In fact, on the one side, the proposed method is an SVM with an adaptive kernel based on fuzzy rules and, on the other side, it is a TSK–FNN with SVM-based learning. As a matter of fact, the defined kernel in FNN is an adaptive kernel based on data characteristics, which is derived from the fuzzy rules generated by the FNN itself. The results obtained from the proposed method for classification and regression outperform the results of SVM with the conventional kernels and fuzzy neural network trained in the traditional way.

Keywords

Adaptive fuzzy kernel Classification Fuzzy neural network (FNN) Regression Support vector machine 

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Human and Animal Rights

This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. Aeberhard S, Coomans D, de Vel O (1992) The classification performance of RDA. Dept of Computer Science and Dept of Mathematics and Statistics, James Cook University of North Queensland, Tech Rep, pp 92–01Google Scholar
  2. Aiolli F, Donini M (2015) Easymkl: a scalable multiple kernel learning algorithm. Neurocomputing 169:215–224Google Scholar
  3. Al-Janabi S (2017) Pragmatic miner to risk analysis for intrusion detection (pmra-id). In: International conference on soft computing in data science, Springer, pp 263–277Google Scholar
  4. Al-Janabi S (2018) Smart system to create an optimal higher education environment using IDA and IOTs. Int J Comput Appl.  https://doi.org/10.1080/1206212X.2018.1512460 CrossRefGoogle Scholar
  5. Al-Janabi S, Abaid Mahdi M (2019) Evaluation prediction techniques to achieve optimal biomedical analysis. Int J Grid Util Comput.  https://doi.org/10.1504/IJGUC.2019.10020511 Google Scholar
  6. Al-Janabi S, Alkaim AF (2019) A nifty collaborative analysis to predicting a novel tool (DRFLLS) for missing values estimation. Soft Comput 1–15Google Scholar
  7. Al-Janabi S, Razaq F (2018) Intelligent big data analysis to design smart predictor for customer churn in telecommunication industry. In: International conference on big data and smart digital environment, Springer, pp 246–272Google Scholar
  8. Al-Janabi S, Al-Shourbaji I, Salman MA (2018) Assessing the suitability of soft computing approaches for forest fires prediction. Appl Comput Inform 14(2):214–224Google Scholar
  9. Al-Janabi S, Rawat S, Patel A, Al-Shourbaji I (2015) Design and evaluation of a hybrid system for detection and prediction of faults in electrical transformers. Int J Electr Power Energy Syst 67:324–335Google Scholar
  10. Argyriou A, Micchelli CA, Pontil M (2005) Learning convex combinations of continuously parameterized basic kernels. In: International conference on computational learning theory, Springer, pp 338–352Google Scholar
  11. Batuwita R, Palade V (2010) FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(3):558–571Google Scholar
  12. Berthold MR, Diamond J (1998) Constructive training of probabilistic neural networks. Neurocomputing 19(1–3):167–183Google Scholar
  13. Blake C, Merz C (1998) UCI machine learning repository. http://archive.ics.uci.edu/ml/datasets/. Accessed 19 July 2018
  14. Chai Y, Jia L, Zhang Z (2009) Mamdani model based adaptive neural fuzzy inference system and its application. Int J Comput Intell 5(1):22–29Google Scholar
  15. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159zbMATHGoogle Scholar
  16. Chen Y, Wang JZ (2003) Support vector learning for fuzzy rule-based classification systems. IEEE Trans Fuzzy Syst 11(6):716–728Google Scholar
  17. Chiang JH, Hao PY (2004) Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Trans Fuzzy Syst 12(1):1–12Google Scholar
  18. Chuang CC, Su SF, Chen SS (2001) Robust tsk fuzzy modeling for function approximation with outliers. IEEE Trans Fuzzy Syst 9(6):810–821Google Scholar
  19. Cortes C, Mohri M, Rostamizadeh A (2010) Two-stage learning kernel algorithms. In: ICML, pp 239–246Google Scholar
  20. Cristianini N, Shawe-Taylor J et al (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, CambridgezbMATHGoogle Scholar
  21. Cristianini N, Shawe-Taylor J, Elisseeff A, Kandola JS (2002) On kernel-target alignment. Advances in neural information processing systems. MIT press, Cambridge, pp 367–373Google Scholar
  22. De Vito S, Massera E, Piga M, Martinotto L, Di Francia G (2008) On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens Actuator B Chem 129(2):750–757Google Scholar
  23. Dickerson JA, Kosko B (1996) Fuzzy function approximation with ellipsoidal rules. IEEE Trans Syst Man Cybern Part B Cybern 26(4):542–560Google Scholar
  24. Ebadzadeh MM, Salimi-Badr A (2015) Cfnn: correlated fuzzy neural network. Neurocomputing 148:430–444Google Scholar
  25. Ekong U, Lam H, Xiao B, Ouyang G, Liu H, Chan KY, Ling SH (2016) Classification of epilepsy seizure phase using interval type-2 fuzzy support vector machines. Neurocomputing 199:66–76Google Scholar
  26. Esme E, Karlik B (2016) Fuzzy c-means based support vector machines classifier for perfume recognition. Appl Soft Comput 46:452–458Google Scholar
  27. Ester M, Kriegel HP, Sander J, Xu X et al (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. Kdd 96:226–231Google Scholar
  28. Fan Q, Wang Z, Li D, Gao D, Zha H (2017) Entropy-based fuzzy support vector machine for imbalanced datasets. Knowl Based Syst 115:87–99Google Scholar
  29. Ferreira RP, Martiniano A, Ferreira A, Ferreira A, Sassi RJ (2016) Study on daily demand forecasting orders using artificial neural network. IEEE Latin Am Trans 14(3):1519–1525Google Scholar
  30. Fraley C, Raftery AE (1998) How many clusters? Which clustering method? Answers via model-based cluster analysis. Comput J 41(8):578–588zbMATHGoogle Scholar
  31. Fraley C, Raftery AE (2002) Model-based clustering, discriminant analysis, and density estimation. J Am Stat Assoc 97(458):611–631MathSciNetzbMATHGoogle Scholar
  32. Gabrys B, Bargiela A (2000) General fuzzy min-max neural network for clustering and classification. IEEE Trans Neural Netw 11(3):769–783Google Scholar
  33. Ganji H, Khadivi S, Ebadzadeh MM (2017) Support vector-based fuzzy classifier with adaptive kernel. Neural Comput Appl.  https://doi.org/10.1007/s00521-017-3170-x CrossRefGoogle Scholar
  34. González J, Rojas H, Ortega J, Prieto A (2002) A new clustering technique for function approximation. IEEE Trans Neural Netw 13(1):132–142Google Scholar
  35. Guevara J (2016) Supervised machine learning with kernel embeddings of fuzzy sets and probability measures. PhD Thesis, IME USPGoogle Scholar
  36. Guevara J, Hirata Jr R, Canu S (2013) Kernel functions in takagi-sugeno-kang fuzzy system with nonsingleton fuzzy input. In: 2013 IEEE international conference on fuzzy systems, pp 1–8Google Scholar
  37. Guevara J, Hirata R, Canu S (2014) Positive definite kernel functions on fuzzy sets. In: 2014 ieee international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 439–446Google Scholar
  38. Guevara J, Hirata R, Canu S (2017) Cross product kernels for fuzzy set similarity. In: 2017 IEEE international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 1–6Google Scholar
  39. Heo G, Gader P (2011) Robust kernel discriminant analysis using fuzzy memberships. Pattern Recogn 44(3):716–723zbMATHGoogle Scholar
  40. Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425Google Scholar
  41. Huang HC, Chuang YY, Chen CS (2012) Multiple kernel fuzzy clustering. IEEE Trans Fuzzy Syst 20(1):120–134Google Scholar
  42. Jang JS (1993) Anfis: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 23(3):665–685Google Scholar
  43. Jawanpuria P, Nath JS, Ramakrishnan G (2015) Generalized hierarchical kernel learning. J Mach Learn Res 16(1):617–652MathSciNetzbMATHGoogle Scholar
  44. Juang CF (2002) A tsk-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms. IEEE Trans Fuzzy Syst 10(2):155–170Google Scholar
  45. Juang CF, Chen GC (2012) A ts fuzzy system learned through a support vector machine in principal component space for real-time object detection. IEEE Trans Ind Electron 59(8):3309–3320Google Scholar
  46. Juang CF, Lin CT (1998) An online self-constructing neural fuzzy inference network and its applications. IEEE Trans Fuzzy Syst 6(1):12–32Google Scholar
  47. Juang CF, Chiu SH, Chang SW (2007) A self-organizing ts-type fuzzy network with support vector learning and its application to classification problems. IEEE Trans Fuzzy Syst 15(5):998–1008Google Scholar
  48. Kanzawa Y, Endo Y, Miyamoto S (2010) On kernel fuzzy c-means for data with tolerance using explicit mapping for kernel data analysis. In: 2010 IEEE international conference on fuzzy systems (FUZZ). IEEE, pp 1–6Google Scholar
  49. Karypis G, Han EH, Kumar V (1999) Chameleon: Hierarchical clustering using dynamic modeling. Computer 32(8):68–75Google Scholar
  50. Khan NM, Ksantini R, Ahmad IS, Guan L (2014) Sn-svm: a sparse nonparametric support vector machine classifier. Signal Image Video Process 8(8):1625–1637Google Scholar
  51. Khayat O, Ebadzadeh MM, Shahdoosti HR, Rajaei R, Khajehnasiri I (2009) A novel hybrid algorithm for creating self-organizing fuzzy neural networks. Neurocomputing 73(1–3):517–524Google Scholar
  52. Klawonn F, Kruse R (1997) Constructing a fuzzy controller from data. Fuzzy Sets Syst 85(2):177–193MathSciNetGoogle Scholar
  53. Kloft M, Brefeld U, Sonnenburg S, Zien A (2011) Lp-norm multiple kernel learning. J Mach Learn Res 12(Mar):953–997MathSciNetzbMATHGoogle Scholar
  54. Kosko B (1994) Fuzzy systems as universal approximators. IEEE Trans Comput 43(11):1329–1333zbMATHGoogle Scholar
  55. Kroll A (1996) Identification of functional fuzzy models using multidimensional reference fuzzy sets. Fuzzy Sets Syst 80(2):149–158MathSciNetGoogle Scholar
  56. Lanckriet GR, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5(Jan):27–72MathSciNetzbMATHGoogle Scholar
  57. Lee MM, Keerthi SS, Ong CJ, DeCoste D (2004) An efficient method for computing leave-one-out error in support vector machines with Gaussian kernels. IEEE Trans Neural Netw 15(3):750–757Google Scholar
  58. Leng G, Prasad G, McGinnity TM (2004) An on-line algorithm for creating self-organizing fuzzy neural networks. Neural Netw 17(10):1477–1493zbMATHGoogle Scholar
  59. Leng G, McGinnity TM, Prasad G (2006) Design for self-organizing fuzzy neural networks based on genetic algorithms. IEEE Trans Fuzzy Syst 14(6):755–766Google Scholar
  60. Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(2):464–471Google Scholar
  61. Lin CT, Lee CG, Lin CT, Lin C (1996) Neural fuzzy systems: a neuro-fuzzy synergism to intelligent systems, vol 205. Prentice hall PTR, Upper Saddle River, NJGoogle Scholar
  62. Lin CT, Yeh CM, Hsu CF et al (2004) Fuzzy neural network classification design using support vector machine. In: ISCAS, vol 5. pp 724–727Google Scholar
  63. Lin CT, Yeh CM, Liang SF, Chung JF, Kumar N (2006) Support-vector-based fuzzy neural network for pattern classification. IEEE Trans Fuzzy Syst 14(1):31–41Google Scholar
  64. Malek H, Ebadzadeh MM, Rahmati M (2012) Three new fuzzy neural networks learning algorithms based on clustering, training error and genetic algorithm. Appl Intell 37(2):280–289Google Scholar
  65. Michie D, Spiegelhalter DJ, Taylor CC (1994) Machine learning, neural and statistical classification. http://ftp.stams.strath.ac.uk/pub/Statlog/pub//. Accessed 19 July 2018
  66. Nauck D, Kruse R (1999) Neuro-fuzzy systems for function approximation. Fuzzy Sets Syst 101(2):261–271MathSciNetzbMATHGoogle Scholar
  67. Nauck D, Kruse R (1997) Function approximation by nefprox. In: Proceedings of second European workshop on fuzzy decision analysis and neural networks for management, planning, and optimization (EFDAN’97), pp 160–169Google Scholar
  68. Oh SK, Kim WD, Pedrycz W, Park BJ (2011) Polynomial-based radial basis function neural networks (p-rbf nns) realized with the aid of particle swarm optimization. Fuzzy Sets Syst 163(1):54–77MathSciNetzbMATHGoogle Scholar
  69. Ong CS, Smola AJ, Williamson RC (2005) Learning the kernel with hyperkernels. J Mach Learn Res 6:1043–1071MathSciNetzbMATHGoogle Scholar
  70. Patel A, Al-Janabi S, AlShourbaji I, Pedersen J (2015) A novel methodology towards a trusted environment in mashup web applications. Comput Secur 49:107–122Google Scholar
  71. Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9(Nov):2491–2521MathSciNetzbMATHGoogle Scholar
  72. Schilling RJ, Carroll JJ, Al-Ajlouni AF (2001) Approximation of nonlinear systems with radial basis function neural networks. IEEE Trans Neural Netw 12(1):1–15Google Scholar
  73. Sollich P (2002) Bayesian methods for support vector machines: evidence and predictive class probabilities. Mach Learn 46(1–3):21–52zbMATHGoogle Scholar
  74. Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern 1:116–132zbMATHGoogle Scholar
  75. Tsanas A, Xifara A (2012) Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build 49:560–567Google Scholar
  76. Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines for multiclass problems. Neural Netw 16(5–6):785–792Google Scholar
  77. Vapnik V (2013) The nature of statistical learning theory. Springer, New YorkzbMATHGoogle Scholar
  78. Vlassis N, Likas A (2002) A greedy em algorithm for Gaussian mixture learning. Neural Process Lett 15(1):77–87zbMATHGoogle Scholar
  79. Wang JS, Lee CG (2001) Efficient neuro-fuzzy control systems for autonomous underwater vehicle control. In: Proceedings 2001 ICRA. IEEE international conference on robotics and automation, 2001. IEEE, vol 3, pp 2986–2991Google Scholar
  80. Wang Y, Wang S, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Syst 13(6):820–831Google Scholar
  81. Wang S, Chung KF, Zhaohong D, Dewen H (2007) Robust fuzzy clustering neural network based on e-insensitive loss function. Appl Soft Comput 7(2):577–584Google Scholar
  82. Wang J, Hua J, Guo J (2010) Fuzzy maximum scatter discriminant analysis with kernel methods. In: 2010 Seventh international conference on fuzzy systems and knowledge discovery (FSKD). IEEE, vol 2, pp 560–564Google Scholar
  83. Wu S, Er MJ (2000) Dynamic fuzzy neural networks-a novel approach to function approximation. IEEE Trans Syst Man Cybern Part B (Cybern) 30(2):358–364Google Scholar
  84. Wu S, Er MJ, Gao Y (2001) A fast approach for automatic generation of fuzzy rules by generalized dynamic fuzzy neural networks. IEEE Trans Fuzzy Syst 9(4):578–594Google Scholar
  85. Wu Z, Zhang H, Liu J (2014) A fuzzy support vector machine algorithm for classification based on a novel pim fuzzy clustering method. Neurocomputing 125:119–124Google Scholar
  86. Xiong H, Swamy M, Ahmad MO (2005) Optimizing the kernel in the empirical feature space. IEEE Trans Neural Netw 16(2):460–474Google Scholar
  87. Zhang D, Kamel M, Elmasry MI (1994) Fuzzy clustering neural network (FCNN): competitive learning and parallel architecture. J Intell Fuzzy Syst 2(4):289–298Google Scholar
  88. Zhong S, Chen D, Xu Q, Chen T (2013) Optimizing the Gaussian kernel function with the formulated kernel target alignment criterion for two-class pattern classification. Pattern Recogn 46(7):2045–2054zbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Theoretical Physics of Condensed Matter (LPTMC)Sorbonne Universités, UPMCParisFrance
  2. 2.Computer Engineering DepartmentAmirkabir University of TechnologyTehranIran

Personalised recommendations