Skip to main content
SpringerLink
Log in

Fuzzy rules extraction from support vector machines for multi-class classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a new method for fuzzy rule extraction from trained support vector machines (SVMs) for multi-class problems, named FREx_SVM. SVMs have been used in a variety of applications. However, they are considered “black box models,” where no interpretation about the input–output mapping is provided. Some methods to reduce this limitation have already been proposed, but they are restricted to binary classification problems and to the extraction of symbolic rules with intervals or functions in their antecedents. In order to improve the interpretability of the generated rules, this paper presents a new model for extracting fuzzy rules from a trained SVM. The proposed model is suited for classification in multi-class problems and includes a wrapper feature selection algorithm. It is evaluated in four benchmark databases, and results obtained demonstrate its capacity to generate a reduced set of interpretable fuzzy rules that explains both the classification database and the influence of each input variable on the determination of the final class.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Abe S, Inoue T (2002) Fuzzy support vector machines for multiclass problems, In: European symposium on artificial neural networks proceedings (ESANN), pp 113–118

  2. Barakat N, Bradley A (2010) Rule extraction from support vector machines: a review. Neurocomputing 74:178–190

    Article  Google Scholar 

  3. Barakat N, Diederich J (2004) Learning-based rule-extraction from support vector machines: performance on benchmark data sets. In: Proceedings of the conference on neuro-computing and evolving intelligence, Knowledge Engineering and Discovery Research Institute (KEDRI), Auckland, New Zealand

  4. Barakat N, Diederich J (2005) Eclectic rule-extraction from support vector machines. Int J Comput Intell 2:59–62

    Google Scholar 

  5. Castro J, Flores-Hidalgo L, Mantas C, Puche J (2007) Extraction of fuzzy rules from support vector machines. Fuzzy Sets Syst 158:2057–2077

    Article  MathSciNet  MATH  Google Scholar 

  6. Chen Z, Li J, Wei L (2007) A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue. Artif Intell Med 41:161–175

    Article  Google Scholar 

  7. Chiang J, Hao P (2004) Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Trans Fuzzy Syst 12(1):1–12

    Article  Google Scholar 

  8. Crammer K, Singer Y (2000) On the learnability and design of output codes for multiclass problems. In: COLT’00 proceedings of the thirteenth annual conference on computational learning theory, pp 35–46

  9. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel—based learning methods. Cambridge University Press, Cambridge, MA

    Book  Google Scholar 

  10. Diederich J (ed) (2008) Rule extraction from support vector machines, studies in computational intelligence, vol 80. Springer, Berlin, pp 3–30

    Book  Google Scholar 

  11. Fu X, Ong CJ, Keerthi S, Hung GG, Goh L (2004) Extracting the knowledge embedded in support vector machines, international joint conference on neural networks (IJCNN’04), Budapest, July 25–29, 2004, CDROM

  12. Fung G, Sandilya S, Rao R (2005) Rule extraction from linear support vector machines. In: Proceedings of the eleventh SIGKDD international conference on knowledge discovery and data mining

  13. Gonçalves LB, Vellasco MMBR, Pacheco MAC, de Souza FJ (2006) Inverted hierarchical neuro—fuzzy BSP system: a novel neuro—fuzzy model for pattern classification and rule extraction in databases. IEEE Transact Syst Man Cybern Part C Appl Rev 36–2:236–248

    Article  Google Scholar 

  14. Haykin S (1999) Neural networks—a comprehensive foundation. Macmillan College Publishing Company, New York

    MATH  Google Scholar 

  15. Heiler M, Cremers D, Schnörr C (2001) Efficient feature subset selection for support vector machines, technical report 21/2001, computer sciences series, University of Mannheim, Germany

  16. Hsu C-W, Lin C-J (2002) A comparison on methods for multi-class support vector machines. IEEE Transact Neural Netw 13–2:415–425

    Google Scholar 

  17. John G, Kohavi R, Pfleger K (1994) Irrelevant features and the subset selection problem. In: Proceedings of the 11th international conference on machine learning, pp 121–129

  18. Fayyad UM, Piatetsky-Shapiro G, Smyth P, Uthurusamy R (1996) Advances in knowledge discovery and data mining. MIT Press, Cambridge

  19. Kressel UH-G (1999) Pairwise classification and support vector machines. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods: support vector learning. MIT Press, Cambridge, MA, pp 225–268

    Google Scholar 

  20. Lanas AI (2000) Sistemas neuro—fuzzy hierárquicos BSP para Previsão e Extração de Regras Fuzzy em Aplicações de Mineração de Dados, M.S. thesis, Department Electrical Engineering, PUC-Rio (in Portuguese)

  21. Leng G, McGinnity TM (2004) G. Prasad, an approach for on-line extraction of fuzzy rules using a self-organising fuzzy neural network, in: fuzzy sets system 150:211–243

    MathSciNet  Google Scholar 

  22. Mendel JM (1995) Fuzzy logic systems for engineering: a tutorial. Proceed IEEE 83–3:345–377

    Article  Google Scholar 

  23. Moody J, Utans J (1992) Principled architecture selection for neural networks: application to corporate bond rating prediction, advances in neural information processing systems 4. Morgan Kaufmann Publishers, Los Altos, CA, pp 683–690

    Google Scholar 

  24. Núñez H, Angulo C, Català A (2002) Rule extraction from support vectors machines, European symposium on artificial neural networks (ESANN), pp 107–112

  25. Núñez H, Angulo C, Catala A (2006) Rule-based learning systems for support vector machines. Neural Process Lett 24:1–18

    Article  Google Scholar 

  26. Pitiraggon P, Benjathepanun N, Banditvilai S, Boonjing V (2010) Fuzzy rules generation and extraction from support vector machine based on kernel function firing signals. Int J Eng Appl Sci 6(4):244–251

    Google Scholar 

  27. Rifkin R, Klautau A (2004) In defense of one-vs-all classification. J Mach Leran Res 5:101–141

    MathSciNet  MATH  Google Scholar 

  28. Sculley D, Wachman G (2007) Relaxed online SVMs for spam filtering, In: the thirtieth annual ACM SIGIR conference proceedings

  29. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  30. Vapnik VN, Golowich SE, Smola A (1997) Support vector method for function approximation, regression estimation, and signal processing, In: advances in neural information processing system 9, Morgan Kaufmann, pp 281–287

  31. Wang L-X, Mendel JM (1992) Generating fuzzy rules by learning from examples. IEEE Transact Syst Man Cybern 22–6:1414–1427

    Article  MathSciNet  Google Scholar 

  32. Weston J, Watkins C (1998) Multi-class support vector machines, technical report CSD-TR-98-04, Royal Holloway

  33. Zhang J, Jin R, Yang YM, Hauptmann A (2003) A modified logistic regression: an approximation to SVM and its applications in large-scale text categorization, In: Proceedings of the twentieth international conference, Washington, DC, USA, August 21–24, 2003, pp 888–895

Download references

Acknowledgments

This research has been funded by the Rio de Janeiro Research Foundation (FAPERJ) under process number E-26/170.878/2007.

Author information

Authors and Affiliations

  1. Electrical Engineering Department, Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Rua Marquês de São Vicente 225, Prédio Cardeal Leme, 4o andar, Rio de Janeiro, 22451-900, Brazil

    Adriana da Costa F. Chaves, Marley Maria B. R. Vellasco & Ricardo Tanscheit

Authors
  1. Adriana da Costa F. Chaves
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marley Maria B. R. Vellasco
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ricardo Tanscheit
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marley Maria B. R. Vellasco.

Rights and permissions

Reprints and permissions

About this article

Cite this article

da Costa F. Chaves, A., Vellasco, M.M.B.R. & Tanscheit, R. Fuzzy rules extraction from support vector machines for multi-class classification. Neural Comput & Applic 22, 1571–1580 (2013). https://doi.org/10.1007/s00521-012-1048-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-1048-5

Keywords

Search

Navigation