Skip to main content

Advertisement

SpringerLink
Log in

Rule extraction from support vector machines by genetic algorithms

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Support vector machines (SVMs) are state-of-the-art tools used to address issues pertinent to classification. However, the explanation capabilities of SVMs are also their main weakness, which is why SVMs are typically regarded as incomprehensible black box models. In the present study, a rule extraction algorithm to extract the comprehensible rule from SVMs and enhance their explanation capability is proposed. The proposed algorithm seeks to use the support vectors from a training model of SVMs and combine genetic algorithms for constructing rule sets. The proposed method can not only generate rule sets from SVMs based on the mixed discrete and continuous variables but can also select important variables in the rule set simultaneously. Measurements of accuracy, sensitivity, specificity, and fidelity are utilized to compare the performance of the proposed method with direct learner algorithms and several rule-extraction techniques from SVMs. The results indicate that the proposed method performs at least as well as with the most successful direct rule learners. Finally, an actual case of pressure ulcer was studied, and the results indicated the practicality of our proposed method in real applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Barakat N, Bradley A (2007) Rule extraction from support vector machines: a sequential covering approach. IEEE Trans Knowl Data Eng 19(6):729–741

    Article  Google Scholar 

  2. Barakat N, Bradley AP (2010) Rule extraction from support vector machines: a review. Neurocomputing 74(1–3):178–190

    Article  Google Scholar 

  3. Barakat N, Diederich J (2004) Learning-based rule-extraction from support vector machines: performance on benchmark data sets. Proc Conf Neuro Comput Evol Intell

  4. Bojarczuk CC, Lopes HS, Freitas AA (2000) Genetic programming for knowledge discovery in chest-pain diagnosis. IEEE Eng Med Biol Mag 19(4):38–44

    Article  Google Scholar 

  5. Breiman L, Friedman J, Olshen R, Stone C (1994) Classification and regression trees. Wadsworth and Brooks, Monterey

    Google Scholar 

  6. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2(2):121–167

    Article  Google Scholar 

  7. Cohen WW (1995) Fast effective rule induction. In: Proceedings of 12th international conference maching learning, pp 115–123

  8. Craven MW, Shavlik JW (1996) Extracting tree-structured representations of trained networks. Adv Neural Inf Process Syst 8:24–30

    Google Scholar 

  9. Davis LD, Mitchell M (1991) Handbook of genetic algorithms. Van Nostrand Reinhold, New York

    Google Scholar 

  10. Demsar J (2006) Statistical comparison of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  11. Diederich J (2008) Rule extraction from support vector machines: an introduction. In: Rule extraction from support vector machines, vol 80. Studies in computational intelligence. Springer, Berlin, pp 3–31. doi:10.1007/978-3-540-75390-2_1

  12. Frank A, Asuncion A (2010) UCI machine learning repository. (2011/02/24). Retrieved from http://archive.ics.uci.edu/ml

  13. Freitas AA (2002) Data mining and knowledge discovery with evolutionary algorithms. Springer, New York

    Book  MATH  Google Scholar 

  14. Fu L (1994) Rule generation from neural network. IEEE Trans Syst Man Cybernet 23(8):1114–1124

    Google Scholar 

  15. Fung G, Sandilya S, Rao RB (2005) Rule extraction from linear support vector machines. In: Proceedings of the eleventh ACM SIGKDD international conference on knowledge discovery in data mining, Chicago, Illinois, USA, ACM

  16. Goldberg D (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Boston

    MATH  Google Scholar 

  17. Hall M, Frank E, Geoffrey H, Pfahringer B, Reutemann P, Witten IH (2009) The weka data mining software: An update. SIGKDD Explor 11(1):10–18

    Article  Google Scholar 

  18. Holland J (1975) Adaptation in natural and artificial systems. The University of Michigan Press, Ann Arbor

    Google Scholar 

  19. Johansson U, KÄonig R, Niklasson L (2003) Rule extraction from trained neural networks using genetic programming. In: 13th international conference on artificial neural networks, Istanbul, Turkey, pp 13–16

  20. Johansson U, KÄonig, R, Niklasson L (2004) The truth is in there—rule extraction from opaque models using genetic programming. In: Proceedings of the 17th international Florida AI research symposium conference (FLAIRS)

  21. Markowska-Kaczmar U, Chumieja M (2004) Discovering the mysteries of neural networks. Int J Hybrid Intell Syst 1(3,4):153–163

    MATH  Google Scholar 

  22. Martens D, Baesens BB, Van Gestel T (2009) Decompositional rule extraction from support vector machines by active learning. IEEE Trans Knowl Data Eng 21(2):178–191

    Article  Google Scholar 

  23. Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs. Springer, Berlin

    MATH  Google Scholar 

  24. Nello C, John S-T (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    Google Scholar 

  25. Nunez H, Angulo C (2002). Rule extraction from support vector machines. Proc Eur Symp Artif Neural Netw 107–112

  26. Pang-Ning T, Steinbach M, Kumar V (2005) Introduction to data mining. Addison-Wesley, Boston

  27. Provost F, Fawcett T (2001) Robust classification for imprecise environments. Mach Learn 42(3):203–231

    Article  MATH  Google Scholar 

  28. Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco, CA

    Google Scholar 

  29. Sexton RS, McMurtrey S, Cleavenger D (2006) Knowledge discovery using a neural network simultaneous optimization algorithm on a real world classification problem. Eur J Oper Res 168(3):1009–1018

    Article  MATH  Google Scholar 

  30. Tan KC, Yu Q, Heng CM, Lee TH (2003) Evolutionary computing for knowledge discovery in medical diagnosis. Artif Intell Med 27:129–155

    Article  Google Scholar 

  31. Vapnik V (1995) The nature of statistical learning theory. Springer, New York

    Book  MATH  Google Scholar 

  32. Yuchun T, Yan-Qing Z, Chawla NV, Krasser S (2009) Svms modeling for highly imbalanced classification. IEEE Trans Syst Man Cybern B Cybern 39(1):281–288

    Article  Google Scholar 

Download references

Acknowledgment

This work was supported in part by the National Science Council, Taiwan, under grant NSC-98-2221-E-007-071-MY3 and 100-2410-H-007-050-MY2.

Author information

Authors and Affiliations

  1. Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Room 820, Engineering Building I, 101, Sec. 2, Kuang Fu Road, Hsinchu, 30013, Taiwan

    Chao-Ton Su

  2. Institute of Manufacturing Information and Systems, National Cheng Kung University, Tainan City, Taiwan

    Taho Yang

  3. Chung-Shan Institute of Science and Technology, No. 15, Shi Qi Zi, Gaoping village, Longtan Township, Taoyuan County, Taiwan

    Yan-Cheng Chen

Authors
  1. Yan-Cheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao-Ton Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Taho Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao-Ton Su.

Appendix

Appendix

See Table 9.

Table 9 The Pseudo code of GASVM
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, YC., Su, CT. & Yang, T. Rule extraction from support vector machines by genetic algorithms. Neural Comput & Applic 23, 729–739 (2013). https://doi.org/10.1007/s00521-012-0985-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-0985-3

Keywords

Search

Navigation