A Combination-of-Tools Method for Learning Interpretable Fuzzy Rule-Based Classifiers from Support Vector Machines

  • Tamas Kenesei
  • Johannes A. Roubos
  • Janos Abonyi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4881)


A new approach is proposed for the data-based identification of transparent fuzzy rule-based classifiers. It is observed that fuzzy rule-based classifiers work in a similar manner as kernel function-based support vector machines (SVMs) since both model the input space by nonlinearly maps into a feature space where the decision can be easily made. Accordingly, trained SVM can be used for the construction of fuzzy rule-based classifiers. However, the transformed SVM does not automatically result in an interpretable fuzzy model because the SVM results in a complex rule-base, where the number of rules is approximately 40-60% of the number of the training data. Hence, reduction of the SVM-initialized classifier is an essential task. For this purpose, a three-step reduction algorithm is developed based on the combination of previously published model reduction techniques. In the first step, the identification of the SVM is followed by the application of the Reduced Set method to decrease the number of kernel functions. The reduced SVM is then transformed into a fuzzy rule-based classifier. The interpretability of a fuzzy model highly depends on the distribution of the membership functions. Hence, the second reduction step is achieved by merging similar fuzzy sets based on a similarity measure. Finally, in the third step, an orthogonal least-squares method is used to reduce the number of rules and re-estimate the consequent parameters of the fuzzy rule-based classifier. The proposed approach is applied for the Wisconsin Breast Cancer, Iris and Wine classification problems to compare its performance to other methods.


Classification Fuzzy classifier Support Vector Machine Model Reduction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    de Valente Oliveira, J.: Semantic constraints for membership function optimization. IEEE Trans. FS 19, 128–138 (1999)Google Scholar
  2. 2.
    Nauck, D., Kruse, R.: Obtaining interpretable fuzzy classification rules from medical data. Artificial Intelligence in Medicine 16, 149–169 (1999)CrossRefGoogle Scholar
  3. 3.
    Setnes, M., Babuška, R., Kaymak, U., van Nauta Lemke, H.R.: Similarity measures in fuzzy rule base simplification. IEEE Trans. SMC-B 28, 376–386 (1998)Google Scholar
  4. 4.
    Jin, Y.: Fuzzy Modeling of High-Dimensional Systems. IEEE Trans. FS 8, 212–221 (2000)Google Scholar
  5. 5.
    Setnes, M., Roubos, J.A.: GA-Fuzzy Modeling and Classification: Complexity and Performance. IEEE Trans. FS 8, 509–522 (2000)Google Scholar
  6. 6.
    Roubos, J.A., Abonyi, J., Setnes, M.: Learning fuzzy classification rules from labeled data. International Journal of Information Sciences 150, 77–93 (2003)MathSciNetGoogle Scholar
  7. 7.
    Ishibuchi, H., Nakashima, T., Murata, T.: Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems. IEEE Trans. SMC–B 29, 601–618 (1999)Google Scholar
  8. 8.
    Schölkopf, B., Sung, K., Burges, C., Girosi, F., Niyogi, P., Poggio, T., Vapnik, V.: Comparing support vector machines with gaussian kernels to radial basis function classifiers. IEEE Trans Sign. Processing 45, 2758–2765 (1997)CrossRefGoogle Scholar
  9. 9.
    Chan, W.C., Cheung, K.C., Harris, C.J.: Modelling of Nonlinear Dynamic Systems Using Support Vector Machines. In: Proc. of the IFAC Symposium on Artificial Intelligence, Budapest, Hungary, pp. 217–222 (2000)Google Scholar
  10. 10.
    Jeng, J.T., Lee, T.T.: Support Vector Machines for Fuzzy Neural Networks. In: Proc. of the IEEE SMC conference, pp. VI–115–120 (1999)Google Scholar
  11. 11.
    Jang, J.-S.R., Sun, C.-T.: Functional Equivalence Between Radial Basis Function Networks and Fuzzy Inference Systems. IEEE Trans. NN 4, 156–159 (1993)Google Scholar
  12. 12.
    Camps–Valls, G., Martn–Guerrero, J.D., Rojo-lvarez, J.L., Soria–Olivas, E.: Fuzzy sigmoid kernel for support vector classifiers. Neurocomputing 62, 501–506 (2004)CrossRefGoogle Scholar
  13. 13.
    Yao, C.–C., Yu, P.-T.: Fuzzy regression based on asymmetric support vector machines. Applied Mathematics and Computation 182, 175–193Google Scholar
  14. 14.
    Celikyilmaz, A., Türksen, B.I.: Fuzzy Functions with Support Vector Machines, Accepted Manuscript to appear in Information SciencesGoogle Scholar
  15. 15.
    Leski, J.M.: On support vector regression machines with linguistic interpretation of the kernel matrix. Fuzzy Sets and Systems 157, 1092–1113 (2006)zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Huang, X., Chen, S.: SVM–based fuzzy modeling for the arc welding process. Material Science and Engineering A 427, 181–187 (2006)CrossRefGoogle Scholar
  17. 17.
    Wang, T.–Y., Chiang, H.-m.: Fuzzy support vector machine for multi–class text categorization. Information Processing and Management 43, 914–929 (2007)CrossRefGoogle Scholar
  18. 18.
    Schölkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Müller, K.-R., Rätsch, G., Smola, A.: Input space vs. feature space in kernel-based methods. IEEE Trans. on Neural Networks 10(5), 1000–1017 (1999)CrossRefGoogle Scholar
  19. 19.
    Yen, J., Wang, L.: Simplifying fuzzy rule-based models using orthogonal transformation methods. IEEE Trans. SMC-B 29, 13–24 (1999)Google Scholar
  20. 20.
    Setnes, M., Hellendoorn, H.: Orthogonal transforms for ordering and reduction of fuzzy rules. In: FUZZ-IEEE, San Antonio, Texas, USA, pp. 700–705 (2000)Google Scholar
  21. 21.
    Takagi, T., Sugeno, M.: Fuzzy identification of systems and its application to modeling and control. IEEE Trans. SMC 15, 116–132 (1985)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Tamas Kenesei
    • 1
  • Johannes A. Roubos
    • 1
  • Janos Abonyi
    • 1
  1. 1.Department of Process Engineering, University of Pannonia, Veszprem P.O.Box 158, H-8201Hungary

Personalised recommendations