Soft Computing

, Volume 15, Issue 10, pp 1959–1980 | Cite as

HILK++: an interpretability-guided fuzzy modeling methodology for learning readable and comprehensible fuzzy rule-based classifiers

Focus

Abstract

This work presents a methodology for building interpretable fuzzy systems for classification problems. We consider interpretability from two points of view: (1) readability of the system description and (2) comprehensibility of the system behavior explanations. The fuzzy modeling methodology named as Highly Interpretable Linguistic Knowledge (HILK) is upgraded. Firstly, a feature selection procedure based on crisp decision trees is carried out. Secondly, several strong fuzzy partitions are automatically generated from experimental data for all the selected inputs. For each input, all partitions are compared and the best one according to data distribution is selected. Thirdly, a set of linguistic rules are defined combining the previously generated linguistic variables. Then, a linguistic simplification procedure guided by a novel interpretability index is applied to get a more compact and general set of rules with a minimum loss of accuracy. Finally, partition tuning based on two efficient search strategies increases the system accuracy while preserving the high interpretability. Results obtained in several benchmark classification problems are encouraging because they show the ability of the new methodology for generating highly interpretable fuzzy rule-based classifiers while yielding accuracy comparable to that achieved by other methods like neural networks and C4.5. The best configuration of HILK will depend on each specific problem under consideration but it is important to remark that HILK is flexible enough (thanks to the combination of several algorithms in each modeling stage) to be easily adaptable to a wide range of problems.

Keywords

Fuzzy modeling Interpretability Classification Simplification Tuning 

References

  1. Abonyi J, Roubos JA, Szeifert F (2003) Data-driven generation of compact, accurate, and linguistically sound fuzzy classifiers based on a decision-tree initialization. Int J Approx Reason 32:1–21MATHCrossRefGoogle Scholar
  2. Alcalá R, Alcalá-Fdez J, Gacto MJ, Herrera F (2007) Rule base reduction and genetic tuning of fuzzy systems based on the linguistic 3-tuples representation. Soft Comput 11(5):401–419CrossRefGoogle Scholar
  3. Alonso JM, Cordón O, Guillaume S, Magdalena L (2007) Highly interpretable linguistic knowledge bases optimization: genetic tuning versus Solis-Wetts. Looking for a good interpretability–accuracy trade-off. In: Annual IEEE international conference on fuzzy systems, London, UK, pp 901–906Google Scholar
  4. Alonso JM, Magdalena L, Guillaume S (2008) HILK: a new methodology for designing highly interpretable linguistic knowledge bases using the fuzzy logic formalism. Int J Intell Syst 23(7):761–794MATHCrossRefGoogle Scholar
  5. Alonso JM, Magdalena L, González-Rodríguez G (2009) Looking for a good fuzzy system interpretability index: an experimental approach. Int J Approx Reason 51:115–134CrossRefGoogle Scholar
  6. Antonelli M, Ducange P, Lazzerini B, Marcelloni F (2009) Exploiting a new interpretability index in the multi-objective evolutionary learning of mamdani fuzzy rule-based systems. In: Ninth international conference on intelligent system design and applications, IEEE, Pisa, Italy, pp 115–120Google Scholar
  7. Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum Press, New YorkMATHGoogle Scholar
  8. Casillas J, Cordón O, Herrera F, Magdalena L (2003a) Accuracy improvements in linguistic fuzzy modeling, vol 129. Studies in fuzziness and soft computing. Springer, HeidelbergGoogle Scholar
  9. Casillas J, Cordón O, Herrera F, Magdalena L (2003b) Interpretability issues in fuzzy modeling, vol 128. Studies in fuzziness and soft computing. Springer, HeidelbergGoogle Scholar
  10. Casillas J, Cordón O, Del Jesús MJ, Herrera F (2005) Genetic tuning of fuzzy rule deep structures preserving interpretability and its interaction with fuzzy rule set reduction. IEEE Trans Fuzzy Syst 13:13–29CrossRefGoogle Scholar
  11. Castro JL (1995) Fuzzy logic controllers are universal approximators. IEEE Trans Syst Man Cybern 25(4):629–635CrossRefGoogle Scholar
  12. Chen MY (2002) Establishing interpretable fuzzy models from numeric data. In: 4th world congress on intelligent control and automation IEEE, pp 1857–1861Google Scholar
  13. Cordón O, Herrera F (1997) A three-stage evolutionary process for learning descriptive and approximate fuzzy-logic-controller knowledge bases from examples. Int J Approx Reason 17(4):369–407MATHCrossRefGoogle Scholar
  14. Cordón O, Herrera F, Hoffmann F, Magdalena L (2001) Genetic fuzzy systems: evolutionary tuning and learning of fuzzy knowledge bases, vol 19. Advances in fuzzy systems—applications and theory. World Scientific Publishing Co. Pvt. Ltd., SingaporeGoogle Scholar
  15. Cordón O, Gomide F, Herrera F, Hoffmann F, Magdalena L (2004) Ten years of genetic fuzzy systems: current framework and new trends. Fuzzy Sets Syst 141:5–31MATHCrossRefGoogle Scholar
  16. De Oliveira JV (1999) Semantic constraints for membership function optimization. IEEE Trans Syst Man Cybern A Syst Hum 29(1):128–138CrossRefGoogle Scholar
  17. Gacto MJ, Alcalá R, Herrera F (2010) Integration of an index to preserve the semantic interpretability in the multi-objective evolutionary rule selection and tuning of linguistic fuzzy systems. IEEE Trans Fuzzy Syst. doi:10.1109/TFUZZ.2010.2041008
  18. Glorennec PY (1999) Algorithmes d’apprentissage pour systèmes d’inférence floue. Editions Hermès, ParisGoogle Scholar
  19. Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, New YorkMATHGoogle Scholar
  20. Guillaume S (2001) Designing fuzzy inference systems from data: an interpretability-oriented review. IEEE Trans Fuzzy Syst 9(3):426–443MathSciNetCrossRefGoogle Scholar
  21. Guillaume S, Charnomordic B (2004) Generating an interpretable family of fuzzy partitions. IEEE Trans Fuzzy Syst 12(3):324–335CrossRefGoogle Scholar
  22. Hartigan JA, Wong MA (1979) A k-means clustering algorithm. Appl Stat 28:100–108MATHCrossRefGoogle Scholar
  23. Herrera F (2008) Genetic fuzzy systems: taxonomy, current research trends and prospects. Evol Intell 1(1):27–46Google Scholar
  24. Hjorth JSU (1994) Computer intensive statistical methods validation, model selection, and bootstrap. Chapman & Hall, LondonMATHGoogle Scholar
  25. Hüllermeier E (2005) Fuzzy methods in machine learning and data mining: status and prospects. Fuzzy Sets Syst 156:387–406CrossRefGoogle Scholar
  26. Ichihashi H, Shirai T, Nagasaka K, Miyoshi T (1996) Neuro-fuzzy ID3: a method of inducing fuzzy decision trees with linear programming for maximizing entropy and an algebraic method for incremental learning. Fuzzy Sets Syst 81:157–167MathSciNetCrossRefGoogle Scholar
  27. Ishibuchi H, Nojima Y (2007) Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning. Int J Approx Reason 44:4–31MathSciNetMATHCrossRefGoogle Scholar
  28. Karr CL (1991) Genetic algorithms for fuzzy controllers. AI Expert 6(2):26–33Google Scholar
  29. Loquin K, Strauss O (2006) Fuzzy histograms and density estimation. Adv Soft Comput 6:45–52CrossRefGoogle Scholar
  30. Mamdani EH (1977) Application of fuzzy logic to approximate reasoning using linguistic synthesis. IEEE Trans Comput 26(12):1182–1191MATHCrossRefGoogle Scholar
  31. Mencar C, Fanelli AM (2008) Interpretability constraints for fuzzy information granulation. Inf Sci 178:4585–4618MathSciNetCrossRefGoogle Scholar
  32. Miller GA (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 101(2):343–352CrossRefGoogle Scholar
  33. Plutowski M, Sakata S, White H (1994) Cross-validation estimates IMSE. In: Cowan JD, Tesauro G, Alspector J (eds.) Advances in neural information processing systems 6. Morgan Kaufman, San Mateo, pp 391–398Google Scholar
  34. Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers, San MateoGoogle Scholar
  35. Quinlan JR (1996) Improved use of continuous attributes in C4.5. J Artif Intell Res 4:77–90MATHGoogle Scholar
  36. Ruspini EH (1969) A new approach to clustering. Inf Control 15(1):22–32MATHCrossRefGoogle Scholar
  37. Saaty TL, Ozdemir MS (2003) Why the magic number seven plus or minus two. Math Comput Model 38:233–244MathSciNetMATHCrossRefGoogle Scholar
  38. Solis FJ, Wets RJB (1981) Minimization by random search techniques. Math Oper Res 6:19–30MathSciNetMATHCrossRefGoogle Scholar
  39. Van Broekhoven E, Adriaenssens V, De Baets B (2007) Interpretability-preserving genetic optimization of linguistic terms in fuzzy models for fuzzy ordered classification: an ecological case study. Int J Approx Reason 44:65–90MathSciNetMATHCrossRefGoogle Scholar
  40. Wang LX, Mendel JM (1992) Generating fuzzy rules by learning from examples. IEEE Trans Syst Man Cybern 22(6):1414–1427MathSciNetCrossRefGoogle Scholar
  41. Zadeh LA (1975) The concept of a linguistic variable and its application to approximate reasoning. Part I. Inf Sci 8:199–249Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.European Centre for Soft Computing (ECSC)MieresSpain

Personalised recommendations