On the Usefulness of Fuzzy Rule Based Systems Based on Hierarchical Linguistic Fuzzy Partitions

  • Alberto Fernández
  • Victoria López
  • María José del Jesus
  • Francisco Herrera
Part of the Intelligent Systems Reference Library book series (ISRL, volume 13)


In the recent years, a high number of fuzzy rule learning algorithms have been developed with the aim of building the Knowledge Base of Linguistic Fuzzy Rule Based Systems. In this context, it emerges the necessity of managing a flexible structure of the Knowledge Base with the aim of modeling the problems with a higher precision. In this work, we present a short overview on the Hierarchical Fuzzy Rule Based Systems, which consists in a hierarchical extension of the Knowledge Base, preserving its structure and descriptive power and reinforcing the modeling of those problem subspaces with more difficulties by means of a hierarchical treatment (higher granularity) of the rules generated in these areas. Finally, this methodology includes a summarisation step by means of a genetic rule selection process in order to obtain a compact and accurate model. We will show the goodness of this methodology by means of a case of study in the framework of imbalanced data-sets in which we compare this learning scheme with some basic Fuzzy Rule Based Classification Systems and with the well-known C4.5 decision tree, using the proper statistical analysis as suggested in the specialised literature. Finally, we will develop a discussion on the usefulness of this methodology, analysing its advantages and proposing some new trends for future work on the topic in order to extract the highest potential of this technique for Fuzzy Rule Based Systems.


Fuzzy Rule Based Classification Systems Hierarchical Fuzzy Partitions Hierarchical Systems of Linguistic Rules Learning Methodology Granularity Imbalanced Data-sets 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Alcalá, R., Cano, J.R., Cordón, O., Herrera, F., Villar, P., Zwir, I.: Linguistic modeling with hierarchical systems of weighted linguistic rules. Int. J. Approx. Reason. 32(2–3), 187–215 (2003)MATHCrossRefGoogle Scholar
  2. Alcalá, R., Alcalá-Fdez, J., Herrera, F.: A proposal for the genetic lateral tuning of linguistic fuzzy systems and its interaction with rule selection. IEEE T. Fuzzy Syst. 15(4), 616–635 (2007)CrossRefGoogle Scholar
  3. Asuncion, A., Newman, D.: UCI machine learning repository (2007),
  4. Barandela, R., Sánchez, J.S., García, V., Rangel, E.: Strategies for learning in class imbalance problems. Pattern Recogn. 36(3), 849–851 (2003)CrossRefGoogle Scholar
  5. Bastian, A.: How to handle the flexibility of linguistic variables with applications. Int. J. Uncertain Fuzz 2(4), 463–484 (1994)MathSciNetCrossRefGoogle Scholar
  6. Batista, G., Prati, R.C., Monard, M.C.: A study of the behaviour of several methods for balancing machine learning training data. SIGKDD Explor. 6(1), 20–29 (2004)CrossRefGoogle Scholar
  7. Bonissone, P.P., Decker, K.: Selecting uncertainty calculi and granularity: An experiment in trading-off precision and complexity. In: Kanal, L.N., Lemmer, J.F. (eds.) UAI, pp. 217–248. Elsevier, Amsterdam (1985)Google Scholar
  8. Carse, B., Fogarty, T.C., Munro, A.: Evolving fuzzy rule based controllers using genetic algorithms. Fuzzy Set Syst. 80, 273–294 (1996)CrossRefGoogle Scholar
  9. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: Synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)MATHGoogle Scholar
  10. Chawla, N.V., Japkowicz, N., Kolcz, A.: Editorial: special issue on learning from imbalanced data sets. SIGKDD Explor. 6(1), 1–6 (2004)CrossRefGoogle Scholar
  11. Chi, Z., Yan, H., Pham, T.: Fuzzy algorithms with applications to image processing and pattern recognition. World Scientific, Singapore (1996)MATHGoogle Scholar
  12. Cordón, O., Herrera, F.: A proposal for improving the accuracy of linguistic modeling. IEEE T. Fuzzy Syst. 8(4), 335–344 (2000)CrossRefGoogle Scholar
  13. Cordón, O., del Jesus, M.J., Herrera, F.: A proposal on reasoning methods in fuzzy rule-based classification systems. Int. J. Approx Reason 20(1), 21–45 (1999)Google Scholar
  14. Cordón, O., Herrera, F., Hoffmann, F., Magdalena, L.: Genetic Fuzzy Systems. In: Evolutionary tuning and learning of fuzzy knowledge bases. World Scientific, Singapore (2001a)Google Scholar
  15. Cordón, O., Herrera, F., Villar, P.: Generating the knowledge base of a fuzzy rule-based system by the genetic learning of data base. IEEE T. Fuzzy Syst. 9(4), 667–674 (2001b)CrossRefGoogle Scholar
  16. Cordón, O., Herrera, F., Zwir, I.: Linguistic modeling by hierarchical systems of linguistic rules. IEEE T. Fuzzy Syst. 10(1), 2–20 (2002)CrossRefGoogle Scholar
  17. Cordón, O., Herrera, F., Zwir, I.: A hierarchical knowledge-based environment for linguistic modeling: Models and iterative methodology. Fuzzy Set Syst. 138(2), 307–341 (2003)CrossRefGoogle Scholar
  18. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetGoogle Scholar
  19. Domingos, P.: Metacost: a general method for making classifiers cost sensitive. In: Proceedings of the 5th International Conference on Knowledge Discovery and Data Mining, pp. 155–164 (1999)Google Scholar
  20. Estabrooks, A., Jo, T., Japkowicz, N.: A multiple resampling method for learning from imbalanced data sets. Comput. Intell. 20(1), 18–36 (2004)MathSciNetCrossRefGoogle Scholar
  21. Fernández, A., García, S., del Jesus, M.J., Herrera, F.: A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets. Fuzzy Set Syst. 159(18), 2378–2398 (2008)CrossRefGoogle Scholar
  22. Fernández, A., del Jesus, M., Herrera, F.: Hierarchical fuzzy rule based classification system with genetic rule selection for imbalanced data-set. Int. J. Approx. Reason. 50, 561–577 (2009)MATHCrossRefGoogle Scholar
  23. Gacto, M.J., Alcalá, R., Herrera, F.: Adaptation and application of multi-objective evolutionary algorithms for rule reduction and parameter tuning of fuzzy rule-based systems. Soft Comput. 13(5), 419–436 (2009)CrossRefGoogle Scholar
  24. García, S., Herrera, F.: An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J. Mach. Learn. Res. 9, 2677–2694 (2008)Google Scholar
  25. García, S., Fernández, A., Luengo, J., Herrera, F.: A study of statistical techniques and performance measures for genetics–based machine learning: Accuracy and interpretability. Soft Comput. 13(10), 959–977 (2009)CrossRefGoogle Scholar
  26. García, S., Fernández, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Inform. Sciences 180, 2044–2064 (2010)CrossRefGoogle Scholar
  27. García, V., Mollineda, R., Sánchez, J.S.: On the k-NN performance in a challenging scenario of imbalance and overlapping. Pattern Anal. Appl. 11(3–4), 269–280 (2008)Google Scholar
  28. He, H., Garcia, E.A.: Learning from imbalanced data. IEEE T. Knowl. Data En. 21(9), 1263–1284 (2009)CrossRefGoogle Scholar
  29. Herrera, F., Martínez, L.: A model based on linguistic 2-tuples for dealing with multigranular hierarchical linguistic contexts in multi-expert decision making. IEEE T. Syst. Man. CY B 31(2), 227–234 (2001)CrossRefGoogle Scholar
  30. Herrera, F., Herrera-Viedma, E., Martínez, L.: A fusion approach for managing multi-granularity linguistic terms sets in decision making. Fuzzy Set Syst. 114(1), 43–58 (2000)MATHCrossRefGoogle Scholar
  31. Holm, S.: A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65–70 (1979)MathSciNetMATHGoogle Scholar
  32. Ishibuchi, H., Nakashima, T.: Effect of rule weights in fuzzy rule-based classification systems. IEEE T. Fuzzy Syst. 9(4), 506–515 (2001)CrossRefGoogle Scholar
  33. Ishibuchi, H., Yamamoto, T.: Rule weight specification in fuzzy rule-based classification systems. IEEE T. Fuzzy Syst. 13, 428–435 (2005)CrossRefGoogle Scholar
  34. Ishibuchi, H., Nozaki, K., Tanaka, H.: Efficient fuzzy partition of pattern space for classification problems. Fuzzy Set Syst. 59, 295–304 (1993)CrossRefGoogle Scholar
  35. Ishibuchi, H., Nozaki, K., Yamamoto, N., Tanaka, H.: Selecting fuzzy if-then rules for classification problems using genetic algorithms. IEEE T. Fuzzy Syst. 9(3), 260–270 (1995)CrossRefGoogle Scholar
  36. Ishibuchi, H., Nakashima, T., Nii, M.: Classification and modeling with linguistic information granules: Advanced approaches to linguistic Data Mining. Springer, Heidelberg (2004)Google Scholar
  37. Kuncheva, L.: Fuzzy classifier design. Springer, Berlin (2000)MATHGoogle Scholar
  38. Magdalena, L., Monasterio-Huelin, F.: A fuzzy controller with learning through the evolution of its knowledge base. Int. J. Approx. Reason. 16(3), 335–358 (1997)MATHCrossRefGoogle Scholar
  39. Michalewicz, Z.: Genetic algorithms + data structures = evolution programs. Springer, Heidelberg (1996)MATHGoogle Scholar
  40. Orriols-Puig, A., Bernadó-Mansilla, E.: Evolutionary rule-based systems for imbalanced data-sets. Soft Comput. 13(3), 213–225 (2009)CrossRefGoogle Scholar
  41. Orriols-Puig, A., Bernadó-Mansilla, E., Goldberg, D.E., Sastry, K., Lanzi, P.L.: Facetwise analysis of XCS for problems with class imbalances. IEEE T. Evolut. Comput. 13, 260–283 (2009)CrossRefGoogle Scholar
  42. Palm, R., Driankov, D., Hellendoorn, H.: Model based fuzzy control. Springer, Berlin (1997)MATHGoogle Scholar
  43. Pedrycz, W.: Fuzzy modelling: paradigms and practice. Kluwer Academic Press, Dordrecht (1996)MATHGoogle Scholar
  44. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Mateo (1993)Google Scholar
  45. Peña Reyes, C.A., Sipper, M.: Fuzzy CoCo: a cooperative coevolutionary approach to fuzzy modeling. IEEE T. Fuzzy Syst. 9(5), 727–737 (2001)CrossRefGoogle Scholar
  46. Sanz, J., Fernández, A., Bustince, H., Herrera, F.: Improving the performance of fuzzy rule-based classification systems with interval-valued fuzzy sets and genetic amplitude tuning. Inform. Sciences 180(19), 3674–3685 (2010)CrossRefGoogle Scholar
  47. Sheskin, D.: Handbook of parametric and nonparametric statistical procedures, 2nd edn. Chapman & Hall/CRC (2006)Google Scholar
  48. Su, C.T., Hsiao, Y.H.: An evaluation of the robustness of MTS for imbalanced data. IEEE T. Knowl. Data En. 19(10), 1321–1332 (2007)CrossRefGoogle Scholar
  49. Su, C.T., Chen, L.S., Yih, Y.: Knowledge acquisition through information granulation for imbalanced data. Expert Syst. Appl. 31, 531–541 (2006)CrossRefGoogle Scholar
  50. Sun, Y., Kamel, M.S., Wong, A.K., Wang, Y.: Cost-sensitive boosting for classification of imbalanced data. Pattern Recogn. 40, 3358–3378 (2007)MATHCrossRefGoogle Scholar
  51. Sun, Y., Wong, A.K.C., Kamel, M.S.: Classification of imbalanced data: A review. Int. J. Pattern Recogn. 23(4), 687–719 (2009)CrossRefGoogle Scholar
  52. Thrift, P.: Fuzzy logic synthesis with genetic algorithms. In: 4th International Conference on Genetic Algorithms (ICGA 1991), pp. 509–513. Morgan Kaufmann, San Mateo (1991)Google Scholar
  53. Wang, L.X., Mendel, J.M.: Generating fuzzy rules by learning from examples. IEEE T. Syst. Man CYB 25(2), 353–361 (1992)Google Scholar
  54. Weiss, G., Provost, F.: Learning when training data are costly: The effect of class distribution on tree induction. J.Artif. Intell. Res. 19, 315–354 (2003)MATHGoogle Scholar
  55. Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics 1, 80–83 (1945)CrossRefGoogle Scholar
  56. Wu, G., Chang, E.: KBA: Kernel boundary alignment considering imbalanced data distribution. IEEE T. Knowl. Data En. 17(6), 786–795 (2005)CrossRefGoogle Scholar
  57. Xu, L., Chow, M.Y., Taylor, L.S.: Power distribution fault cause identification with imbalanced data using the data mining-based fuzzy classification e-algorithm. IEEE T. Power Syst. 22(1), 164–171 (2007)MATHCrossRefGoogle Scholar
  58. Yager, R.R., Filev, D.: Essentials of Fuzzy Modeling and Control. John Wiley & Sons, Chichester (1994)Google Scholar
  59. Yang, Q., Wu, X.: 10 challenging problems in data mining research. Int. J. Inf. Tech. Decis. 5(4), 597–604 (2006)CrossRefGoogle Scholar
  60. Yen, J., Wang, L., Gillespie, C.W.: Improving the interpretability of TSK fuzzy models by combining global learning and local learning. IEEE T. Fuzzy Syst. 6(4), 530–537 (1998)CrossRefGoogle Scholar
  61. Zadeh, L.A.: The concept of a linguistic variable and its application to approximate reasoning. Inform. Sciences 8(3), 199–249 (1975)MathSciNetCrossRefGoogle Scholar
  62. Zhou, Z., Liu, X.: Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE T. Knowl Data En. 18(1), 63–77 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Alberto Fernández
    • 1
  • Victoria López
    • 2
  • María José del Jesus
    • 1
  • Francisco Herrera
    • 2
  1. 1.Dept. of Computer ScienceUniversity of JaénSpain
  2. 2.Dept. of Computer Science and A.I.University of GranadaSpain

Personalised recommendations