Advertisement

Soft Computing

, Volume 16, Issue 3, pp 451–470 | Cite as

Complexity reduction and interpretability improvement for fuzzy rule systems based on simple interpretability measures and indices by bi-objective evolutionary rule selection

  • Marta Galende-Hernández
  • Gregorio I. Sainz-Palmero
  • Maria J. Fuente-Aparicio
Original Paper

Abstract

The aim of this paper is to develop a general post-processing methodology to reduce the complexity of data-driven linguistic fuzzy models, in order to reach simpler fuzzy models preserving enough accuracy and better fuzzy linguistic performance with respect to their initial values. This post-processing approach is based on rule selection via the formulation of a bi-objective problem with one objective focusing on accuracy and the other on interpretability. The latter is defined via the aggregation of several interpretability measures, based on the concepts of similarity and complexity of fuzzy systems and rules. In this way, a measure of the fuzzy model interpretability is given. Two neuro-fuzzy systems for providing initial fuzzy models, Fuzzy Adaptive System ART based and Neuro-Fuzzy Function Approximation and several case studies, data sets from KEEL Project Repository, are used to check this approach. Both fuzzy and neuro-fuzzy systems generate Mamdani-type fuzzy rule-based systems, each with its own particularities and complexities from the point of view of the fuzzy sets and the rule generation. Based on these systems and data sets, several fuzzy models are generated to check the performance of the proposal under different restrictions of complexity and fuzziness.

Keywords

Fuzzy modeling Accuracy Interpretability Complexity Genetic algorithms 

Notes

Acknowledgments

The authors would like to thank Francisco Herrera and the reviewers for their valuable and useful comments and support in the preparation of this manuscript. This work was supported by the Spanish Ministry of Science and Innovation under Grants no. CIT-460000-2009-46 and DPI2009-14410-C02-02.

References

  1. Alcalá R, Alcalá-Fdez J, Casillas J, Cordón O, Herrera F (2006) Hybrid learning models to get the interpretability-accuracy trade-off in fuzzy modeling. Soft Comput 10(9):717–734CrossRefGoogle Scholar
  2. Alcalá R, Alcalá-Fdez J, Herrera F, Otero J (2007a) Genetic learning of accurate and compact fuzzy rule based systems based on the 2-tuples linguistic representation. Int J Approx Reason 44:45–64zbMATHCrossRefGoogle Scholar
  3. Alcalá R, Gacto MJ, Herrera F, Alcalá-Fdez J (2007b) A multi-objective genetic algorithm for tuning and rule selection to obtain accurate and compact linguistic fuzzy rule-based systems. Int J Uncertain Fuzziness Knowl Based Syst 15(5):539–557zbMATHCrossRefGoogle Scholar
  4. Alcalá R, Ducange P, Herrera F, Lazzerini B, Marcelloni F (2009) A multiobjective evolutionary approach to concurrently learn rule and data bases of linguistic fuzzy-rule-based systems. IEEE Trans Fuzzy Syst 17(5):1106–1122CrossRefGoogle Scholar
  5. Alcalá R, Nojima Y, Herrera F, Ishibuchi H (2011) Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions. Soft Comput. doi: 10.1007/s00500-010-0671-2
  6. Alcalá-Fdez J, Sánchez L, García S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM, Fernndez JC, Herrera F (2009) KEEL: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput Fusion Found Methodol Appl 13(3):307–318CrossRefGoogle Scholar
  7. Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J Multiple Valued Logic Soft Comput 17:2–3 255–287Google Scholar
  8. Alonso JM, Magdalena L, González-Rodríguez G (2009) Looking for a good fuzzy system interpretability index: an experimental approach. Int J Approx Reason 51(1):115–134CrossRefGoogle Scholar
  9. Alonso JM, Magdalena L (2010) HILK++: an interpretability-guided fuzzy modeling methodology for learning readable and comprehensible fuzzy rule-based classifiers. Soft Comput Fusion Found Methodol Appl (online first)Google Scholar
  10. Bonissoene PP, Chen Y-T, Goebel K, Khedkar PS (1999) Hybrid soft computing systems: industrial and commercial applications. Proc IEEE 87(9):1641–1667CrossRefGoogle Scholar
  11. Botta A, Lazzerini B, Marcelloni F, Stefanescu DC (2009) Context adaptation of fuzzy systems through a multi-objective evolutionary approach based on a novel interpretability index. Soft Comput 13(5):437–449CrossRefGoogle Scholar
  12. Cano Izquierdo JM, Dimitriadis YA, Gómez Sánchez E, López Coronado J (2001) Learning from noisy information in FasArt and Fasback neuro-fuzzy systems. Neural Netw 14(4–5):407–425CrossRefGoogle Scholar
  13. Casillas J, Cordón O, Herrera F, Magdalena L (eds) (2003a) Accuracy improvements in linguistic fuzzy ,modelling. Studies in fuzziness and soft computing, vol 129. Springer, BerlinGoogle Scholar
  14. Casillas J, Cordón O, Herrera F, Magdalena L (eds) (2003b) Interpretability Issues in fuzzy modeling. Studies in fuzziness and soft computing, vol 128. Springer, BerlinGoogle Scholar
  15. Chen MY, Linkens DA (2004) Rule-base self-generation and simplification for data-driven fuzzy models. Fuzzy Sets Syst 142(2):265–265MathSciNetCrossRefGoogle Scholar
  16. Cococcioni M, Ducange P, Lazzerini B, Marcelloni F (2007) A Pareto-based multi-objective evolutionary approach to the identification of Mamdani fuzzy systems. Soft Comput 11:1013–1031CrossRefGoogle Scholar
  17. Cordón O, Herrera F, Hoffmann F, Magdalena L (2001) Genetic fuzzy systems: evolutionary tuning and learning of fuzzy knowledge bases. Advances in fuzzy systems—applications and theory. World Scientific, SingaporeGoogle Scholar
  18. Cpalka K (2009) A new method for design and reduction of neuro-fuzzy classification systems. IEEE Trans Neural Netw 20(4):701–714CrossRefGoogle Scholar
  19. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197CrossRefGoogle Scholar
  20. Delgado MR, Von Zuben F, Gomide F (2003) Hierarchical genetic fuzzy systems: accuracy, interpretability and design autonomy. In: Interpretability Issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 379–405Google Scholar
  21. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetzbMATHGoogle Scholar
  22. Destercke S, Guillaume S, Charnomordic B (2007) Building an interpretable fuzzy rule base from data using orthogonal least squares-application to a depollution problem. Fuzzy Sets Syst 158(18):2078–2094MathSciNetzbMATHCrossRefGoogle Scholar
  23. Eshelman LJ (1991) The CHC adaptive search algorithm: how to have safe search when engaging in nontraditional genetic recombination. In: Foundations of genetic algorithms 1. Morgan Kaufmann, San Mateo, CA, pp 265–283Google Scholar
  24. Espinosa J, Vandewalle J (2000) Constructing fuzzy models with linguistic integrity from numerical data-AFRELI algorithm. IEEE Trans Fuzzy Syst 8(5):591–600CrossRefGoogle Scholar
  25. Fiordaliso A (2003) About the trade-off between accuracy and interpretability of Takagi-Sugeno models in the context of nonlinear time series forecasting. In: Interpretability issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 406–430Google Scholar
  26. Gacto MJ, Alcalá R, Herrera F (2009) Adaptation and application of multi-objective evolutionary algorithms for rule reduction and parameter tuning of fuzzy rule-based systems. Soft Comput Fusion Found Methodol Appl 13(5):419–436CrossRefGoogle Scholar
  27. Gacto MJ, Alcalá R, Herrera F (2010) Integration of an index to preserve the semantic interpretability in the multi-objective evolutionary rule selection and tuning of linguistic fuzzy systems. IEEE Trans Fuzzy Syst 18(3):515–531CrossRefGoogle Scholar
  28. Gacto MJ, Alcalá R, Herrera F (2011) Interpretability of linguistic fuzzy rule-based systems: an overview of interpretability measures. Inf Sci 181:4340–4360Google Scholar
  29. Galende M, Sainz GI, Fuente MJ, Herreros A (2008) Interpretability-accuracy improvement in a neuro-fuzzy ART based model of a DC motor. In: Proceedings of the 17th IFAC world congress, Seoul, Korea, 6–11 July 2008, pp 7034–7039Google Scholar
  30. Galende M, Sainz GI, Fuente MJ (2009) Accuracy-interpretability balancing in fuzzy models based on multiobjective genetic algorithm. In: Proceedings of European control conference 2009 (ECC’09), Budapest, Hungary, 23–26 August 2009, pp 3915–3920Google Scholar
  31. García S, Herrera F (2008) An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694zbMATHGoogle Scholar
  32. García S, Fernández A, Luengo J, Herrera F (2009) A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput 13(10):959–977CrossRefGoogle Scholar
  33. García S, Molina D, Lozano M, Herrera F (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC 2005 special session on real parameter optimization. J Heuristics 15:617–644zbMATHCrossRefGoogle Scholar
  34. Gómez-Sánchez E, Dimitriadis YA, Cano-Izquierdo JM, López-Coronado J (2002) μARTMAP: use of mutual information for category reduction in fuzzy ARTMAP. IEEE Trans Neural Netw 13(1):58–69CrossRefGoogle Scholar
  35. González J, Rojas I, Pomares H, Herrera LJ, Guillén A, Palomares JM, Rojas F (2007) Improving the accuracy while preserving the interpretability of fuzzy function approximators by means of multi-objective evolutionary algorithms. Int J Approx Reason 44:32–44zbMATHCrossRefGoogle Scholar
  36. Guillaume S, Charnomordic B (2003) A new method for inducing a set of interpretable fuzzy partitions and fuzzy inference systems from data. In: Interpretability issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 148–175Google Scholar
  37. Herrera F (2008) Genetic fuzzy systems: taxonomy, current research trends and prospects. Evol Intel 1:27–46CrossRefGoogle Scholar
  38. Ishibuchi H, Nojima Y (2009) Discussions on interpretability of fuzzy systems using simple examples. In: Proceedings of 13th IFSA world congress and 6th conference of EUSFLAT, pp 1649–1654Google Scholar
  39. Ishibuchi H, Nojima Y (2007) Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning. Int J Approx Reason 44(1):4–31MathSciNetzbMATHCrossRefGoogle Scholar
  40. Ishibuchi H, Yamamoto T (2004) Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining. Fuzzy Sets Syst 141(1):59–88MathSciNetzbMATHCrossRefGoogle Scholar
  41. Ishibuchi H, Nozaki K, Yamamoto N, Tanaka H (1995) Selecting fuzzy if-then rules for classification problems using genetic algorithms. IEEE Trans Fuzzy Syst 3(3):260–270CrossRefGoogle Scholar
  42. Ishibuchi H, Murata T, Türksen IB (1997) Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems. Fuzzy Sets Syst 89(2):135–150CrossRefGoogle Scholar
  43. Ishibuchi H, Nakashima T, Murata T (2001) Three-objective genetics-based machine learning for linguistic rule extraction. Inf Sci 136(1–4):109–133zbMATHCrossRefGoogle Scholar
  44. Ishibuchi H, Kaisho Y, Nojima Y (2009a) Complexity, interpretability and explanation capability of fuzzy rule-based classifiers. In: IEEE international conference on fuzzy systems, 2009. FUZZ-IEEE 2009, 20–24 August 2009, pp 1730–1735Google Scholar
  45. Ishibuchi H, Nakashima Y, Nojima Y (2009b) Search ability of evolutionary multiobjective optimization algorithms for multiobjective fuzzy genetics-based machine learning. In: IEEE international conference on fuzzy systems, 2009. FUZZ-IEEE 2009, 20–24 August 2009, pp 1724–1729Google Scholar
  46. Jimenez F, Gómez-Skarmeta AF, Sanchez G, Roubos H, Babuška R (2003) Accurate, transparent and compact fuzzy models by multi-objective evolutionary algorithms. In: Interpretability Issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 431–451Google Scholar
  47. Jin Y (2000) Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement. IEEE Trans Fuzzy Syst 8(2):212–221CrossRefGoogle Scholar
  48. Jin Y, Von Seelen W, Sendhoff B (1999) On generating FC 3 fuzzy rule systems from data using evolution strategies. IEEE Trans Syst Man Cybern Part B Cybern 29(6):829–845CrossRefGoogle Scholar
  49. Karray FO, de De Silva C (2004) Soft computing and intelligent systems design. Tools and applications. Addison-Wesley, ReadingGoogle Scholar
  50. Konar A (2005) Computational intelligence: principles, techniques and applications. Springer, BerlinzbMATHGoogle Scholar
  51. Mencar C, Fanelli A (2008) Interpretability constraints for fuzzy information granulation. Inf Sci 178(24):4585–4618MathSciNetCrossRefGoogle Scholar
  52. Mikut R, Jäkel J, Gröll L (2005) Interpretability issues in data-based learning of fuzzy systems. Fuzzy Sets Syst 150(2):179–197zbMATHCrossRefGoogle Scholar
  53. Nauck D, Kruse R (1999) Neuro-fuzzy systems for function approximation. Fuzzy Sets Syst 101(2):261–271zbMATHCrossRefGoogle Scholar
  54. Nojima Y, Ishibuchi H (2009) Incorporation of user preference into multi-objective genetic fuzzy rule selection for pattern classifi cation problems. Artif Life Robot 14(3):418–421Google Scholar
  55. Parrado-Hernández E, Gómez-Sánchez E, Dimitriadis YA (2003) Study of distributed learning as a solution to category proliferation in fuzzy ARTMAP based neural systems. Neural Netw 16(7):1039–1057CrossRefGoogle Scholar
  56. Pulkkinen P, Koivisto H (2008) Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms. Int J Approx Reason 48(2):526–543CrossRefGoogle Scholar
  57. Pulkkinen P, Koivisto H (2010) A dynamically constrained multiobjective genetic fuzzy system for regression problems. IEEE Trans Fuzzy Syst 18(1):161–177CrossRefGoogle Scholar
  58. Roubos H, Setnes M (2001) Compact and transparent fuzzy models and classfiers through iterative complexity reduction. IEEE Trans Fuzzy Syst 9(4):516–524CrossRefGoogle Scholar
  59. Sainz Palmero GI, Dimitriadis YA, Cano Izquierdo JM, Gómez Sánchez E, Parrado Hernández E (2000) ART based model set for pattern recognition: FasArt family. In: Bunke H, Kandel A (eds) Neuro-fuzzy pattern recognition, chap 1. World Scientific, Singapore, pp 147–177Google Scholar
  60. Sainz Palmero GI, Juez Santamaria J, Moya de la Torre EJ, Perán González JR (2005) Fault detection and fuzzy rule extraction in AC motors by a neuro-fuzzy ART-based system. Eng Appl Artif Intell 18:867–874Google Scholar
  61. Sainz GI, Fuente MJ, Vega P (2004) Recurrent neuro-fuzzy modelling of a wastewater treatment plant. Eur J Control 10:83–95CrossRefGoogle Scholar
  62. Setnes M (2003) Simplification and reduction of fuzzy rules. In: Interpretability issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 278–302Google Scholar
  63. Setnes M, Babuška R (2001) Rule base reduction: some comments on the use of orthogonal transforms. IEEE Trans Syst Man Cybern Part C Appl Rev 31(2):199–206CrossRefGoogle Scholar
  64. Setnes M, Babuška R, Kaymak U, van Nauta Lemke HR (1998) Similarity measures in fuzzy rule base simplification. IEEE Trans Syst Man Cybern Part B Cybern 28(3):376–386CrossRefGoogle Scholar
  65. Sheskin DJ (2003) Handbook of parametric and nonparametric statistical procedures. Chapman & Hall/CRC, LondonGoogle Scholar
  66. Suzuki T, Furuhashi T (2003) Conciseness of fuzzy models. In: Interpretability issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 569–586Google Scholar
  67. Wang L-X, Mendel JM (1992) Generating fuzzy rules by learning from examples. IEEE Trans Syst Man Cybern 22(6):1414–1427MathSciNetCrossRefGoogle Scholar
  68. Yen J, Wang L (1999) Simplifying fuzzy rule-based models using orthogonal transformation methods. IEEE Trans Syst Man Cybern Part B Cybern 29(1):13–24CrossRefGoogle Scholar
  69. Zar JH (1999) Biostatistical analysis. Prentice-Hall, Englewood CliffsGoogle Scholar
  70. Zhou S-M, Gan JQ (2008) Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling. Fuzzy Sets Syst 159:3091–3131MathSciNetCrossRefGoogle Scholar
  71. Zong-Yi X, Li-Min J, Yong Z, Wei-Li H, Yong Q (2005) A case study of data-driven interpretable fuzzy modeling. Acta Autom Sin 31(6):815–824Google Scholar
  72. Zong-Yi X, Yong Z, Yuan-Long H, Guo-Qiang C (2008) Multi-objective fuzzy modeling using NSGA-II. In: IEEE conference on cybernetics and intelligent systems, 21–24 September 2008, pp 119–124Google Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Marta Galende-Hernández
    • 1
  • Gregorio I. Sainz-Palmero
    • 1
    • 2
  • Maria J. Fuente-Aparicio
    • 2
  1. 1.CARTIF Centro TecnológicoBoecilloSpain
  2. 2.Department of Systems Engineering and Control, School of Industrial EngineeringUniversity of ValladolidValladolidSpain

Personalised recommendations