Skip to main content

Advertisement

SpringerLink
Log in

Complexity reduction and interpretability improvement for fuzzy rule systems based on simple interpretability measures and indices by bi-objective evolutionary rule selection

  • Original Paper
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The aim of this paper is to develop a general post-processing methodology to reduce the complexity of data-driven linguistic fuzzy models, in order to reach simpler fuzzy models preserving enough accuracy and better fuzzy linguistic performance with respect to their initial values. This post-processing approach is based on rule selection via the formulation of a bi-objective problem with one objective focusing on accuracy and the other on interpretability. The latter is defined via the aggregation of several interpretability measures, based on the concepts of similarity and complexity of fuzzy systems and rules. In this way, a measure of the fuzzy model interpretability is given. Two neuro-fuzzy systems for providing initial fuzzy models, Fuzzy Adaptive System ART based and Neuro-Fuzzy Function Approximation and several case studies, data sets from KEEL Project Repository, are used to check this approach. Both fuzzy and neuro-fuzzy systems generate Mamdani-type fuzzy rule-based systems, each with its own particularities and complexities from the point of view of the fuzzy sets and the rule generation. Based on these systems and data sets, several fuzzy models are generated to check the performance of the proposal under different restrictions of complexity and fuzziness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. A complete list of papers in this area is available from the web page http://www.iet.unipi.it/m.cococcioni/emofrbss.html.

  2. http://fuzzy.cs.uni-magdeburg.de/nefprox/.

  3. http://www.iitk.ac.in/kangal/codes.shtml.

  4. http://sci2s.ugr.es/keel/datasets.php.

  5. http://sci2s.ugr.es/sicidm.

References

  • Alcalá R, Alcalá-Fdez J, Casillas J, Cordón O, Herrera F (2006) Hybrid learning models to get the interpretability-accuracy trade-off in fuzzy modeling. Soft Comput 10(9):717–734

    Article  Google Scholar 

  • Alcalá R, Alcalá-Fdez J, Herrera F, Otero J (2007a) Genetic learning of accurate and compact fuzzy rule based systems based on the 2-tuples linguistic representation. Int J Approx Reason 44:45–64

    Article  MATH  Google Scholar 

  • Alcalá R, Gacto MJ, Herrera F, Alcalá-Fdez J (2007b) A multi-objective genetic algorithm for tuning and rule selection to obtain accurate and compact linguistic fuzzy rule-based systems. Int J Uncertain Fuzziness Knowl Based Syst 15(5):539–557

    Article  MATH  Google Scholar 

  • Alcalá R, Ducange P, Herrera F, Lazzerini B, Marcelloni F (2009) A multiobjective evolutionary approach to concurrently learn rule and data bases of linguistic fuzzy-rule-based systems. IEEE Trans Fuzzy Syst 17(5):1106–1122

    Article  Google Scholar 

  • Alcalá R, Nojima Y, Herrera F, Ishibuchi H (2011) Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions. Soft Comput. doi:10.1007/s00500-010-0671-2

  • Alcalá-Fdez J, Sánchez L, García S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM, Fernndez JC, Herrera F (2009) KEEL: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput Fusion Found Methodol Appl 13(3):307–318

    Article  Google Scholar 

  • Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J Multiple Valued Logic Soft Comput 17:2–3 255–287

    Google Scholar 

  • Alonso JM, Magdalena L, González-Rodríguez G (2009) Looking for a good fuzzy system interpretability index: an experimental approach. Int J Approx Reason 51(1):115–134

    Article  Google Scholar 

  • Alonso JM, Magdalena L (2010) HILK++: an interpretability-guided fuzzy modeling methodology for learning readable and comprehensible fuzzy rule-based classifiers. Soft Comput Fusion Found Methodol Appl (online first)

  • Bonissoene PP, Chen Y-T, Goebel K, Khedkar PS (1999) Hybrid soft computing systems: industrial and commercial applications. Proc IEEE 87(9):1641–1667

    Article  Google Scholar 

  • Botta A, Lazzerini B, Marcelloni F, Stefanescu DC (2009) Context adaptation of fuzzy systems through a multi-objective evolutionary approach based on a novel interpretability index. Soft Comput 13(5):437–449

    Article  Google Scholar 

  • Cano Izquierdo JM, Dimitriadis YA, Gómez Sánchez E, López Coronado J (2001) Learning from noisy information in FasArt and Fasback neuro-fuzzy systems. Neural Netw 14(4–5):407–425

    Article  Google Scholar 

  • Casillas J, Cordón O, Herrera F, Magdalena L (eds) (2003a) Accuracy improvements in linguistic fuzzy ,modelling. Studies in fuzziness and soft computing, vol 129. Springer, Berlin

  • Casillas J, Cordón O, Herrera F, Magdalena L (eds) (2003b) Interpretability Issues in fuzzy modeling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin

  • Chen MY, Linkens DA (2004) Rule-base self-generation and simplification for data-driven fuzzy models. Fuzzy Sets Syst 142(2):265–265

    Article  MathSciNet  Google Scholar 

  • Cococcioni M, Ducange P, Lazzerini B, Marcelloni F (2007) A Pareto-based multi-objective evolutionary approach to the identification of Mamdani fuzzy systems. Soft Comput 11:1013–1031

    Article  Google Scholar 

  • Cordón O, Herrera F, Hoffmann F, Magdalena L (2001) Genetic fuzzy systems: evolutionary tuning and learning of fuzzy knowledge bases. Advances in fuzzy systems—applications and theory. World Scientific, Singapore

  • Cpalka K (2009) A new method for design and reduction of neuro-fuzzy classification systems. IEEE Trans Neural Netw 20(4):701–714

    Article  Google Scholar 

  • Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197

    Article  Google Scholar 

  • Delgado MR, Von Zuben F, Gomide F (2003) Hierarchical genetic fuzzy systems: accuracy, interpretability and design autonomy. In: Interpretability Issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 379–405

  • Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  • Destercke S, Guillaume S, Charnomordic B (2007) Building an interpretable fuzzy rule base from data using orthogonal least squares-application to a depollution problem. Fuzzy Sets Syst 158(18):2078–2094

    Article  MathSciNet  MATH  Google Scholar 

  • Eshelman LJ (1991) The CHC adaptive search algorithm: how to have safe search when engaging in nontraditional genetic recombination. In: Foundations of genetic algorithms 1. Morgan Kaufmann, San Mateo, CA, pp 265–283

  • Espinosa J, Vandewalle J (2000) Constructing fuzzy models with linguistic integrity from numerical data-AFRELI algorithm. IEEE Trans Fuzzy Syst 8(5):591–600

    Article  Google Scholar 

  • Fiordaliso A (2003) About the trade-off between accuracy and interpretability of Takagi-Sugeno models in the context of nonlinear time series forecasting. In: Interpretability issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 406–430

  • Gacto MJ, Alcalá R, Herrera F (2009) Adaptation and application of multi-objective evolutionary algorithms for rule reduction and parameter tuning of fuzzy rule-based systems. Soft Comput Fusion Found Methodol Appl 13(5):419–436

    Article  Google Scholar 

  • Gacto MJ, Alcalá R, Herrera F (2010) Integration of an index to preserve the semantic interpretability in the multi-objective evolutionary rule selection and tuning of linguistic fuzzy systems. IEEE Trans Fuzzy Syst 18(3):515–531

    Article  Google Scholar 

  • Gacto MJ, Alcalá R, Herrera F (2011) Interpretability of linguistic fuzzy rule-based systems: an overview of interpretability measures. Inf Sci 181:4340–4360

    Google Scholar 

  • Galende M, Sainz GI, Fuente MJ, Herreros A (2008) Interpretability-accuracy improvement in a neuro-fuzzy ART based model of a DC motor. In: Proceedings of the 17th IFAC world congress, Seoul, Korea, 6–11 July 2008, pp 7034–7039

  • Galende M, Sainz GI, Fuente MJ (2009) Accuracy-interpretability balancing in fuzzy models based on multiobjective genetic algorithm. In: Proceedings of European control conference 2009 (ECC’09), Budapest, Hungary, 23–26 August 2009, pp 3915–3920

  • García S, Herrera F (2008) An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694

    MATH  Google Scholar 

  • García S, Fernández A, Luengo J, Herrera F (2009) A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput 13(10):959–977

    Article  Google Scholar 

  • García S, Molina D, Lozano M, Herrera F (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC 2005 special session on real parameter optimization. J Heuristics 15:617–644

    Article  MATH  Google Scholar 

  • Gómez-Sánchez E, Dimitriadis YA, Cano-Izquierdo JM, López-Coronado J (2002) μARTMAP: use of mutual information for category reduction in fuzzy ARTMAP. IEEE Trans Neural Netw 13(1):58–69

    Article  Google Scholar 

  • González J, Rojas I, Pomares H, Herrera LJ, Guillén A, Palomares JM, Rojas F (2007) Improving the accuracy while preserving the interpretability of fuzzy function approximators by means of multi-objective evolutionary algorithms. Int J Approx Reason 44:32–44

    Article  MATH  Google Scholar 

  • Guillaume S, Charnomordic B (2003) A new method for inducing a set of interpretable fuzzy partitions and fuzzy inference systems from data. In: Interpretability issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 148–175

  • Herrera F (2008) Genetic fuzzy systems: taxonomy, current research trends and prospects. Evol Intel 1:27–46

    Article  Google Scholar 

  • Ishibuchi H, Nojima Y (2009) Discussions on interpretability of fuzzy systems using simple examples. In: Proceedings of 13th IFSA world congress and 6th conference of EUSFLAT, pp 1649–1654

  • Ishibuchi H, Nojima Y (2007) Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning. Int J Approx Reason 44(1):4–31

    Article  MathSciNet  MATH  Google Scholar 

  • Ishibuchi H, Yamamoto T (2004) Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining. Fuzzy Sets Syst 141(1):59–88

    Article  MathSciNet  MATH  Google Scholar 

  • Ishibuchi H, Nozaki K, Yamamoto N, Tanaka H (1995) Selecting fuzzy if-then rules for classification problems using genetic algorithms. IEEE Trans Fuzzy Syst 3(3):260–270

    Article  Google Scholar 

  • Ishibuchi H, Murata T, Türksen IB (1997) Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems. Fuzzy Sets Syst 89(2):135–150

    Article  Google Scholar 

  • Ishibuchi H, Nakashima T, Murata T (2001) Three-objective genetics-based machine learning for linguistic rule extraction. Inf Sci 136(1–4):109–133

    Article  MATH  Google Scholar 

  • Ishibuchi H, Kaisho Y, Nojima Y (2009a) Complexity, interpretability and explanation capability of fuzzy rule-based classifiers. In: IEEE international conference on fuzzy systems, 2009. FUZZ-IEEE 2009, 20–24 August 2009, pp 1730–1735

  • Ishibuchi H, Nakashima Y, Nojima Y (2009b) Search ability of evolutionary multiobjective optimization algorithms for multiobjective fuzzy genetics-based machine learning. In: IEEE international conference on fuzzy systems, 2009. FUZZ-IEEE 2009, 20–24 August 2009, pp 1724–1729

  • Jimenez F, Gómez-Skarmeta AF, Sanchez G, Roubos H, Babuška R (2003) Accurate, transparent and compact fuzzy models by multi-objective evolutionary algorithms. In: Interpretability Issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 431–451

  • Jin Y (2000) Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement. IEEE Trans Fuzzy Syst 8(2):212–221

    Article  Google Scholar 

  • Jin Y, Von Seelen W, Sendhoff B (1999) On generating FC 3 fuzzy rule systems from data using evolution strategies. IEEE Trans Syst Man Cybern Part B Cybern 29(6):829–845

    Article  Google Scholar 

  • Karray FO, de De Silva C (2004) Soft computing and intelligent systems design. Tools and applications. Addison-Wesley, Reading

    Google Scholar 

  • Konar A (2005) Computational intelligence: principles, techniques and applications. Springer, Berlin

    MATH  Google Scholar 

  • Mencar C, Fanelli A (2008) Interpretability constraints for fuzzy information granulation. Inf Sci 178(24):4585–4618

    Article  MathSciNet  Google Scholar 

  • Mikut R, Jäkel J, Gröll L (2005) Interpretability issues in data-based learning of fuzzy systems. Fuzzy Sets Syst 150(2):179–197

    Article  MATH  Google Scholar 

  • Nauck D, Kruse R (1999) Neuro-fuzzy systems for function approximation. Fuzzy Sets Syst 101(2):261–271

    Article  MATH  Google Scholar 

  • Nojima Y, Ishibuchi H (2009) Incorporation of user preference into multi-objective genetic fuzzy rule selection for pattern classifi cation problems. Artif Life Robot 14(3):418–421

    Google Scholar 

  • Parrado-Hernández E, Gómez-Sánchez E, Dimitriadis YA (2003) Study of distributed learning as a solution to category proliferation in fuzzy ARTMAP based neural systems. Neural Netw 16(7):1039–1057

    Article  Google Scholar 

  • Pulkkinen P, Koivisto H (2008) Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms. Int J Approx Reason 48(2):526–543

    Article  Google Scholar 

  • Pulkkinen P, Koivisto H (2010) A dynamically constrained multiobjective genetic fuzzy system for regression problems. IEEE Trans Fuzzy Syst 18(1):161–177

    Article  Google Scholar 

  • Roubos H, Setnes M (2001) Compact and transparent fuzzy models and classfiers through iterative complexity reduction. IEEE Trans Fuzzy Syst 9(4):516–524

    Article  Google Scholar 

  • Sainz Palmero GI, Dimitriadis YA, Cano Izquierdo JM, Gómez Sánchez E, Parrado Hernández E (2000) ART based model set for pattern recognition: FasArt family. In: Bunke H, Kandel A (eds) Neuro-fuzzy pattern recognition, chap 1. World Scientific, Singapore, pp 147–177

  • Sainz Palmero GI, Juez Santamaria J, Moya de la Torre EJ, Perán González JR (2005) Fault detection and fuzzy rule extraction in AC motors by a neuro-fuzzy ART-based system. Eng Appl Artif Intell 18:867–874

  • Sainz GI, Fuente MJ, Vega P (2004) Recurrent neuro-fuzzy modelling of a wastewater treatment plant. Eur J Control 10:83–95

    Article  Google Scholar 

  • Setnes M (2003) Simplification and reduction of fuzzy rules. In: Interpretability issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 278–302

  • Setnes M, Babuška R (2001) Rule base reduction: some comments on the use of orthogonal transforms. IEEE Trans Syst Man Cybern Part C Appl Rev 31(2):199–206

    Article  Google Scholar 

  • Setnes M, Babuška R, Kaymak U, van Nauta Lemke HR (1998) Similarity measures in fuzzy rule base simplification. IEEE Trans Syst Man Cybern Part B Cybern 28(3):376–386

    Article  Google Scholar 

  • Sheskin DJ (2003) Handbook of parametric and nonparametric statistical procedures. Chapman & Hall/CRC, London

  • Suzuki T, Furuhashi T (2003) Conciseness of fuzzy models. In: Interpretability issues in fuzzy modelling. Studies in fuzziness and soft computing, vol 128. Springer, Berlin, pp 569–586

  • Wang L-X, Mendel JM (1992) Generating fuzzy rules by learning from examples. IEEE Trans Syst Man Cybern 22(6):1414–1427

    Article  MathSciNet  Google Scholar 

  • Yen J, Wang L (1999) Simplifying fuzzy rule-based models using orthogonal transformation methods. IEEE Trans Syst Man Cybern Part B Cybern 29(1):13–24

    Article  Google Scholar 

  • Zar JH (1999) Biostatistical analysis. Prentice-Hall, Englewood Cliffs

  • Zhou S-M, Gan JQ (2008) Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling. Fuzzy Sets Syst 159:3091–3131

    Article  MathSciNet  Google Scholar 

  • Zong-Yi X, Li-Min J, Yong Z, Wei-Li H, Yong Q (2005) A case study of data-driven interpretable fuzzy modeling. Acta Autom Sin 31(6):815–824

    Google Scholar 

  • Zong-Yi X, Yong Z, Yuan-Long H, Guo-Qiang C (2008) Multi-objective fuzzy modeling using NSGA-II. In: IEEE conference on cybernetics and intelligent systems, 21–24 September 2008, pp 119–124

Download references

Acknowledgments

The authors would like to thank Francisco Herrera and the reviewers for their valuable and useful comments and support in the preparation of this manuscript. This work was supported by the Spanish Ministry of Science and Innovation under Grants no. CIT-460000-2009-46 and DPI2009-14410-C02-02.

Author information

Authors and Affiliations

  1. CARTIF Centro Tecnológico, Parque Tecnológico de Boecillo, parcela 205, 47151, Boecillo, Valladolid, Spain

    Marta Galende-Hernández & Gregorio I. Sainz-Palmero

  2. Department of Systems Engineering and Control, School of Industrial Engineering, University of Valladolid, Paseo del Cauce s/n, 47011, Valladolid, Spain

    Gregorio I. Sainz-Palmero & Maria J. Fuente-Aparicio

Authors
  1. Marta Galende-Hernández
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gregorio I. Sainz-Palmero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maria J. Fuente-Aparicio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marta Galende-Hernández.

Appendix: tables of results

Appendix: tables of results

See Tables 12 and 13.

Table 12 Performance of the improved compact fuzzy models
Full size table
Table 13 Performance of the improved complex fuzzy models
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Galende-Hernández, M., Sainz-Palmero, G.I. & Fuente-Aparicio, M.J. Complexity reduction and interpretability improvement for fuzzy rule systems based on simple interpretability measures and indices by bi-objective evolutionary rule selection. Soft Comput 16, 451–470 (2012). https://doi.org/10.1007/s00500-011-0748-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-011-0748-6

Keywords

Search

Navigation