Soft Computing

, Volume 15, Issue 12, pp 2335–2354 | Cite as

Learning knowledge bases of multi-objective evolutionary fuzzy systems by simultaneously optimizing accuracy, complexity and partition integrity

  • Michela Antonelli
  • Pietro Ducange
  • Beatrice Lazzerini
  • Francesco Marcelloni
Focus

Abstract

In the last few years, several papers have exploited multi-objective evolutionary algorithms (MOEAs) to generate Mamdani fuzzy rule-based systems (MFRBSs) with different trade-offs between interpretability and accuracy. In this framework, a common approach is to distinguish between interpretability of the rule base (RB), also known as complexity, and interpretability of fuzzy partitions, also known as integrity of the database (DB). Typically, complexity has been used as one of the objectives of the MOEAs, while partition integrity has been ensured by enforcing constraints on the membership function (MF) parameters. In this paper, we propose to adopt partition integrity as an objective of the evolutionary process. To this aim, we first discuss how partition integrity can be measured by using a purposely defined index based on the similarity between the partitions learned during the evolutionary process and the initial interpretable partitions defined by an expert. Then, we introduce a three-objective evolutionary algorithm which generates a set of MFRBSs with different trade-offs between complexity, accuracy and partition integrity by concurrently learning the RB and the MF parameters of the linguistic variables. Accuracy is assessed in terms of mean squared error between the actual and the predicted values, complexity is calculated as the total number of conditions in the antecedents of the rules and integrity is measured by using the purposely defined index. The proposed approach has been experimented on six real-world regression problems. The results have been compared with those obtained by applying the same MOEA, but with only accuracy and complexity as objectives, both to learn only RBs, and to concurrently learn RBs and MF parameters, with and without constraints on the parameter tuning. We show that our approach achieves the best trade-offs between interpretability and accuracy. Finally, we compare our approach with a similar MOEA recently proposed in the literature.

Keywords

Accuracy-interpretability trade-off Partition integrity index Multi-objective evolutionary fuzzy systems Piecewise linear transformation 

References

  1. Alcalá R, Alcalá-Fdez J, Herrera F, Otero J (2007a) Genetic learning of accurate and compact fuzzy rule based systems based on the 2-tuples linguistic representation. Int J Approx Reason 44:45–64MATHCrossRefGoogle Scholar
  2. Alcalá R, Gacto MJ, Herrera F, Alcalá-Fdez J (2007b) A multi-objective genetic algorithm for tuning and rule selection to obtain accurate and compact linguistic fuzzy rule-based systems. Int J Uncertain Fuzziness Knowl Based Syst 15(5):539–557MATHCrossRefGoogle Scholar
  3. Alcalá R, Ducange P, Herrera F, Lazzerini B, Marcelloni F (2009) A multi-objective evolutionary approach to concurrently learn rule and data bases of linguistic fuzzy rule-based systems. IEEE Trans Fuzzy Syst 17(5):1106–1122CrossRefGoogle Scholar
  4. Alonso JM, Magdalena L, Guillaume S (2008) HILK: a new methodology for designing highly interpretable linguistic knowledge bases using the fuzzy logic formalism. Int J Intell Syst 23:761–794MATHCrossRefGoogle Scholar
  5. Alonso JM, Magdalena L, Gonzalez-Rodriguez G (2009) Looking for a good fuzzy system interpretability index: an experimental approach. Int J Approx Reason 51:115–134MathSciNetCrossRefGoogle Scholar
  6. Antonelli M, Ducange P, Lazzerini B, Marcelloni F (2009a) Exploiting a new interpretability index in the multi-objective evolutionary learning of Mamdani fuzzy rule-based systems. In: Proceedings of the 9th international conference on intelligent systems design and applications (ISDA’09), 30 Nov–2 Dec, Pisa, Italy, pp 115–150Google Scholar
  7. Antonelli M, Ducange P, Lazzerini B, Marcelloni F (2009b) A three-objective evolutionary approach to generate Mamdani fuzzy rule-based systems. In: Proceedings of the 4th international conference on hybrid artificial intelligence systems (HAIS’09), 10–12 June, Salamanca, Spain, pp 613–620Google Scholar
  8. Antonelli M, Ducange P, Lazzerini B, Marcelloni F (2009c) Learning concurrently partition granularities and rule bases of Mamdani fuzzy systems in a multi-objective evolutionary framework. Int J Approx Reason 50(7):1066–1080CrossRefGoogle Scholar
  9. Antonelli M, Ducange P, Lazzerini B, Marcelloni F (2009d) Multi-objective evolutionary learning of granularity, membership function parameters and rules of Mamdani fuzzy systems. Evol Intell 2(1–2):21–37CrossRefGoogle Scholar
  10. Botta A, Lazzerini B, Marcelloni F, Stefanescu D (2009) Context adaptation of fuzzy systems through a multi-objective evolutionary approach based on a novel interpretability index. Soft Comput 13(5):437–449CrossRefGoogle Scholar
  11. Casillas J, Cordón O, Herrera F (2002) COR: a methodology to improve ad hoc data-driven linguistic rule learning methods by inducing cooperation among rules. IEEE Trans Syst Man Cybern 32(4):526–537Google Scholar
  12. Casillas J, Cordon O, Herrera F, Magdalena L (eds) (2003) Interpretability issues in fuzzy modeling. Studies in fuzziness and soft computing, vol 128. Springer, HeidelbergGoogle Scholar
  13. Cococcioni M, Ducange P, Lazzerini B, Marcelloni F (2007) A Pareto-based multi-objective evolutionary approach to the identification of Mamdani fuzzy systems. Soft Comput 11(11):1013–1031CrossRefGoogle Scholar
  14. Coello Coello CA, Lamont GB (2004) Applications of multi-objective evolutionary algorithms. Advances in natural computation, vol 1. World ScientificGoogle Scholar
  15. Cordón O, del Jesus MJ, Herrera F, Lozano M (1999) MOGUL: a methodology to obtain genetic fuzzy rule-based systems under the iterative rule learning approach. Int J Intell Syst 14(11):1123–1153MATHCrossRefGoogle Scholar
  16. de Oliveira JV (1999) Semantic constraints for membership function optimization. IEEE Trans Syst Man Cybern Part A 29(1):128–138CrossRefGoogle Scholar
  17. Deb K (2001) Multi-objective optimization using evolutionary algorithms. WileyGoogle Scholar
  18. Ducange P, Lazzerini B, Marcelloni F (2009) Multi-objective genetic fuzzy classifiers for imbalanced and cost-sensitive datasets. Soft Comput 14(7):713–728CrossRefGoogle Scholar
  19. Fazendeiro P, de Oliveira JV, Pedrycz W (2007) A multiobjective design of a patient and anaesthetist-friendly neuromuscular blockade controller. IEEE Trans Biomed Eng 54(9):667–1678CrossRefGoogle Scholar
  20. Gacto MJ, Alcalá R, Herrera F (2009) Adaptation and application of multi-objective evolutionary algorithms for rule reduction and parameter tuning of fuzzy rule-based systems. Soft Comput 13(5):419–436CrossRefGoogle Scholar
  21. Gacto MJ, Alcalá R, Herrera F (2010) Integration of an index to preserve the semantic interpretability in the multi-objective evolutionary rule selection and tuning of linguistic fuzzy systems. IEEE Trans Fuzzy Syst 18(3):515–531CrossRefGoogle Scholar
  22. González A, Pérez R (1999) SLAVE: a genetic learning system based on the iterative approach. IEEE Trans Fuzzy Syst 7:176–191CrossRefGoogle Scholar
  23. Guillaume S (2001) Designing fuzzy inference systems from data: an interpretability-oriented review. IEEE Trans Fuzzy Syst 9(3):426–443MathSciNetCrossRefGoogle Scholar
  24. Herrera F (2008) Genetic fuzzy systems: taxonomy, current research trends and prospects. Evol Intell 1:27–46CrossRefGoogle Scholar
  25. Ishibuchi H (2007) Multiobjective genetic fuzzy systems: review and future research direction. In: Proceedings of FUZZ-IEEE 2007 international conference on fuzzy systems, London, 23–26 July, pp 913–918Google Scholar
  26. Ishibuchi H, Nojima Y (2007) Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning. Int J Approx Reason 44(1):4–31MathSciNetMATHCrossRefGoogle Scholar
  27. Ishibuchi H, Yamamoto T (2004) Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining. Fuzzy Sets Syst 141(1):59–88MathSciNetMATHCrossRefGoogle Scholar
  28. Ishibuchi H, Murata T, Turksen IB (1997) Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems. Fuzzy Sets Syst 89(2):135–150CrossRefGoogle Scholar
  29. Klawonn F (2006) Reducing the number of parameters of a fuzzy system using scaling functions. Soft Comput 10(9):749–756CrossRefGoogle Scholar
  30. Knowles JD, Corne DW (2000) Approximating the non dominated front using the Pareto archived evolution strategy. Evol Comput 8(2):149–172CrossRefGoogle Scholar
  31. Mamdani EH, Assilian S (1975) An experiment in linguistic synthesis with a fuzzy logic controller. Int J Man Mach Stud 7(1):1–13MATHCrossRefGoogle Scholar
  32. Mencar C, Fanelli AM (2008) Interpretability constraints for fuzzy information granulation. Inf Sci 178:4585–4618MathSciNetCrossRefGoogle Scholar
  33. Mencar C, Castellano G, Fanelli AM (2007) Distinguishability quantification of fuzzy sets. Inf Sci 177:130–149MathSciNetMATHCrossRefGoogle Scholar
  34. Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley-IEEE PressGoogle Scholar
  35. Pulkkinen P, Koivisto H (2008) Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms. Int J Approx Reason 48:526–543CrossRefGoogle Scholar
  36. Pulkkinen P, Hytönen J, Koivisto H (2008) Developing a bioaerosol detector using hybrid genetic fuzzy systems. Eng Appl Artif Intell 21(8):1330–1346CrossRefGoogle Scholar
  37. Ruspini EH (1969) A new approach to clustering. Inform Control 15(1):22–32MATHCrossRefGoogle Scholar
  38. Wang LX, Mendel JM (1992) Generating fuzzy rules by learning from examples. IEEE Trans Syst Man Cybern 22(6):1414–1427MathSciNetCrossRefGoogle Scholar
  39. Zhou SM, Gan JQ (2008) Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling. Fuzzy Sets Syst 159:3091–3131MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Michela Antonelli
    • 1
  • Pietro Ducange
    • 1
  • Beatrice Lazzerini
    • 1
  • Francesco Marcelloni
    • 1
  1. 1.Dipartimento di Ingegneria dell’Informazione: Elettronica, Informatica, TelecomunicazioniUniversity of PisaPisaItaly

Personalised recommendations