Skip to main content
SpringerLink
Log in
Book cover

International Work-Conference on Artificial Neural Networks

IWANN 2013: Advances in Computational Intelligence pp 36–58Cite as

A Novel Framework to Design Fuzzy Rule-Based Ensembles Using Diversity Induction and Evolutionary Algorithms-Based Classifier Selection and Fusion

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 7902))

Abstract

Fuzzy rule-based systems have shown a high capability of knowledge extraction and representation when modeling complex, non-linear classification problems. However, they suffer from the so-called curse of dimensionality when applied to high dimensional datasets, which consist of a large number of variables and/or examples. Multiclassification systems have shown to be a good approach to deal with this kind of problems. In this contribution, we propose an multiclassification system-based global framework allowing fuzzy rule-based systems to deal with high dimensional datasets avoiding the curse of dimensionality. Having this goal in mind, the proposed framework will incorporate several multiclassification system methodologies as well as evolutionary algorithms to design fuzzy rule-based multiclassification systems. The proposed framework follows a two-stage structure: 1) fuzzy rule-based multiclassification system design from classical and advanced multiclassification system design approaches, and 2) novel designs of evolutionary component classifier combination. By using our methodology, different fuzzy rule-based multiclassification systems can be designed dealing with several aspects such as improvement of the performance in terms of accuracy, and obtaining a good accuracy-complexity trade-off.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley (2004)

    Google Scholar 

  2. Ho, T.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)

    Article  Google Scholar 

  3. Optiz, D., Maclin, R.: Popular ensemble methods: An empirical study. Journal of Artificial Intelligence Research 11, 169–198 (1999)

    Google Scholar 

  4. Canul-Reich, J., Shoemaker, L., Hall, L.O.: Ensembles of fuzzy classifiers. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), London, pp. 1–6 (2007)

    Google Scholar 

  5. Pedrycz, W., Kwak, K.C.: Boosting of granular models. Fuzzy Sets and Systems 157(22), 2934–2953 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  6. Cordón, O., Quirin, A.: Comparing two genetic overproduce-and-choose strategies for fuzzy rule-based multiclassification systems generated by bagging and mutual information-based feature selection. International Journal of Hybrid Intelligent Systems 7(1), 45–64 (2010)

    MATH  Google Scholar 

  7. Ishibuchi, H., Nakashima, T., Nii, M.: Classification and Modeling With Linguistic Information Granules. Springer (2005)

    Google Scholar 

  8. Casillas, J., Cordon, O., Herrera, F., Magdalena, L.: Interpretability Issues in Fuzzy Modeling. Springer, Heidelberg (2003)

    Book  MATH  Google Scholar 

  9. Alonso, J.M., Magdalena, L., González-Rodríguez, G.: Looking for a good fuzzy system interpretability index: An experimental approach. International Journal of Approximate Reasoning 51, 115–134 (2009)

    Article  MathSciNet  Google Scholar 

  10. Dasarathy, B.V., Sheela, B.V.: A composite classifier system design: Concepts and methodology. Proceedings of IEEE 67(5), 708–713 (1979)

    Article  Google Scholar 

  11. Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)

    MATH  MathSciNet  Google Scholar 

  12. Schapire, R.: The strength of weak learnability. Machine Learning 5(2), 197–227 (1990)

    Google Scholar 

  13. Tsymbal, A., Pechenizkiy, M., Cunningham, P.: Diversity in search strategies for ensemble feature selection. Information Fusion 6(1), 83–98 (2005)

    Article  Google Scholar 

  14. Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  15. Xu, L., Krzyzak, A., Suen, C.Y.: Methods of combining multiple classifiers and their application to handwriting recognition. IEEE Transactions on Systems, Man, and Cybernetics 22(3), 418–435 (1992)

    Article  Google Scholar 

  16. Woods, K., Kegelmeyer, W.P., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(4), 405–410 (1997)

    Article  Google Scholar 

  17. Giacinto, G., Roli, F.: Dynamic classifier selection based on multiple classifier behaviour. Pattern Recognition 34(9), 1879–1881 (2001)

    Article  MATH  Google Scholar 

  18. Partridge, D., Yates, W.B.: Engineering multiversion neural-net systems. Neural Computation 8(4), 869–893 (1996)

    Article  Google Scholar 

  19. Hernández-Lobato, D., Martínez-Muñoz, G., Suárez, A.: How large should ensembles of classi ers be? Pattern Recognition 46(5), 1323–1336 (2013)

    Article  MATH  Google Scholar 

  20. Hühn, J.C., Hüllermeier, E.: FURIA: an algorithm for unordered fuzzy rule induction. Data Mining and Knowledge Discovery 19(3), 293–319 (2009)

    Article  MathSciNet  Google Scholar 

  21. Cohen, W.W.: Fast effective rule induction. In: Proceedings of the Twelfth International Conference on Machine Learning, pp. 115–123. Morgan Kaufmann (1995)

    Google Scholar 

  22. Ishibuchi, H., Nakashima, T., Morisawa, T.: Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets and Systems 103(2), 223–238 (1999)

    Article  Google Scholar 

  23. Cordón, O., del Jesus, M.J., Herrera, F.: A proposal on reasoning methods in fuzzy rule-based classification systems. International Journal of Approximate Reasoning 20, 21–45 (1999)

    Article  Google Scholar 

  24. Takahashi, H., Honda, H.: Lymphoma prognostication from expression profiling using a combination method of boosting and projective adaptive resonance theory. Journal of Chemical Engineering of Japan 39(7), 767–771 (2006)

    Article  Google Scholar 

  25. Bonissone, P.P., Cadenas, J.M., Garrido, M.C., Díaz-Valladares, R.A.: A fuzzy random forest. International Journal of Approximate Reasoning 51(7), 729–747 (2010)

    Article  MathSciNet  Google Scholar 

  26. Marsala, C.: Data mining with ensembles of fuzzy decision trees. In: IEEE Symposium on Computational Intelligence and Data Mining, Nashville, USA, pp. 348–354 (2009)

    Google Scholar 

  27. Janikow, C.Z.: Fuzzy decision trees: issues and methods. IEEE Transactions on Systems, Man, and Cybernetics, Part B 28(1), 1–14 (1998)

    Article  Google Scholar 

  28. Aguilera, J.J., Chica, M., del Jesus, M.J., Herrera, F.: Niching genetic feature selection algorithms applied to the design of fuzzy rule based classification systems. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), London, pp. 1794–1799 (2007)

    Google Scholar 

  29. Goldberg, D.E.: Genetic Algorithms in Search Optimization and Machine Learning. Addison-Wesley (1989)

    Google Scholar 

  30. Nojima, Y., Ishibuchi, H.: Designing fuzzy ensemble classifiers by evolutionary multiobjective optimization with an entropy-based diversity criterion. In: International Conference on Hybrid Intelligent Systems and Conference on Neuro-Computing and Evolving Intelligence, CD-ROM, 4 pages (2006)

    Google Scholar 

  31. Nojima, Y., Ishibuchi, H.: Genetic rule selection with a multi-classifier coding scheme for ensemble classifier design. International Journal of Hybrid Intelligent Systems 4(3), 157–169 (2007)

    MATH  Google Scholar 

  32. Ishibuchi, H., Nojima, Y.: Evolutionary multiobjective optimization for the design of fuzzy rule-based ensemble classifiers. International Journal of Hybrid Intelligent Systems 3(3), 129–145 (2006)

    MATH  Google Scholar 

  33. Yager, R.R., Filev, D.P.: Essentials of fuzzy modeling and control. Wiley-Interscience, New York (1994)

    Google Scholar 

  34. Cordón, O., Herrera, F., Hoffmann, F., Magdalena, L.: Genetic Fuzzy Systems. Evolutionary Tuning and Learning of Fuzzy Knowledge Bases. World Scientific (2001)

    Google Scholar 

  35. Cordón, O., Gomide, F., Herrera, F., Hoffmann, F., Magdalena, L.: Ten years of genetic fuzzy systems: Current framework and new trends. Fuzzy Sets and Systems 141(1), 5–31 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  36. Herrera, F.: Genetic fuzzy systems: taxonomy, current research trends and prospects. Evolutionary Intelligence 1, 27–46 (2008)

    Article  Google Scholar 

  37. Cordón, O.: A historical review of evolutionary learning methods for mamdani-type fuzzy rule-based systems: Designing interpretable genetic fuzzy systems. International Journal of Approximate Reasoning 52(6), 894–913 (2011)

    Article  Google Scholar 

  38. Kuncheva, L.I., Rodríguez, J.J.: Classifier ensembles with a random linear oracle. IEEE Transactions on Knowledge and Data Engineering 19(4), 500–508 (2007)

    Article  Google Scholar 

  39. Rodríguez, J.J., Kuncheva, L.I.: Naïve bayes ensembles with a random oracle. In: Haindl, M., Kittler, J., Roli, F. (eds.) MCS 2007. LNCS, vol. 4472, pp. 450–458. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  40. Sharkey, A.J.C., Sharkey, N.E.: The test and select approach to ensemble combination. In: International Workshop on Multiclassifier Systems, Cagliari, pp. 30–44 (2000)

    Google Scholar 

  41. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 182–197 (2002)

    Article  Google Scholar 

  42. Panov, P., Džeroski, S.: Combining bagging and random subspaces to create better ensembles. In: Berthold, M., Shawe-Taylor, J., Lavrač, N. (eds.) IDA 2007. LNCS, vol. 4723, pp. 118–129. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  43. Stefanowski, J.: An experimental study of methods combining multiple classifiers - diversified both by feature selection and bootstrap sampling. In: Atanassov, K.T., Kacprzyk, J., Krawczak, M., Szmidt, E. (eds.) Issues in the Representation and Processing of Uncertain and Imprecise Information, pp. 337–354. Akademicka Oficyna Wydawnicza EXIT, Warsaw (2005)

    Google Scholar 

  44. Trawiński, K., Cordón, O., Quirin, A.: On designing fuzzy rule-based multiclassification systems by combining furia with bagging and feature selection. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 19(4), 589–633 (2011)

    Article  Google Scholar 

  45. Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks 5(4), 537–550 (1994)

    Article  Google Scholar 

  46. Feo, T.A., Resende, M.G.C.: Greedy randomized adaptive search procedures. Journal of Global Optimization 6, 109–133 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  47. Shannon, C.E., Weaver, W.: The Mathematical Theory of Communication. University of Illlinois Press (1949)

    Google Scholar 

  48. Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998), http://archive.ics.uci.edu/ml

  49. Dietterich, T.G.: Approximate statistical test for comparing supervised classification learning algorithms. Neural Computation 10(7), 1895–1923 (1998)

    Article  Google Scholar 

  50. Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco (1993)

    Google Scholar 

  51. Dietterich, T.G.: An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40(2), 139–157 (2000)

    Article  Google Scholar 

  52. Trawiński, K., Cordón, O., Sánchez, L., Quirin, A.: Multiobjective genetic classifier selection for random oracles fuzzy rule-based multiclassifiers: How benefical is the additional diversity? Technical Report AFE 2012-17, European Centre for Soft Computing, Mieres, Spain (2012)

    Google Scholar 

  53. Dos Santos, E.M., Sabourin, R., Maupin, P.: A dynamic overproduce-and-choose strategy for the selection of classifier ensembles. Pattern Recognition 41(10), 2993–3009 (2008)

    Article  MATH  Google Scholar 

  54. Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 51(2), 181–207 (2003)

    Article  MATH  Google Scholar 

  55. Domingos, P., Pazzani, M.J.: On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning 29(2-3), 103–130 (1997)

    Article  MATH  Google Scholar 

  56. Alcalá-Fdez, J., Fernández, A., Luengo, J., Derrac, J., García, S.: Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. Journal of Multiple-Valued Logic and Soft Computing 17(2-3), 255–287 (2011)

    Google Scholar 

  57. Margineantu, D.D., Dietterich, T.G.: Pruning adaptive boosting. In: Proceedings of the Fourteenth International Conference on Machine Learning, ICML 19897, pp. 211–218. Morgan Kaufmann Publishers Inc., San Francisco (1997)

    Google Scholar 

  58. Trawiński, K., Quirin, A., Cordón, O.: A study on the use of multi-objective genetic algorithms for classifier selection in furia-based fuzzy multiclassifers. International Journal of Computational Intelligence Systems 5(2), 231–253 (2012)

    Article  Google Scholar 

  59. Coello, C.A., Lamont, G.B., van Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-Objective Problems, 2nd edn. Springer (2007)

    Google Scholar 

  60. Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Transactions on Evolutionary Computation 3, 257–271 (1999)

    Article  Google Scholar 

  61. Trawiński, K., Cordón, O., Sánchez, L., Quirin, A.: A genetic fuzzy linguistic combination method for fuzzy rule-based multiclassifiers. IEEE Transactions on Fuzzy Systems (in press, 2013), doi:10.1109/TFUZZ.2012.2236844.

    Google Scholar 

  62. Torra, V.: A review of the construction of hierarchical fuzzy systems. International Journal of Intelligent Systems 17(5), 531–543 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  63. Gegov, A.E., Frank, P.M.: Hierarchical fuzzy control of multivariable systems. Fuzzy Sets and Systems 72, 299–310 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  64. Yager, R.R.: On the construction of hierarchical fuzzy systems model. IEEE Transactions on Systems, Man, and Cybernetics - Part B 28(1), 55–66 (1998)

    Article  Google Scholar 

  65. Cordón, O., Herrera, F., Zwir, I.: A hierarchical knowledge-based environment for linguistic modeling: Models and iterative methodology. Fuzzy Sets and Systems 138(2), 307–341 (2003)

    Article  MathSciNet  Google Scholar 

  66. Wolpert, D.: Stacked generalization. Neural Networks 5(2), 241–259 (1992)

    Article  MathSciNet  Google Scholar 

  67. Dimililer, N., Varoglu, E., Altincay, H.: Classifier subset selection for biomedical named entity recognition. Applied Intelligence 31, 267–282 (2009)

    Article  Google Scholar 

  68. Kuncheva, L.I., Bezdek, J.C., Duin, R.P.W.: Decision templates for multiple classifier fusion: An experimental comparison. Pattern Recognition 34(2), 299–314 (2001)

    Article  MATH  Google Scholar 

  69. Kuncheva, L.I.: “Fuzzy” versus “nonfuzzy” in combining classifiers designed by boosting. IEEE Transactions on Fuzzy Systems 11(6), 729–741 (2003)

    Article  Google Scholar 

  70. Ruta, D., Gabrys, B.: Classifier selection for majority voting. Information Fusion 6(1), 63–81 (2005)

    Article  Google Scholar 

  71. Trawiński, K., Alonso, J.M., Hernández, N.: A multiclassifier approach for topology-based wifi indoor localization. Soft Computing (in press, 2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Edificio Científico-Tecnológico, European Centre for Soft Computing, planta 3, C. Gonzalo Gutiérrez Quirós s/n, 33600, Mieres, Asturias, Spain

    Oscar Cordón & Krzysztof Trawiński

  2. Dept. of Computer Science and Artificial Intelligence (DECSAI) and the Research Center on Information and Communication Technologies (CITIC-UGR), University of Granada, 18071, Granada, Spain

    Oscar Cordón

Authors
  1. Oscar Cordón
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Krzysztof Trawiński
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Architecture and Computer Technology, University of Granada,, Periodista Daniel Saucedo Aranda s/n,, 18071, Granada, Spain

    Ignacio Rojas

  2. Department of Electronics Technology, University of Malaga, 29071, Malaga, Spain

    Gonzalo Joya

  3. Department of Electronics Engineering, Universitat Politecnica de Catalunya, 08034, Barcelona, Spain

    Joan Gabestany

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Cordón, O., Trawiński, K. (2013). A Novel Framework to Design Fuzzy Rule-Based Ensembles Using Diversity Induction and Evolutionary Algorithms-Based Classifier Selection and Fusion. In: Rojas, I., Joya, G., Gabestany, J. (eds) Advances in Computational Intelligence. IWANN 2013. Lecture Notes in Computer Science, vol 7902. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38679-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-38679-4_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-38678-7

  • Online ISBN: 978-3-642-38679-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation