Application of Base Learners as Conditional Input for Fuzzy Rule-Based Combined System

  • Athanasios Tsakonas
  • Bogdan Gabrys
Part of the Studies in Computational Intelligence book series (SCI, volume 577)


The aim of this work is to examine the possibility of using the output of base learners as antecedents for fuzzy rule-based hybrid ensembles. We select a flexible, grammar-driven framework for generating ensembles that combines multilayer perceptrons and support vector machines by means of genetic programming. We assess the proposed model in three real-world regression problems and we test it against multi-level, hierarchical ensembles. Our first results show that for a given large size of the base learners pool, the outputs of some of them can be useful in the antecedent parts to produce accurate ensembles, while at the same time other more accurate members of the same pool contribute in the consequent part.


Support Vector Machine Genetic Programming Fuzzy Rule Multilayer Perceptrons Base Learner 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alba, E., Cotta, C., Troya, J.: Evolutionary design of fuzzy logic controllers using strongly-typed GP. In: Proc. 1996 IEEE Int’l Symposium on Intelligent Control, New York, NY (1996)Google Scholar
  2. 2.
    Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity creation methods: a survey and categorisation. Inf. Fusion 6(1), 5–20 (2005)CrossRefGoogle Scholar
  3. 3.
    Chandra, A., Yao, X.: Divace: Diverse and accurate ensemble learning algorithm. In: Yang, Z.R., Yin, H., Everson, R.M. (eds.) IDEAL 2004. LNCS, vol. 3177, pp. 619–625. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Chandra, A., Yao, X.: Evolving hybrid ensembles of learning machines for better generalisation. Neurocomputing 69, 686–700 (2006)CrossRefGoogle Scholar
  5. 5.
    Clemen, R.: Combining forecasts: A review and annotated bibliography. International Journal of Forecasting 5(4), 559–583 (1989)CrossRefGoogle Scholar
  6. 6.
    Coelho, A., Fernandes, E., Faceli, K.: Multi-objective design of hierarchical consensus functions for clustering ensembles via genetic programming. Decision Support Systems 51(4), 794–809 (2011)CrossRefGoogle Scholar
  7. 7.
    Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  8. 8.
    Duin, R.: The combining classifier: to train or not to train? In: Proc. of the 16th Int’l Conf. on Pattern Recognition, pp. 765–770 (2002)Google Scholar
  9. 9.
    Ein-Dor, P., Feldmesser, J.: Attributes of the performance of central processing units: A relative performance prediction model. Commun. ACM 30(30), 308–317 (1984)Google Scholar
  10. 10.
    Evangelista, P., Bonissone, P., Embrechts, M., Szymanski, B.: Unsupervised fuzzy ensembles and their use in intrusion detection. In: European Symposium on Artificial Neural Networks (ESANN 2005), Bruges, Belgium (2005)Google Scholar
  11. 11.
    Fernandez, F., Tommassini, M., Vanneschi, L.: An empirical study of multipopulation genetic programming. Genetic Programming and Evolvable Machines 4(1) (2003)Google Scholar
  12. 12.
    Folino, G., Pizzuti, C., Spezzano, G.: Ensemble techniques for parallel genetic programming based classifiers. In: Ryan, C., Soule, T., Keijzer, M., Tsang, E.P.K., Poli, R., Costa, E. (eds.) EuroGP 2003. LNCS, vol. 2610, pp. 59–69. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Frank, A., Asuncion, A.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA (2010),
  14. 14.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.: The weka data mining software: An update. SIGKDD Explorations 11(1) (2009)Google Scholar
  15. 15.
    Hong, J., Cho, S.: The classification of cancer based on dna microarray data that uses diverse ensemble genetic programming. Artif. Intell. in Med. 36(1), 43–58 (2006)CrossRefGoogle Scholar
  16. 16.
    Ishibuchi, H.: Multiobjective genetic fuzzy systems: Review and future research directions. In: IEEE Int’l Conf. on Fuzzy Systems (FUZZ-IEEE 2007), pp. 59–69. Imperial College (2007)Google Scholar
  17. 17.
    Jacobs, R.: Bias-variance analyses of mixture-of-experts architectures. Neural Computation, 369–383 (1997)Google Scholar
  18. 18.
    Jensen, R., Shen, Q.: New approaches to fuzzy-rough feature selection. IEEE Trans. on Fuzzy Systems 17(4), 824–838 (2009)CrossRefGoogle Scholar
  19. 19.
    Kadlec, P., Gabrys, B.: Local learning-based adaptive soft sensor for catalyst activation prediction. AIChE Journal 57(5), 1288–1301 (2011)CrossRefGoogle Scholar
  20. 20.
    Koza, J.: Genetic programming - On the programming of computers by means of natural selection. The MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  21. 21.
    Kuncheva, L.: Fuzzy versus nonfuzzy in combining classifiers designed by boosting. IEEE Trans. on Fuzzy Systems 11(6), 729–741 (2003)CrossRefGoogle Scholar
  22. 22.
    Kuncheva, L.: Combining pattern classifiers: methods and algorithms. John Wiley and Sons, New York (2004)Google Scholar
  23. 23.
    Liaw, A., Wiener, M.: Classification and regression by randomforest. Expert Systems with Applications (2002) (under review)Google Scholar
  24. 24.
    Medina-Chico, V., Suárez, A., Lutsko, J.F.: Backpropagation in decision trees for regression. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 348–359. Springer, Heidelberg (2001)Google Scholar
  25. 25.
    Quinlan, J.R.: Learning with continuous classes. In: AI 1992. World Scientific, Singapore (1992)Google Scholar
  26. 26.
    Ruta, D., Gabrys, B.: An overview of classifier fusion methods. Computing and Information Systems 7(2), 1–10 (2000)Google Scholar
  27. 27.
    Scholkopf, B., Smola, A.: Learning with Kernels - Support Vector Machines, Regularization, Optimization and Beyond. The MIT Press, Cambridge (2002)Google Scholar
  28. 28.
    Sharkey, A., Sharkey, N., Gerecke, U., Chandroth, G.: The test and select approach to ensemble combination. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 30–44. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  29. 29.
    Tsakonas, A.: A comparison of classification accuracy of four genetic programming evolved intelligent structures. Information Sciences 17(1), 691–724 (2006)CrossRefGoogle Scholar
  30. 30.
    Tsakonas, A., Gabrys, B.: Evolving takagi-sugeno-kang fuzzy systems using multi-population grammar guided genetic programmings. In: Int’l Conf. Evol. Comp. Theory and Appl. (ECTA 2011), Paris, France (2011)Google Scholar
  31. 31.
    Tsakonas, A., Gabrys, B.: Gradient: Grammar-driven genetic programming framework for building multi-component, hierarchical predictive systems. Expert Systems with Applications 39(18), 13253–13266 (2012)CrossRefGoogle Scholar
  32. 32.
    Tsakonas, A., Gabrys, B.: A fuzzy evolutionary framework for combining ensembles. Applied Soft Computing (2013), Scholar
  33. 33.
    Yeh, I.-C.: Modeling slump of concrete with fly ash and superplasticizer. Computers and Concrete 5(6), 559–572 (2008)CrossRefGoogle Scholar
  34. 34.
    Zhang, Y., Bhattacharyya, S.: Genetic programming in classifying large-scale data: an ensemble method. Information Sciences 163(1), 85–101 (2004)CrossRefGoogle Scholar
  35. 35.
    Zhou, Z., Wu, J., Jiang, Y., Chen, R.: Genetic algorithm based selective neural network ensemble. In: 17th Int’l Joint Conf. Artif. Intell., pp. 797–802. Morgan Kaufmann, USA (2001)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Athanasios Tsakonas
    • 1
  • Bogdan Gabrys
    • 1
  1. 1.Smart Technology Research CentreBournemouth UniversityFern BarrowU.K.

Personalised recommendations