Skip to main content
SpringerLink
Log in

Selection of Heterogeneous Fuzzy Model Ensembles Using Self-adaptive Genetic Algorithms

  • Published:
New Generation Computing Aims and scope Submit manuscript

Abstract

The problem of model selection to compose a heterogeneous bagging ensemble was addressed in the paper. To solve the problem, three self-adapting genetic algorithms were proposed with different control parameters of mutation, crossover, and selection adjusted during the execution. The algorithms were applied to create heterogeneous ensembles comprising regression fuzzy models to aid in real estate appraisals. The results of experiments revealed that the self-adaptive algorithms converged faster than the classic genetic algorithms. The heterogeneous ensembles created by self-adapting methods showed a very good predictive accuracy when compared with the homogeneous ensembles obtained in earlier research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alcalá R., Alcalá-Fdez J., Casillas J., Cordón O., Herrera F.: “Local identification of prototypes for genetic learning of accurate TSK fuzzy rule-based systems”. International Journal of Intelligent Systems, 22(9), 909–941 (2007)

    Article  MATH  Google Scholar 

  2. Alcalá R., Cordón O., Herrera F., “Combining Rule Weight Learning and Rule Selection to Obtain Simpler and More Accurate Linguistic Fuzzy Models,” Modeling with Words (Lawry, J. ed.), LNCS 2873, pp. 44–63, Springer, Heidelberg, 2003.

  3. Alcalá-Fdez, J., Fernandez, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F., “KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework,” Journal of Multiple-Valued Logic and Soft Computing, in press, 2011.

  4. Alcalá-Fdez J., Sánchez L, García S, del Jesus M.J., Ventura S., Garrell J.M., Otero J., Romero C., Bacardit J., Rivas V.M., Fernández J.C., Herrera F.: “KEEL: A Software Tool to Assess Evolutionary Algorithms to Data Mining Problems”. Soft Computing 13(3), 307–318 (2009)

    Article  Google Scholar 

  5. Angeline P.J.: “Adaptive and self-adaptive evolutionary computations”. In: Palaniswami, M., Attikiouzel, Y. (eds) Computational Intelligence: A Dynamic Systems Perspective., pp. 152–163. IEEE Press, New York (1995)

    Google Scholar 

  6. Bäck T., Schwefel H.-P.: “An Overview of Evolutionary Algorithms for Parameter Optimization”. Evolutionary Computation 1(1), 1–23 (1993)

    Article  Google Scholar 

  7. Bäck, T., “Self-adaptation in genetic algorithms,” in Toward a Practice of Autonomous Systems (Varela, F. J. and Bourgine, P. eds.), in Proc. First European Conference on Artificial Life, pp. 263–271, MIT Press, 1992.

  8. Bańczyk, K., Kempa, O., Lasota, T., Trawiński, B., “Empirical Comparison of Bagging Ensembles Created Using Weak Learners for a Regression Problem,” ACIIDS 2011 (Nguyen, N. T., Kim, C.-G. and Janiak, A. eds.), LNAI 6592, pp. 312–322, Springer, Heidelberg, 2011.

  9. Banfield R.E., Hall L.O., Bowyer K.W., Kegelmeyer W.P.: “A comparison of decision tree ensemble creation techniques”. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(1), 173–180 (2007)

    Article  Google Scholar 

  10. Biau G., Cérou F., Guyader A.: “On the Rate of Convergence of the Bagged Nearest Neighbor Estimate”. Journal of Machine Learning Research 11, 687–712 (2010)

    Google Scholar 

  11. Brown G., Wyatt J., Tiño P.: “Managing Diversity in Regression Ensembles”. Journal of Machine Learning Research 6, 1621–1650 (2005)

    Google Scholar 

  12. Búhlmann P., Yu B.: “Analyzing bagging”. Annals of Statistics 30, 927–961 (2002)

    Article  MathSciNet  Google Scholar 

  13. Carse B., Fogarty T.C., Munro A.: “Evolving fuzzy rule based controllers using genetic algorithms”. Fuzzy Sets and Systems 80(3), 273–293 (1996)

    Article  Google Scholar 

  14. Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A., “Ensemble selection from libraries of models,” in Proc. of the 21st International Conference on Machine Learning, ACM Press, pp. 137–144, 2004.

  15. Casillas J., Cordón O., Herrera F.: “COR: A Methodology to Improve ad hoc Data-Driven Linguistic Rule Learning Methods by Inducing Cooperation Among Rules”. IEEE Trans. on System, Man and Cybernetics, Part B: Cybernetics 32(4), 526–537 (2002)

    Article  Google Scholar 

  16. Chandra A., Yao X.: “Evolving hybrid ensembles of learning machines for better generalization”. Neurocomputing 69, 686–700 (2006)

    Article  Google Scholar 

  17. Cordón O., Herrera F.: “A three-stage evolutionary process for learning descriptive and approximate fuzzy logic controller knowledge bases from examples”. International Journal of Approximate Reasoning 17(4), 369–407 (1997)

    Article  MATH  Google Scholar 

  18. Cordón O., Herrera F.: “Hybridizing genetic algorithms with sharing scheme and evolution strategies for designing approximate fuzzy rule-based systems”. Fuzzy Sets and Systems 118(2), 235–255 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  19. Cordón O., Quirin A.: “Comparing Two Genetic Overproduce-and-choose Strategies for Fuzzy Rule-based Multiclassification Systems Generated by Bagging and Mutual Information-based Feature Selection”. International Journal of Hybrid Intelligent Systems 7(1), 45–64 (2010)

    MATH  Google Scholar 

  20. De Jong, K., “An analysis of the behavior of a class of genetic adaptive systems” Ph.D. thesis, University of Michigan, 1975.

  21. Deb K., Beyer H.-G.: “Self-adaptive genetic algorithms with simulated binary crossover”. Evolutionary Computation 9(2), 197–221 (2001)

    Article  Google Scholar 

  22. Digalakis J.G., Margaritis K.G.: “An Experimental Study of Benchmarking Functions for Genetic Algorithms”. Int. J. Computer Math 79(4), 403–416 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  23. Dos Santos E.M., Sabourin R., Maupin P.: “A dynamic overproduce-and choose strategy for the selection of classifier ensembles”. Pattern Recognition 41, 2993–3009 (2008)

    Article  MATH  Google Scholar 

  24. Dutta, H., “Measuring Diversity in Regression Ensembles,” in Proc. of the 4th Indian International Conference on Artificial Intelligence (Prasad, B., Lingras, P., Ram, A. eds.), IICAI 2009, Tumkur, Karnataka, India, pp. 2220–2236, 2009.

  25. Eiben E., Hinterding R., Michalewicz Z.: “Parameter control in evolutionary algorithms”. IEEE Transactions on Evolutionary Computation 3(2), 124–141 (1999)

    Article  Google Scholar 

  26. García-Pedrajas N.: “Constructing Ensembles of Classifiers by Means of Weighted Instance Selection”. IEEE Transactions on Neural Networks 20(2), 258–277 (2009)

    Article  Google Scholar 

  27. Graczyk, M., Lasota, T., Trawiński, B., “Comparative Analysis of Premises Valuation Models Using KEEL, RapidMiner, and WEKA,” in ICCCI 2009, LNAI 5796 (Nguyen, N. T. et al. eds.), pp. 800–812, Springer, Heidelberg, 2009.

  28. Graczyk, M., Lasota, T., Telec, Z., Trawiński, B., “A Multi-agent System to Assist with Property Valuation Using Heterogeneous Ensembles of Fuzzy Models,” in KES-AMSTA 2010, LNAI 6070 (Jedrzejowicz, P. et al. eds.), pp. 420–429, Springer, Heidelberg, 2010.

  29. Graczyk, M., Lasota, T., Trawiński, B., Trawiński, K., “Comparison of Bagging, Boosting and Stacking Ensembles Applied to Real Estate Appraisal,” in ACIIDS 2010, LNAI 5991 (Nguyen, N. T. et al. eds.), pp. 340–350, Springer, Heidelberg, 2010.

  30. Hansen N., Ostermeier A.: “Completely derandomized self-adaptation in evolution strategies”. Evolutionary Computation 9(2), 159–195 (2001)

    Article  Google Scholar 

  31. Herrera F., Lozano M., Verdegay J.L.: “Tuning Fuzzy Logic Controllers by Genetic Algorithms”. International Journal of Approximate Reasoning 12, 299–315 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  32. Hinterding, R., Michalewicz, Z., Eiben, A. E., “Adaptation in Evolutionary Computation: A Survey,” in Proc. of the Fourth International Conference on Evolutionary Computation (ICEC 97), pp. 65–69, IEEE Press, New York, 1997.

  33. Ho T.: “The random subspace method for constructing decision forests”. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)

    Article  Google Scholar 

  34. Jackowski K., Woźniak M.: “Method of classifier selection using the genetic approach”. Expert Systems 27(2), 114–128 (2010)

    Article  Google Scholar 

  35. Juang C.-F., Lin J.-Y., Lin C.-T.: “Genetic reinforcement learning through symbiotic evolution for fuzzy controller design”. IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics 30(2), 290–302 (2000)

    Article  MathSciNet  Google Scholar 

  36. Kégl, B., “Robust regression by boosting the median,” in Proc. of the 16th Conference on Computational Learning Theory, pp. 258–272, 2003.

  37. Kempa, O., Lasota, T., Telec, Z. and Trawiński, B., “Investigation of Bagging Ensembles of Genetic Neural Networks and Fuzzy Systems for Real Estate Appraisal,” ACIIDS 2011, LNAI 6592 (Nguyen, N. T. Kim, C.-G. and Janiak, A. eds.), pp. 323–332, Springer, Heidelberg, 2011.

  38. Kim Y.-W., Oh I.-S.: “Classifier ensemble selection using hybrid genetic algorithms”. Pattern Recognition Letters 29, 796–802 (2008)

    Article  Google Scholar 

  39. Król D., Lasota T., Trawiński B., Trawiński K.: “Investigation of evolutionary optimization methods of TSK fuzzy model for real estate appraisal”. International Journal of Hybrid Intelligent Systems 5(3), 111–128 (2008)

    MATH  Google Scholar 

  40. Krzystanek, M., Lasota, T., Telec, Z., Trawiński, B., “Analysis of Bagging Ensembles of Fuzzy Models for Premises Valuation,” in ACIIDS 2010, Part II, LNCS (LNAI) 5991 (Nguyen, N. T., Le, M. T. and Świa̧tek, J. eds.), pp. 330– 339, Springer, Heidelberg, 2010.

  41. Kuncheva L.I., Whitaker C.J.: “Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy”. Machine Learning 51, 181–207 (2003)

    Article  MATH  Google Scholar 

  42. Lasota T., Mazurkiewicz J., Trawiński B., Trawiński K.: “Comparison of Data Driven Models for the Validation of Residential Premises using KEEL”. International Journal of Hybrid Intelligent Systems 7(1), 3–16 (2010)

    MATH  Google Scholar 

  43. Lasota, T., Telec, Z., Trawiński, B., Trawiński K., “Exploration of Bagging Ensembles Comprising Genetic Fuzzy Models to Assist with Real Estate Appraisals,” in IDEAL 2009, LNCS 5788 (Yin, H., Corchado, E. eds.) pp. 554–561, Springer, Heidelberg, 2009.

  44. Lofstrom, T., Johansson, U., Bostrom, H. “Bostrom, H.: Ensemble member selection using multi-objective optimization”. IEEE Symposium on Computational Intelligence and Data Mining, CIDM’09, pp. 245–251, 2009.

  45. Maruo, M. H., Lopes, H. S., Delgado, M. R., “Self-Adapting Evolutionary Parameters Encoding Aspects for Combinatorial Optimization Problems,” in Evo- COP 2005, LNCS 3448 (Raidl, G. R., Gottlieb, J. eds.), pp. 154–165, Springer, Heidelberg, 2005.

  46. Meyer-Nieberg, S., Beyer, H-G., “Self-Adaptation in Evolutionary Algorithms,” in SCI (Lobo, F. G., Lima, C. F., Michalewicz, Z. eds.), 54, pp. 47–75, Springer, Heidelberg, 2007.

  47. Partalas I., Tsoumakas G., Vlahavas I.: “Pruning an Ensemble of Classifiers via Reinforcement Learning”. Neurocomputing 72(79), 1900–1909 (2009)

    Article  Google Scholar 

  48. Partridge D., Yates W.B.: “Engineering multiversion neural-net systems”. Neural Computation 8(4), 869–893 (1996)

    Article  Google Scholar 

  49. Sánchez L., Couso I.: “Combining GP operators with SA search to evolve fuzzy rule based classifiers”. Information Sciences 136, 175–192 (2001)

    Article  MATH  Google Scholar 

  50. Sánchez, L., Couso, I., “Fuzzy random variables-based modeling with GA-P Algorithms,” in Information, Uncertainty and Fusion (Yager, R., Bouchon-Menier, B., Zadeh, L. eds.), Kluwer Editors, pp. 245–256, 2000.

  51. Sánchez L.: “A random sets-based method for identifying fuzzy models”. Fuzzy Sets and Systems 98(3), 343–354 (1998)

    Article  MathSciNet  Google Scholar 

  52. Schaffer, J. D., Morishima, A., “An adaptive crossover distribution mechanism for genetic algorithms,” in Proc. Second Int. Conference on Genetic Algorithms, L. Erlbaum Associates Inc., New York, pp. 36–40, 1987.

  53. Scherbart, A., Nattkemper, T. W., “The Diversity of Regression Ensembles Combining Bagging and Random Subspace Method,” in Advances in Neuro- Information Processing (Kppen, M., Kasabov, N., Coghill, G. eds.), ICONIP 2008, Springer, Heidelberg, pp. 911–918, 2009.

  54. Smith J.E., Fogarty T.C.: “Operator and parameter adaptation in genetic algorithms”. Soft Computing 1(2), 81–87 (1997)

    Article  Google Scholar 

  55. Srinivas M., Patnaik L.M.: “Adaptive Probabilities of Crossover and Mutation in Genetic Algorithms”. IEEE Trans. on Systems, Man, and Cybernetics 24(4), 656–667 (1994)

    Article  Google Scholar 

  56. Tamon, C., Xiang, J., “On the boosting pruning problem,” Proc. 11th European Conference on Machine Learning, pp. 404–412, 2000.

  57. Tang, K., Li, X., Suganthan, P. N., Yang, Z., Weise, T., “Benchmark Functions for the CEC’2010 Special Session and Competition on Large Scale Global Optimization,” Technical Report, Nature Inspired Computation and Applications Laboratory, USTC, China, 2009. http://nical.ustc.edu.cn/cec10ss.php

  58. Thrift, P., “Fuzzy logic synthesis with genetic algorithms,” in Proc. of the Fourth Int. Conference on Genetic Algorithms (ICGA’91) San Diego, pp. 509–513, 1991.

  59. Tsoumakas, G., Partalas, I., Vlahavas, I., “A Taxonomy and Short Review of Ensemble Selection,” in ECAI 2008, Workshop on Supervised and Unsupervised Ensemble Methods and Their Applications Patras, Greece, 2008.

  60. Tsymbal A., Pechenizkiy M., Cunningham P.: “Diversity in search strategies for ensemble feature selection”. Information Fusion 6(1), 83–98 (2005)

    Article  Google Scholar 

  61. Wang L.X., Mendel J.M.: “Generating Fuzzy Rules by Learning from Examples”. IEEE Trans. on Systems, Man and Cybernetics 22(6), 1414–1427 (1992)

    Article  MathSciNet  Google Scholar 

  62. Yao X., Liu Y., Lin G.: “Evolutionary Programming Made Faster”. IEEE Trans. Evol. Comput. 3(2), 9196 (1999)

    Google Scholar 

  63. Yao X., Liu Y.: “Fast evolution strategies”. Contr. Cybern. 26(3), 467–496 (1997)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Wrocław University of Technology, Wybrzeże Wyspiańskiego 27, Wrocław, Poland

    Magdalena Smȩtek & Bogdan Trawiński

Authors
  1. Magdalena Smȩtek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bogdan Trawiński
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Magdalena Smȩtek.

About this article

Cite this article

Smȩtek, M., Trawiński, B. Selection of Heterogeneous Fuzzy Model Ensembles Using Self-adaptive Genetic Algorithms. New Gener. Comput. 29, 309–327 (2011). https://doi.org/10.1007/s00354-010-0305-3

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00354-010-0305-3

Keywords

Search

Navigation