Comparison of Bagging, Boosting and Stacking Ensembles Applied to Real Estate Appraisal

  • Magdalena Graczyk
  • Tadeusz Lasota
  • Bogdan Trawiński
  • Krzysztof Trawiński
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5991)


The experiments, aimed to compare three methods to create ensemble models implemented in a popular data mining system called WEKA, were carried out. Six common algorithms comprising two neural network algorithms, two decision trees for regression, linear regression, and support vector machine were used to construct ensemble models. All algorithms were employed to real-world datasets derived from the cadastral system and the registry of real estate transactions. Nonparametric Wilcoxon signed-rank tests to evaluate the differences between ensembles and original models were conducted. The results obtained show there is no single algorithm which produces the best ensembles and it is worth to seek an optimal hybrid multi-model solution.


ensemble models bagging stacking boosting property valuation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Banfield, R.E., et al.: A Comparison of Decision Tree Ensemble Creation Techniques. IEEE Trans. on Pattern Analysis and Machine Intelligence 29(1), 173–180 (2007)CrossRefGoogle Scholar
  2. 2.
    Breiman, L.: Bagging Predictors. Machine Learning 24(2), 123–140 (1996)zbMATHMathSciNetGoogle Scholar
  3. 3.
    Breiman, L.: Stacked Regressions. Machine Learning 24(1), 49–64 (1996)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity Creation Methods: A Survey and Categorisation. Journal of Information Fusion 6(1), 5–20 (2005)CrossRefGoogle Scholar
  5. 5.
    Büchlmann, P., Yu, B.: Analyzing bagging. Annals of Statistics 30, 927–961 (2002)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Chawla, N.V., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: Learning Ensembles From Bites: A Scalable and Accurate Approach. J. of Mach. Learn. Res. 5, 421–451 (2004)MathSciNetGoogle Scholar
  7. 7.
    Cordón, O., Quirin, A.: Comparing Two Genetic Overproduce-and-choose Strategies for Fuzzy Rule-based Multiclassification Systems Generated by Bagging and Mutual Information-based Feature Selection. Int. J. Hybrid Intelligent Systems (2009) (in press)Google Scholar
  8. 8.
    Cunningham, S.J., Frank, E., Hall, M., Holmes, G., Trigg, L., Witten, I.H.: WEKA: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, New Zealand (2005)Google Scholar
  9. 9.
    Freund, Y., Schapire, R.E.: Decision-theoretic generalization of on-line learning and an application to boosting. J. Computer and System Sciences 55(1), 119–139 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Graczyk, M., Lasota, T., Trawiński, B.: Comparative Analysis of Premises Valuation Models Using KEEL, RapidMiner, and WEKA. In: Nguyen, N.T., Kowalczyk, R., Chen, S.-M. (eds.) ICCCI 2009. LNCS, vol. 5796, pp. 800–812. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
    Hansen, L., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(10), 993–1001 (1990)CrossRefGoogle Scholar
  12. 12.
    Hashem, S.: Optimal linear combinations of neural networks. Neural Net. 10(4), 599–614 (1997)CrossRefGoogle Scholar
  13. 13.
    Hernandez-Lobato, D., Martinez-Munoz, G., Suarez, A.: Pruning in ordered regression bagging ensembles. In: Yen, G.G. (ed.) Proceedings of the IEEE World Congress on Computational Intelligence, pp. 1266–1273 (2006)Google Scholar
  14. 14.
    Ho, K.T.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)CrossRefGoogle Scholar
  15. 15.
    Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Advances in Neural Inf. Proc. Systems, pp. 231–238. MIT Press, Cambridge (1995)Google Scholar
  16. 16.
    Król, D., Lasota, T., Trawiński, B., Trawiński, K.: Investigation of Evolutionary Optimization Methods of TSK Fuzzy Model for Real Estate Appraisal. International Journal of Hybrid Intelligent Systems 5(3), 111–128 (2008)zbMATHGoogle Scholar
  17. 17.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, Chichester (2004)zbMATHCrossRefGoogle Scholar
  18. 18.
    Lasota, T., Mazurkiewicz, J., Trawiński, B., Trawiński, K.: Comparison of Data Driven Models for the Validation of Residential Premises Using KEEL. International Journal of Hybrid Intelligent Systems (2009) (in press)Google Scholar
  19. 19.
    Liu, Y., Yao, X.: Ensemble learning via negative correlation. Neural Net. 12, 1399–1404 (1999)CrossRefGoogle Scholar
  20. 20.
    Margineantu, D.D., Dietterich, T.G.: Pruning Adaptive Boosting. In: Proc. 14th Int. Conf. Machine Learning, pp. 211–218 (1997)Google Scholar
  21. 21.
    Opitz, D., Shavlik, J.W.: Actively searching for an effective neural network ensemble. Connection Science 8(3-4), 337–353 (1996)CrossRefGoogle Scholar
  22. 22.
    Polikar, R.: Ensemble Learning. Scholarpedia 4(1), 2776 (2009)CrossRefGoogle Scholar
  23. 23.
    Prodromidis, A.L., Chan, P.K., Stolfo, S.J.: Meta-Learning in a Distributed Data Mining System: Issues and Approaches. In: Kargupta, H., Chan, P.K. (eds.) Advances of Distributed Data Mining. AAAI Press, Menlo Park (2000)Google Scholar
  24. 24.
    Rokach, L.: Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography. Comp. Stat. and Data Anal. 53, 4046–4072 (2009)zbMATHCrossRefGoogle Scholar
  25. 25.
    Schapire, R.E.: The Strength of Weak Learnability. Mach. Learning 5(2), 197–227 (1990)Google Scholar
  26. 26.
    Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  27. 27.
    Wolpert, D.H.: Stacked Generalization. Neural Networks 5(2), 241–259 (1992)CrossRefGoogle Scholar
  28. 28.
    Yao, X., Liu, Y.: Making Use of Population Information in Evolutionary Artificial Neural Networks. IEEE Trans. Systems, Man, and Cybernetics, Part B 28(3), 417–425 (1998)MathSciNetGoogle Scholar
  29. 29.
    Zhou, Z.H., Wu, J., Tang, W.: Ensembling Neural Networks: Many Could Be Better Than All. Artificial Intelligence 137, 239–263 (2002)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Magdalena Graczyk
    • 1
  • Tadeusz Lasota
    • 2
  • Bogdan Trawiński
    • 1
  • Krzysztof Trawiński
    • 3
  1. 1.Institute of InformaticsWrocław University of TechnologyWrocławPoland
  2. 2.Dept. of Spatial ManagementWrocław University of Environmental and Life SciencesWroclawPoland
  3. 3.European Centre for Soft ComputingEdificio Científico-Tecnológico, 3a PlantaMieres, AsturiasSpain

Personalised recommendations