Abstract
The experiments, aimed to compare three methods to create ensemble models implemented in a popular data mining system called WEKA, were carried out. Six common algorithms comprising two neural network algorithms, two decision trees for regression, linear regression, and support vector machine were used to construct ensemble models. All algorithms were employed to real-world datasets derived from the cadastral system and the registry of real estate transactions. Nonparametric Wilcoxon signed-rank tests to evaluate the differences between ensembles and original models were conducted. The results obtained show there is no single algorithm which produces the best ensembles and it is worth to seek an optimal hybrid multi-model solution.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Banfield, R.E., et al.: A Comparison of Decision Tree Ensemble Creation Techniques. IEEE Trans. on Pattern Analysis and Machine Intelligence 29(1), 173–180 (2007)
Breiman, L.: Bagging Predictors. Machine Learning 24(2), 123–140 (1996)
Breiman, L.: Stacked Regressions. Machine Learning 24(1), 49–64 (1996)
Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity Creation Methods: A Survey and Categorisation. Journal of Information Fusion 6(1), 5–20 (2005)
Büchlmann, P., Yu, B.: Analyzing bagging. Annals of Statistics 30, 927–961 (2002)
Chawla, N.V., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: Learning Ensembles From Bites: A Scalable and Accurate Approach. J. of Mach. Learn. Res. 5, 421–451 (2004)
Cordón, O., Quirin, A.: Comparing Two Genetic Overproduce-and-choose Strategies for Fuzzy Rule-based Multiclassification Systems Generated by Bagging and Mutual Information-based Feature Selection. Int. J. Hybrid Intelligent Systems (2009) (in press)
Cunningham, S.J., Frank, E., Hall, M., Holmes, G., Trigg, L., Witten, I.H.: WEKA: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, New Zealand (2005)
Freund, Y., Schapire, R.E.: Decision-theoretic generalization of on-line learning and an application to boosting. J. Computer and System Sciences 55(1), 119–139 (1997)
Graczyk, M., Lasota, T., Trawiński, B.: Comparative Analysis of Premises Valuation Models Using KEEL, RapidMiner, and WEKA. In: Nguyen, N.T., Kowalczyk, R., Chen, S.-M. (eds.) ICCCI 2009. LNCS, vol. 5796, pp. 800–812. Springer, Heidelberg (2009)
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(10), 993–1001 (1990)
Hashem, S.: Optimal linear combinations of neural networks. Neural Net. 10(4), 599–614 (1997)
Hernandez-Lobato, D., Martinez-Munoz, G., Suarez, A.: Pruning in ordered regression bagging ensembles. In: Yen, G.G. (ed.) Proceedings of the IEEE World Congress on Computational Intelligence, pp. 1266–1273 (2006)
Ho, K.T.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Advances in Neural Inf. Proc. Systems, pp. 231–238. MIT Press, Cambridge (1995)
Król, D., Lasota, T., Trawiński, B., Trawiński, K.: Investigation of Evolutionary Optimization Methods of TSK Fuzzy Model for Real Estate Appraisal. International Journal of Hybrid Intelligent Systems 5(3), 111–128 (2008)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, Chichester (2004)
Lasota, T., Mazurkiewicz, J., Trawiński, B., Trawiński, K.: Comparison of Data Driven Models for the Validation of Residential Premises Using KEEL. International Journal of Hybrid Intelligent Systems (2009) (in press)
Liu, Y., Yao, X.: Ensemble learning via negative correlation. Neural Net. 12, 1399–1404 (1999)
Margineantu, D.D., Dietterich, T.G.: Pruning Adaptive Boosting. In: Proc. 14th Int. Conf. Machine Learning, pp. 211–218 (1997)
Opitz, D., Shavlik, J.W.: Actively searching for an effective neural network ensemble. Connection Science 8(3-4), 337–353 (1996)
Polikar, R.: Ensemble Learning. Scholarpedia 4(1), 2776 (2009)
Prodromidis, A.L., Chan, P.K., Stolfo, S.J.: Meta-Learning in a Distributed Data Mining System: Issues and Approaches. In: Kargupta, H., Chan, P.K. (eds.) Advances of Distributed Data Mining. AAAI Press, Menlo Park (2000)
Rokach, L.: Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography. Comp. Stat. and Data Anal. 53, 4046–4072 (2009)
Schapire, R.E.: The Strength of Weak Learnability. Mach. Learning 5(2), 197–227 (1990)
Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Wolpert, D.H.: Stacked Generalization. Neural Networks 5(2), 241–259 (1992)
Yao, X., Liu, Y.: Making Use of Population Information in Evolutionary Artificial Neural Networks. IEEE Trans. Systems, Man, and Cybernetics, Part B 28(3), 417–425 (1998)
Zhou, Z.H., Wu, J., Tang, W.: Ensembling Neural Networks: Many Could Be Better Than All. Artificial Intelligence 137, 239–263 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Graczyk, M., Lasota, T., Trawiński, B., Trawiński, K. (2010). Comparison of Bagging, Boosting and Stacking Ensembles Applied to Real Estate Appraisal. In: Nguyen, N.T., Le, M.T., Świątek, J. (eds) Intelligent Information and Database Systems. ACIIDS 2010. Lecture Notes in Computer Science(), vol 5991. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12101-2_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-12101-2_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12100-5
Online ISBN: 978-3-642-12101-2
eBook Packages: Computer ScienceComputer Science (R0)