Abstract
In recent years, a number of works proposing the combination of multiple classifiers to produce a single classification have been reported. The resulting classifier, referred to as an ensemble classifier, is generally found to be more accurate than any of the individual classifiers making up the ensemble. In an ensemble of classifiers, it is hoped that each individual classifier will focus on different aspects of the data and error under different circumstances. By combining a set of so-called base classifiers, the deficiencies of each classifier may be compensated by the efficiency of the others. Ensemble pruning deals with the reduction of an ensemble of predictive models in order to improve its efficiency and performance. Ensemble pruning can be considered as an optimization problem. In our work we propose the use of Harmony search, a music inspired algorithm to prune and select the best combination of classifiers. The work is compared with AdaBoost and Bagging among other popular ensemble methods and our method is shown to perform better than the other methods. We have also compared our work with an ensemble pruning technique based on genetic algorithm and our model has shown better accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kuncheva, L.I.: Combining pattern classifiers: methods and algorithms. John Wiley & Sons Inc. (2004)
Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)
Rodriguez, J.J., Kuncheva, L.I., Alonso, C.J.: Rotation forest: A new classifier ensemble method. IEEE Transactionson Pattern Analysis and Machine Intelligence 28(10), 1619–1630 (2006)
Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: A comparison of decision tree ensemble creation techniques. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(1), 173–180 (2007)
Freud, Y., Schapire, R.E.: Experiments with a New Boosting Algorithm. In: Saitta, L. (ed.) Machine Learning: Proceedings of the Thirteenth International Conference (ICML 1996), pp. 148–156. Morgan Kaufmann, San Francisco (1996)
Wolpert, D.H.: Stacked Generalization. Neural Networks 5(2), 241–259
Ho, T.K.: The Random Subspace Method for Constructing Decision Forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)
Prodromidis, A.L., Stolfo, S.J., Chan, P.K.: Effective and efficient pruning of metaclassifiers in a distributed Data Mining system. Technical report CUCS-017-99, Columbia Univ. (1999)
Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A.: Ensemble selection from libraries of models. In: Twenty-First International Conference on Machine Learning, Banff, Alberta, Canada, July 04-08 (2004)
Hu, Q., Yu, D., Xie, Z., Li, X.: EROS: Ensemble rough subspaces. Pattern Recognition 40, 3728–3739 (2007)
Margineantu, D., Dietterich, T.: Pruning adaptive boosting. In: Proc. Fourteenth Intl. Conf. Machine Learning, pp. 211–218 (1997)
Rokach, L., Arbel, R., Maimon, O.: Selective Voting - Getting More For Less in Sensor Fusion. International Journal of Pattern Recognition and Artificial Intelligence 20(3), 329–350 (2006)
Prodromidis, A.L., Stolfo, S.J.: Cost Complexity-Based Pruning of Ensemble Classifiers. Knowl. Inf. Syst. 3(4), 449–469 (2001)
Zhang, Y., Burer, S., Street, W.N.: Ensemble pruning via semi-definite programming. Journal of Machine Learning Research 7, 1315–1338 (2006)
Zhou, Z.H., Tang, W.: Selective Ensemble of Decision Trees. In: Wang, G., et al. (eds.) RSFDGrC 2003. LNCS (LNAI), vol. 2639, pp. 476–483. Springer, Heidelberg (2003)
Tsoumakas, G., Partalas, I., Vlahavas, I.: An Ensemble Pruning Primer. In: Okun, O., Valentini, G. (eds.) SUEMA 2009. SCI, vol. 245, pp. 1–13. Springer, Heidelberg (2009)
Geem, Z.W., Kim, J.H., Loganathan, G.V.: A new heuristic optimization algorithm: Harmony search. Simulation 76(2), 60–68 (2001)
Geem, Z.W. (ed.): Music-Inspired Harmony Search Algorithm: Theory and Applications. SCI. Springer, New York (2009)
Lee, K.S., Geem, Z.W.: A new metaheuristic algorithm for continuous engineering optimization: Harmony search theory and practice. Comput. Methods Appl. Mech. Eng. 194(36-38), 3902–3933 (2004)
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA Data Mining Software: An Update. SIGKDD Explorations 11(1) (2009)
Sylvester, J., Chawla, N.: Evolutionary Ensembles: Combining Learning Agents using Genetic Algorithms. In: Proc. of AAAI Workshop on Multi-agent Systems, pp. 46–51 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sheen, S., Aishwarya, S.V., Anitha, R., Raghavan, S.V., Bhaskar, S.M. (2012). Ensemble Pruning Using Harmony Search. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, SB. (eds) Hybrid Artificial Intelligent Systems. HAIS 2012. Lecture Notes in Computer Science(), vol 7209. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28931-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-28931-6_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28930-9
Online ISBN: 978-3-642-28931-6
eBook Packages: Computer ScienceComputer Science (R0)