Skip to main content

Ensemble Pruning Using Harmony Search

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 7209)

Abstract

In recent years, a number of works proposing the combination of multiple classifiers to produce a single classification have been reported. The resulting classifier, referred to as an ensemble classifier, is generally found to be more accurate than any of the individual classifiers making up the ensemble. In an ensemble of classifiers, it is hoped that each individual classifier will focus on different aspects of the data and error under different circumstances. By combining a set of so-called base classifiers, the deficiencies of each classifier may be compensated by the efficiency of the others. Ensemble pruning deals with the reduction of an ensemble of predictive models in order to improve its efficiency and performance. Ensemble pruning can be considered as an optimization problem. In our work we propose the use of Harmony search, a music inspired algorithm to prune and select the best combination of classifiers. The work is compared with AdaBoost and Bagging among other popular ensemble methods and our method is shown to perform better than the other methods. We have also compared our work with an ensemble pruning technique based on genetic algorithm and our model has shown better accuracy.

Keywords

  • Ensemble learning
  • Ensemble pruning
  • Harmony search
  • Classification

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Kuncheva, L.I.: Combining pattern classifiers: methods and algorithms. John Wiley & Sons Inc. (2004)

    Google Scholar 

  2. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

    MathSciNet  MATH  Google Scholar 

  3. Rodriguez, J.J., Kuncheva, L.I., Alonso, C.J.: Rotation forest: A new classifier ensemble method. IEEE Transactionson Pattern Analysis and Machine Intelligence 28(10), 1619–1630 (2006)

    CrossRef  Google Scholar 

  4. Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: A comparison of decision tree ensemble creation techniques. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(1), 173–180 (2007)

    CrossRef  Google Scholar 

  5. Freud, Y., Schapire, R.E.: Experiments with a New Boosting Algorithm. In: Saitta, L. (ed.) Machine Learning: Proceedings of the Thirteenth International Conference (ICML 1996), pp. 148–156. Morgan Kaufmann, San Francisco (1996)

    Google Scholar 

  6. Wolpert, D.H.: Stacked Generalization. Neural Networks 5(2), 241–259

    Google Scholar 

  7. Ho, T.K.: The Random Subspace Method for Constructing Decision Forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)

    CrossRef  Google Scholar 

  8. Prodromidis, A.L., Stolfo, S.J., Chan, P.K.: Effective and efficient pruning of metaclassifiers in a distributed Data Mining system. Technical report CUCS-017-99, Columbia Univ. (1999)

    Google Scholar 

  9. Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A.: Ensemble selection from libraries of models. In: Twenty-First International Conference on Machine Learning, Banff, Alberta, Canada, July 04-08 (2004)

    Google Scholar 

  10. Hu, Q., Yu, D., Xie, Z., Li, X.: EROS: Ensemble rough subspaces. Pattern Recognition 40, 3728–3739 (2007)

    CrossRef  MATH  Google Scholar 

  11. Margineantu, D., Dietterich, T.: Pruning adaptive boosting. In: Proc. Fourteenth Intl. Conf. Machine Learning, pp. 211–218 (1997)

    Google Scholar 

  12. Rokach, L., Arbel, R., Maimon, O.: Selective Voting - Getting More For Less in Sensor Fusion. International Journal of Pattern Recognition and Artificial Intelligence 20(3), 329–350 (2006)

    CrossRef  Google Scholar 

  13. Prodromidis, A.L., Stolfo, S.J.: Cost Complexity-Based Pruning of Ensemble Classifiers. Knowl. Inf. Syst. 3(4), 449–469 (2001)

    CrossRef  MATH  Google Scholar 

  14. Zhang, Y., Burer, S., Street, W.N.: Ensemble pruning via semi-definite programming. Journal of Machine Learning Research 7, 1315–1338 (2006)

    MathSciNet  MATH  Google Scholar 

  15. Zhou, Z.H., Tang, W.: Selective Ensemble of Decision Trees. In: Wang, G., et al. (eds.) RSFDGrC 2003. LNCS (LNAI), vol. 2639, pp. 476–483. Springer, Heidelberg (2003)

    Google Scholar 

  16. Tsoumakas, G., Partalas, I., Vlahavas, I.: An Ensemble Pruning Primer. In: Okun, O., Valentini, G. (eds.) SUEMA 2009. SCI, vol. 245, pp. 1–13. Springer, Heidelberg (2009)

    CrossRef  Google Scholar 

  17. Geem, Z.W., Kim, J.H., Loganathan, G.V.: A new heuristic optimization algorithm: Harmony search. Simulation 76(2), 60–68 (2001)

    CrossRef  Google Scholar 

  18. Geem, Z.W. (ed.): Music-Inspired Harmony Search Algorithm: Theory and Applications. SCI. Springer, New York (2009)

    Google Scholar 

  19. Lee, K.S., Geem, Z.W.: A new metaheuristic algorithm for continuous engineering optimization: Harmony search theory and practice. Comput. Methods Appl. Mech. Eng. 194(36-38), 3902–3933 (2004)

    CrossRef  Google Scholar 

  20. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA Data Mining Software: An Update. SIGKDD Explorations 11(1) (2009)

    Google Scholar 

  21. http://repository.seasr.org/Datasets/UCI/arff/

  22. Sylvester, J., Chawla, N.: Evolutionary Ensembles: Combining Learning Agents using Genetic Algorithms. In: Proc. of AAAI Workshop on Multi-agent Systems, pp. 46–51 (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Sheen, S., Aishwarya, S.V., Anitha, R., Raghavan, S.V., Bhaskar, S.M. (2012). Ensemble Pruning Using Harmony Search. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, SB. (eds) Hybrid Artificial Intelligent Systems. HAIS 2012. Lecture Notes in Computer Science(), vol 7209. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28931-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-28931-6_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-28930-9

  • Online ISBN: 978-3-642-28931-6

  • eBook Packages: Computer ScienceComputer Science (R0)