Pruning Ensembles with Cost Constraints

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9011)


The paper presents a cost-sensitive classifier ensemble pruning method, which employs a genetic algorithm to choose the most promising ensemble. In this study the pruning algorithm considers constraints put on the cost of selected features, which is the one of the key-problems in the real-life decision support systems, especially dedicated medical support systems. The proposed method takes into consideration both the overall classification accuracy and the cost constraints, returning balanced solution for the problem at hand. Additionally, also to boost the value of the exploitation cost, we propose to use cost-sensitive decision trees as the base classifiers. The pruning algorithm was evaluated on the basis of the comprehensive computer experiments run on cost-sensitive medical benchmark datasets.


Pattern classification Machine learning Classifier ensemble Classifier ensemble pruning Cost-sensitive classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



This work was supported by the Polish National Science Center under the grant no. DEC-2013/09/B/ST6/02264.


  1. 1.
    Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: Ensemble diversity measures and their application to thinning. Information Fusion 6(1), 49–62 (2005)CrossRefGoogle Scholar
  2. 2.
    Dai, Q.: A competitive ensemble pruning approach based on cross-validation technique. Knowl.-Based Syst. 37, 394–414 (2013)CrossRefGoogle Scholar
  3. 3.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001)zbMATHGoogle Scholar
  4. 4.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010).
  5. 5.
    Gabrys, B., Ruta, D.: Genetic algorithms in classifier fusion. Appl. Soft Comput. 6(4), 337–347 (2006)CrossRefGoogle Scholar
  6. 6.
    García, S., Fernández, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Inf. Sci. 180(10), 2044–2064 (2010)CrossRefGoogle Scholar
  7. 7.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20, 832–844 (1998)CrossRefGoogle Scholar
  8. 8.
    Jackowski, K., Krawczyk, B., Wozniak, M.: Improved adaptive splitting and selection: The hybrid training method of a classifier based on a feature space partitioning. International Journal of Neural Systems 24(03), 1430007 (2014)CrossRefGoogle Scholar
  9. 9.
    Jackowski, K., Krawczyk, B., Woźniak, M.: Cost-sensitive splitting and selection method for medical decision support system. In: Yin, H., Costa, J.A.F., Barreto, G. (eds.) IDEAL 2012. LNCS, vol. 7435, pp. 850–857. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  10. 10.
    Krawczyk, B., Woźniak, M.: Designing cost-sensitive ensemble – genetic approach. In: Choraś, R.S. (ed.) Image Processing and Communications Challenges 3. AISC, vol. 102, pp. 227–234. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  11. 11.
    Lirov, Y., Yue, O.-C.: Automated network troubleshooting knowledge acquisition. Applied Intelligence 1, 121–132 (1991)CrossRefGoogle Scholar
  12. 12.
    Michalewicz, Z.: Genetic algorithms + data structures = evolution programs, 3rd edn. Springer, London (1996) CrossRefzbMATHGoogle Scholar
  13. 13.
    Núñez, M.: The use of background knowledge in decision tree induction. Mach. Learn. 6(3), 231–250 (1991)Google Scholar
  14. 14.
    Núñez, M.: Economic induction: A case study. In: EWSL, pp. 139–145 (1988)Google Scholar
  15. 15.
    Penar, W., Wozniak, M.: Cost-sensitive methods of constructing hierarchical classifiers. Expert Systems 27(3), 146–155 (2010)CrossRefGoogle Scholar
  16. 16.
    Peng, Y., Huang, Q., Jiang, P., Jiang, J.: Cost-sensitive ensemble of support vector machines for effective detection of microcalcification in breast cancer diagnosis. In: Wang, L., Jin, Y. (eds.) FSKD 2005. LNCS (LNAI), vol. 3614, pp. 483–493. Springer, Heidelberg (2005) CrossRefGoogle Scholar
  17. 17.
    Tan, M., Schlimmer, J.C.: Cost-sensitive concept learning of sensor use in approach and recognition. In: Proceedings of the Sixth International Workshop on Machine Learning, pp. 392–395. Morgan Kaufmann Publishers Inc., San Francisco (1989)Google Scholar
  18. 18.
    Turney, P.D.: Cost-sensitive classification: empirical evaluation of a hybrid genetic decision tree induction algorithm. J. Artif. Int. Res. 2(1), 369–409 (1995)Google Scholar
  19. 19.
    Verdenius, F.: A method for inductive cost optimization. In: Proceedings of the European Working Session on Learning on Machine Learning, EWSL 1991, pp. 179–191. Springer-Verlag New York Inc., New York (1991)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Systems and Computer NetworksWrocław University of TechnologyWrocławPoland

Personalised recommendations