Obtaining Pareto Front in Instance Selection with Ensembles and Populations

  • Mirosław KordosEmail author
  • Marcin Wydrzyński
  • Krystian Łapa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10841)


Collective computational intelligence can be used in several ways, for example as taking the decision together by some form of a bagging ensemble or as finding the solutions by multi-objective evolutionary algorithms. In this paper we examine and compare the application of the two approaches to instance selection for creating the Pareto front of the selected subsets, where the two objectives are classification accuracy and data size reduction. As the bagging ensemble members we use DROP5 algorithms. The evolutionary algorithm is based on NSGA-II. The findings are that the evolutionary approach is faster (contrary to the popular belief) and usually provides better quality solutions, with some exceptions, were the outcome of the DROP5 ensemble is better.



This work was supported by the NCN (Polish National Science Center) grant “Evolutionary Methods in Data Selection” No. 2017/01/X/ST6/00202.


  1. 1.
    Kordos, M.: Data selection for neural networks. Schedae Informaticae 25, 153–164 (2017)Google Scholar
  2. 2.
    Arnaiz-González, Á., Blachnik, M., Kordos, M., García-Osorio, C.: Fusion of instance selection methods in regression tasks. Inf. Fusion 30, 69–79 (2016)CrossRefGoogle Scholar
  3. 3.
    Blachnik, M.: Ensembles of instance selection methods based on feature subset. IEEE Proc. Comput. Sci. 35, 388–396 (2014)CrossRefGoogle Scholar
  4. 4.
    Deb, K.: Multi-Objective Optimization using Evolutionary Algorithms. Wiley, Hoboken (2001)zbMATHGoogle Scholar
  5. 5.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, Hoboken (2004)CrossRefGoogle Scholar
  6. 6.
    Tomek, I.: An experiment with the edited nearest-neighbor rule. IEEE Trans. Syst. Man Cybern. 6, 448–452 (1976)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Sebban, M., et al.: Stopping criterion for boosting based data reduction techniques: From binary to multiclass problem. J. Mach. Learn. Res. 3, 863–885 (2002)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Garcia-Pedrajas, N.: Constructing ensembles of classifiers by means of weighted instance selection. IEEE Trans. Neural Netw. 20, 258–277 (2009)CrossRefGoogle Scholar
  9. 9.
    Blachnik, M., Kordos, M.: Bagging of instance selection algorithms. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2014. LNCS (LNAI), vol. 8468, pp. 40–51. Springer, Cham (2014). Scholar
  10. 10.
    García-Pedrajas, N., De Haro-García, A.: Boosting instance selection algorithms. Knowl.-Based Syst. 67, 342–360 (2014)CrossRefGoogle Scholar
  11. 11.
    Wilson, D.R., Martinez, T.R.: Reduction techniques for instance-based learning algorithms. Mach. Learn. 38, 257–286 (2000)CrossRefGoogle Scholar
  12. 12.
    Olvera-López, A., Carrasco-Ochoa, J., Martínez-Trinidad, F., Kittler, J.: A review of instance selection methods. Artif. Intell. Rev. 34(2), 133–143 (2010)CrossRefGoogle Scholar
  13. 13.
    Garcia, S., Derrac, J., Cano, J.R., Herrera, F.: Prototype selection for nearest neighbor classification: Taxonomy and empirical study. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 417–435 (2012)CrossRefGoogle Scholar
  14. 14.
    Goldberg, D.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley, Boston (1989)zbMATHGoogle Scholar
  15. 15.
    Lobo, F.G., Lima, C.F., Michalewicz, Z.: Parameter Setting in Evolutionary Algorithms. Studies in Computational Intelligence, vol. 54. Springer, Heidelberg (2007). Scholar
  16. 16.
    Konak, A., Coit, D., Smith, A.: Multi-objective optimization using genetic algorithms: A tutorial. Reliab. Eng. Syst. Safety 91, 992–1007 (2006)CrossRefGoogle Scholar
  17. 17.
    Antonelli, M., Ducange, P., Marcelloni, F.: Genetic training instance selection in multiobjective evolutionary fuzzy systems: A coevolutionary approach. IEEE Trans. Fuzzy Syst. 20(2), 276–290 (2012)CrossRefGoogle Scholar
  18. 18.
    Tsaia, C.-F., Eberleb, W., Chu, C.-Y.: Genetic algorithms in feature and instance selection. Knowl.-Based Syst. 39, 240–247 (2013)CrossRefGoogle Scholar
  19. 19.
    Cano, J.R., Herrera, F., Lozano, M.: Using evolutionary algorithms as instance selection for data reduction in KDD: An experimental study. IEEE Trans. Evol. Comput. 7(6), 561–575 (2003)CrossRefGoogle Scholar
  20. 20.
    Cano, J.R., Herrera, F., Lozano, M.: Instance selection using evolutionary algorithms: an experimental study. In: Pal, N.R., Jain, L. (eds.) Advanced Information and Knowledge Processing, pp. 127–152. Springer, London (2004). Scholar
  21. 21.
    Derrac, J., et al.: Enhancing evolutionary instance selection algorithms by means of fuzzy rough set based feature selection. Inf. Sci. 186, 73–92 (2012)CrossRefGoogle Scholar
  22. 22.
    Kordos, M.: Optimization of evolutionary instance selection. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2017. LNCS (LNAI), vol. 10245, pp. 359–369. Springer, Cham (2017). Scholar
  23. 23.
    Łapa, K., Cpałka, K., Hayashi, Y.: Hybrid initialization in the process of evolutionary learning. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2017. LNCS (LNAI), vol. 10245, pp. 380–393. Springer, Cham (2017). Scholar
  24. 24.
    Horoba, C., Numann, F.: Benefits and drawbacks for the use of e-dominance in evolutionary multi-objective optimization. In: Genetic and Evolutionary Computation Conference. ACM Press, pp. 641–680 (2008)Google Scholar
  25. 25.
    Alcala-Fdez, J., et al.: KEEL Data-Mining Software Tool and Data Set Repository (2017)
  26. 26.
    Arnaiz-González, Á., Díez-Pastor, J.F., Rodríguez, J.J., García-Osorio, C.: Instance selection for regression: Adapting DROP. Neurocomputing 201, 66–81 (2016)CrossRefGoogle Scholar
  27. 27.
    Kordos, M., Blachnik, M., Perzyk, M., Kozłowski, J., Bystrzycki, O., Gródek, M., Byrdziak, A., Motyka, Z.: A hybrid system with regression trees in steel-making process. In: Corchado, E., Kurzyński, M., Woźniak, M. (eds.) HAIS 2011. LNCS (LNAI), vol. 6678, pp. 222–230. Springer, Heidelberg (2011). Scholar
  28. 28.
    Kordos, M., Duch, W.: Variable step search algorithm for MLP training. In: The 8th IASTED International Conference on Artificial Intelligence and Soft Computing, Marbella, pp. 215–220, September 2004Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Mirosław Kordos
    • 1
    Email author
  • Marcin Wydrzyński
    • 1
  • Krystian Łapa
    • 2
  1. 1.Department of Computer Science and AutomaticsUniversity of Bielsko-BialaBielsko-BiałaPoland
  2. 2.Częstochowa University of Technology, Institute of Computational IntelligenceCzęstochowaPoland

Personalised recommendations