An iterative genetic programming approach to prototype generation

Abstract

In this paper, we propose a genetic programming (GP) approach to the problem of prototype generation for nearest-neighbor (NN) based classification. The problem consists of learning a set of artificial instances that effectively represents the training set of a classification problem, with the goal of reducing the storage requirements and the computational cost inherent in NN classifiers. This work introduces an iterative GP technique to learn such artificial instances based on a non-linear combination of instances available in the training set. Experiments are reported in a benchmark for prototype generation. Experimental results show our approach is very competitive with the state of the art, in terms of accuracy and in its ability to reduce the training set size.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    A mechanism that consists of changing the class labels of samples from \({\mathcal {T}}\), which could be suspicious of having errors, and belonging to other different classes.

  2. 2.

    Please note that this method is hybrid, in the sense that it performs PS (with the SSMA method), followed by PG (with the SFLSDE technique) [18]. Therefore, it may be considered an unfair comparison with the other techniques (which only perform PG). Anyway, we included this method because, to the best of our knowledge, is the method that has obtained the highest performance in the data sets we considered. Also please note that there is a wide variety of additional techniques that could be included (e.g., IPADE [19]). However, we restricted ourselves to those methods that have reported the most competitive performance recently.

  3. 3.

    PG methods aim to obtain representative instances of the Training set that they are smaller than the original one, such that the accuracy is not affected [23].

References

  1. 1.

    J. Alcalá-Fdez, A. Fernandez, J. Luengo, J. Derrac, S. García, L. Sánchez, F. Herrera, KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. Multiple-Valued Logic Soft Comput. 17(2–3), 255–287 (2011)

    Google Scholar 

  2. 2.

    K. Bache, M. Lichman, UCI Machine Learning Repository (University of California, Irvine, School of Information and Computer Sciences, 2013)

  3. 3.

    L. Cordelia, C. De Stefano, F. Fontanella, A. Marcelli, Genetic programming for generating prototypes in classification problems, in The 2005 IEEE Congress on Evolutionary Computation, vol. 2 (2005), pp. 1149–1155

  4. 4.

    L. Cordella, C. De Stefano, F. Fontanella, A. Marcelli, Looking for prototypes by genetic programming, in Advances in Machine Vision, Image Processing, and Pattern Analysis of Lecture Notes in Computer Science, vol. 4153, ed. by N. Zheng, X. Jiang, X. Lan (Springer, Berlin, 2006), pp. 152–159

    Google Scholar 

  5. 5.

    R. Duda, P. Hart, D. Stork, Pattern Classification, 2nd edn. (Wiley, New York, 2000)

    Google Scholar 

  6. 6.

    H.J. Escalante, M. Graff, A. Morales-Reyes, PGGP: prototype generation via genetic programming. Appl. Soft Comput. 40, 569–580 (2016)

    Article  Google Scholar 

  7. 7.

    H.J. Escalante, M. Marin-Castro, A. Morales-Reyes, M. Graff, A. Rosales-Perez, M.M. y Gomez, C.A. Reyes, J.A. Gonzalez, MOPG: a multi-objective evolutionary algorithm for prototype generation. Pattern Anal. Appl. (2015). doi:10.1007/s10044-015-0454-6

    Google Scholar 

  8. 8.

    F. Fernandez, P. Isasi, Evolutionary design of nearest prototype classifiers. J. Heuristics 10, 431–454 (2004)

    Article  Google Scholar 

  9. 9.

    Y.B. Fernandez-Hernandez, R. Bello, Y. Filiberto, M. Frias, L. Coello-Blanco, Y. Caballero, An approach for prototype generation based on similarity relations for problems of classification. Computación y Sistemas 19(1), 109–118 (2015)

    Article  Google Scholar 

  10. 10.

    U. Garain, Prototype reduction using an artificial immune model. Pattern Anal. Appl. 11(3–4), 353–363 (2008a)

    MathSciNet  Article  Google Scholar 

  11. 11.

    U. Garain, Prototype reduction using an artificial immune system. Pattern Anal. Appl. 11(3–4), 353–363 (2008b)

    MathSciNet  Article  Google Scholar 

  12. 12.

    S. Holm, A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6(2), 65–70 (1979); 10011

  13. 13.

    J. Koplowitz, T.A. Brown, On the relation of performance to editing in nearest neighbor rules. Pattern Recogn. 13(3), 251–255 (1981)

    Article  Google Scholar 

  14. 14.

    J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT Press, Cambridge, 1992)

    Google Scholar 

  15. 15.

    J.A. Olvera-Lopez, J.A. Carrasco-Ochoa, J.F. Martinez-Trinidad, Prototype selection methods. Computación y Sistemas 13(4), 449–462 (2010)

    Google Scholar 

  16. 16.

    R. Poli, W.B. Langdon, N.F. McPhee, A Field Guide to Genetic Programming. Published via http://lulu.com and freely available at http://www.gp-field-guide.org.uk. (With contributions by J. R. Koza) (2008)

  17. 17.

    I. Triguero, J. Derrac, S. Garcia, F. Herrera, A taxonomy and experimental study on prototype generation for nearest neighbor classification. IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 42(1), 86–100 (2012)

    Article  Google Scholar 

  18. 18.

    I. Triguero, S. Garca, F. Herrera, Differential evolution for optimizing the positioning of prototypes in nearest neighbor classification. Pattern Recogn. 44(4), 901–916 (2011a)

    Article  Google Scholar 

  19. 19.

    I. Triguero, S. Garcia, F. Herrera, IPADE: iterative prototype adjustment for nearest neighbor classification. IEEE Trans. Neural Netw. 21(12), 1984–1990 (2010)

    Article  Google Scholar 

  20. 20.

    I. Triguero, S. Garcia, F. Herrera, Differential evolution for optimizing the positioning of prototypes in nearest neighbor classification. Pattern Recogn. 44, 901–916 (2011b)

    Article  Google Scholar 

  21. 21.

    I. Triguero, D. Peralta, J. Bacardit, S. Garcia, F. Herrera, MRPR: a MapReduce solution for prototype reduction in big data classification. Neurocomputing 150, 331–345 (2015)

    Article  Google Scholar 

  22. 22.

    F. Wilcoxon, Individual comparisons by ranking methods. Biom. Bull. 1(6), 80 (1945)

    Article  Google Scholar 

  23. 23.

    D. Wilson, T. Martinez, Reduction techniques for instance-based learning algorithms. Mach. Learn. 38(3), 257–286 (2000)

    Article  MATH  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mario Graff.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Valencia-Ramírez, J.M., Graff, M., Escalante, H.J. et al. An iterative genetic programming approach to prototype generation. Genet Program Evolvable Mach 18, 123–147 (2017). https://doi.org/10.1007/s10710-016-9279-3

Download citation

Keywords

  • Genetic programming
  • K-nearest neighbors
  • Prototype generation
  • Pattern classification