Multi Objective Genetic Programming for Feature Construction in Classification Problems

  • Mauro Castelli
  • Luca Manzoni
  • Leonardo Vanneschi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6683)


This work introduces a new technique for features construction in classification problems by means of multi objective genetic programming (MOGP). The final goal is to improve the generalization ability of the final classifier. MOGP can help in finding solutions with a better generalization ability with respect to standard genetic programming as stated in [1]. The main issue is the choice of the criteria that must be optimized by MOGP. In this work the construction of new features is guided by two criteria: the first one is the entropy of the target classes as in [7] while the second is inspired by the concept of margin used in support vector machines.


Support Vector Machine Genetic Programming Target Class Feature Construction Good Generalization Ability 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Castelli, M., Manzoni, L., Silva, S., Vanneschi, L.: A comparison of the generalization ability of different genetic programming frameworks. In: WCCI 2010: Proceedings of IEEE World Congress on Computational Intelligence. Springer, Heidelberg (2010)Google Scholar
  2. 2.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast elitist multi-objective genetic algorithm: Nsga-ii. IEEE Transactions on Evolutionary Computation 6, 182–197 (2000)CrossRefGoogle Scholar
  3. 3.
    Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97(1-2), 273–324 (1997)CrossRefzbMATHGoogle Scholar
  4. 4.
    Krawiec, K.: Genetic programming-based construction of features for machine learning and knowledge discovery tasks. Genetic Programming and Evolvable Machines 3(4), 329–343 (2002)CrossRefzbMATHGoogle Scholar
  5. 5.
    Lee, C., Lee, G.G.: Information gain and divergence-based feature selection for machine learning-based text categorization. Inf. Process. Manage. 42(1), 155–165 (2006)CrossRefGoogle Scholar
  6. 6.
    Neshatian, K., Zhang, M.: Genetic programming and class-wise orthogonal transformation for dimension reduction in classification problems. In: O’Neill, M., Vanneschi, L., Gustafson, S., Esparcia Alcázar, A.I., De Falco, I., Della Cioppa, A., Tarantino, E. (eds.) EuroGP 2008. LNCS, vol. 4971, pp. 242–253. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Neshatian, K., Zhang, M., Johnston, M.: Feature construction and dimension reduction using genetic programming. In: Orgun, M.A., Thornton, J. (eds.) AI 2007. LNCS (LNAI), vol. 4830, pp. 160–170. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Poli, R., Langdon, W.B., McPhee, N.F.: A field guide to genetic programming (2008),,
  9. 9.
    Pollard, J.H.: A handbook of numerical and statistical techniques. Cambridge University Press, Cambridge (1977)CrossRefzbMATHGoogle Scholar
  10. 10.
    Shannon, C.E.: A mathematical theory of communication. SIGMOBILE Mob. Comput. Commun. Rev. 5(1), 3–55 (2001)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Smith, M.G., Bull, L.: Genetic programming with a genetic algorithm for feature construction and selection. Genetic Programming and Evolvable Machines 6(3), 265–281 (2005)CrossRefGoogle Scholar
  12. 12.
    Zhang, Y., Rockett, P.: Domain-independent feature extraction for multi-classification using multi-objective genetic programming. Pattern Analysis & Applications 13, 273–288 (2010), 10.1007/s10044-009-0154-1MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Mauro Castelli
    • 1
  • Luca Manzoni
    • 1
  • Leonardo Vanneschi
    • 1
  1. 1.Dipartimento di Informatica, Sistemistica e Comunicazione (DISCo)Università degli Studi di Milano-BicoccaMilanItaly

Personalised recommendations