Genetic Programming for Feature Subset Ranking in Binary Classification Problems

  • Kourosh Neshatian
  • Mengjie Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5481)


We propose a genetic programming (GP) system for measuring the relevance of subsets of features in binary classification tasks. A virtual program structure and an evaluation function are defined in a way that constructed GP programs can measure the goodness of subsets of features. The proposed system can detect relevant subsets of features in different situations including multimodal class distributions and mutually correlated features where other ranking methods have difficulties. Our empirical results indicate that the proposed system is good at ranking subsets and giving insight into the actual classification performance. The proposed ranking system is also efficient in terms of feature selection.


Feature Selection Genetic Programming Information Gain Feature Subset Relevance Measure 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jong, K., Mary, J., Cornuéjols, A., Marchiori, E., Sebag, M.: Ensemble feature ranking. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) PKDD 2004. LNCS, vol. 3202, pp. 267–278. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Ruiz, R., Riquelme, J.C., Aguilar-Ruiz, J.S.: Fast feature ranking algorithm. In: Knowledge-Based Intelligent Information and Engineering Systems, pp. 325–331 (2003)Google Scholar
  3. 3.
    Biesiada, J., Duch, W., Kachel, A., Maczka, K., Palucha, S.: Feature ranking methods based on information entropy with parzen windows. In: International Conference on Research in Electrotechnology and Applied Informatics (REI 2005), pp. 109–119 (2005)Google Scholar
  4. 4.
    Lin, T.H., Chiu, S.H., Tsai, K.C.: Supervised feature ranking using a genetic algorithm optimized artificial neural network. Journal of Chemical Information and Modeling 46, 1604–1614 (2006)CrossRefGoogle Scholar
  5. 5.
    Cheng, Q., Varshney, P., Arora, M.: Logistic regression for feature selection and soft classification of remote sensing data. Geoscience and Remote Sensing Letters 3, 491–494 (2006)CrossRefGoogle Scholar
  6. 6.
    Neshatian, K., Zhang, M.: Genetic programming for feature ranking in classification problems. In: Li, X., et al. (eds.) SEAL 2008. LNCS, vol. 5361, pp. 544–554. Springer, Heidelberg (2008)Google Scholar
  7. 7.
    Davis, R.A., Charlton, A.J., Oehlschlager, S., Wilson, J.C.: Novel feature selection method for genetic programming using metabolomic 1h NMR data. Chemometrics and Intelligent Laboratory Systems 81, 50–59 (2006)CrossRefGoogle Scholar
  8. 8.
    Liu, H., Yu, L.: Toward integrating feature selection algorithms for classification and clustering. IEEE Trans. on Knowl. and Data Eng. 17(4), 491–502 (2005)CrossRefGoogle Scholar
  9. 9.
    Lin, J.Y., Ke, H.R., Chien, B.C., Yang, W.P.: Classifier design with feature selection and feature extraction using layered genetic programming. Expert Syst. Appl. 34(2), 1384–1393 (2008)CrossRefGoogle Scholar
  10. 10.
    Jolliffe, I.T.: Principal Component Analysis (2002)Google Scholar
  11. 11.
    Hall, M.A.: Correlation-based feature selection for discrete and numeric class machine learning. In: Proceedings of the Seventeenth International Conference on Machine Learning table of contents, pp. 359–366. Morgan Kaufmann Publishers Inc., San Francisco (2000)Google Scholar
  12. 12.
    Agresti, A., Agresti, A.: Categorical Data Analysis. Wiley, Chichester (2003)zbMATHGoogle Scholar
  13. 13.
    Asuncion, A., Newman, D.: UCI machine learning repository (2007),
  14. 14.
    Koza, J.R.: Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press, Cambridge (1994)zbMATHGoogle Scholar
  15. 15.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  16. 16.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  17. 17.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  18. 18.
    Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.K.: Improvements to platt’s SMO algorithm for SVM classifier design. Neural Comp. 13, 637–649 (2001)CrossRefzbMATHGoogle Scholar
  19. 19.
    Lowry, R.: Concepts and Applications of Inferential Statistics. VassarStat (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Kourosh Neshatian
    • 1
  • Mengjie Zhang
    • 1
  1. 1.School of Engineering and Computer ScienceVictoria University of WellingtonWellingtonNew Zealand

Personalised recommendations