Genetic Programming for Feature Ranking in Classification Problems
Feature ranking (FR) provides a measure of usefulness for the attributes of a classification task. Most existing FR methods focus on the relevance of a single feature to the class labels. Here, we use GP to see how a set of features can contribute towards discriminating different classes and then we score the participating features accordingly. The scoring mechanism is based on the frequency of appearance of each feature in a collection of GP programs and the fitness of those programs. Our results show that the proposed FR method can detect important features of a problem. A variety of different classifiers restricted to just a few of these high-ranked features work well. The ranking mechanism can also shrink the search space of size O(2 n ) of subsets of features to a search space of size O(n) in which there are points that may improve the classification performance.
KeywordsFeature Selection Genetic Programming Class Label Feature Ranking Decision Stump
Unable to display preview. Download preview PDF.
- 2.Ruiz, R., Riquelme, J.C., Aguilar-Ruiz, J.S.: Fast feature ranking algorithm. Knowledge-Based Intelligent Information and Engineering Systems, 325–331 (2003)Google Scholar
- 3.Biesiada, J., Duch, W., Kachel, A., Maczka, K., Palucha, S.: Feature ranking methods based on information entropy with parzen windows. In: International Conference on Research in Electrotechnology and Applied Informatics (REI 2005), pp. 109–119 (2005)Google Scholar
- 5.Oh, I.S., Lee, J.S., Moon, B.R.: Hybrid genetic algorithms for feature selection. IEEE Transactions on Pattern Analysis and Machine Intellignece, 1424–1437 (2004)Google Scholar
- 11.Landry, J.A., Costa, L.D., Bernier, T.: Discriminant feature selection by genetic programming: Towards a domain independent multi-class object detection system. Systemics, Cybernetics and Informatics 3, 76–81 (2006)Google Scholar
- 12.Parrott, D., Li, X., Ciesielski, V.: Multi-objective techniques in genetic programming for evolving classifiers. In: The 2005 IEEE Congress on Evolutionary Computation, 2005, vol. 2, pp. 1141–1148 (2005)Google Scholar
- 14.Asuncion, A., Newman, D.: Uci machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
- 16.Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
- 18.John, G., Langley, P.: Estimating continuous distributions in bayesian classifiers, pp. 338–345 (1995)Google Scholar