Image Categorization Using ESFS: A New Embedded Feature Selection Method Based on SFS
Feature subset selection is an important subject when training classifiers in Machine Learning (ML) problems. Too many input features in a ML problem may lead to the so-called “curse of dimensionality”, which describes the fact that the complexity of the classifier parameters adjustment during training increases exponentially with the number of features. Thus, ML algorithms are known to suffer from important decrease of the prediction accuracy when faced with many features that are not necessary. In this paper, we introduce a novel embedded feature selection method, called ESFS, which is inspired from the wrapper method SFS since it relies on the simple principle to add incrementally most relevant features. Its originality concerns the use of mass functions from the evidence theory that allows to merge elegantly the information carried by features, in an embedded way, and so leading to a lower computational cost than original SFS. This approach has successfully been applied to the domain of image categorization and has shown its effectiveness through the comparison with other feature selection methods.
KeywordsImage Categorization Feature Selection Evidence Theory
Unable to display preview. Download preview PDF.
- 1.Hall, M.A., Smith, L.A.: Feature Subset Selection: A Correlation Based Filter Approach. In: International Conference on Neural Information Processing and Intelligent Information Systems, pp. 855–858. Springer, Heidelberg (1997)Google Scholar
- 5.Kojadinovic, I., Wottka, T.: Comparison Between a Filter and a Wrapper Approach to Variable Subset Selection in Regression Problems. In: European Symposium on Intelligent Techniques, Aachen, Germany, September 14-15 (2000)Google Scholar
- 6.Arauzo-Azofra, A., Benitez, J.M., Castro, J.L.: A Feature Set Measure Based on Relief. In: Proceedings of the 5th International Conference on Recent Advances in Soft Computing, pp. 104–109. Nottingham (2004)Google Scholar
- 7.Almuallim, H., Dietterich, T.G.: Learning with Many Irrelevant Features. In: Proceedings of the 9th National Conference on Artificial Intelligence, pp. 547–552. AAAI Press, San Jose (1991)Google Scholar
- 13.Somol, P., Pudil, P.: Oscillating Search Algorithms for Feature Selection. In: Proceedings of the 15th International Conference on Pattern Recognition, pp. 406–409 (2000)Google Scholar
- 14.Spence, C., Sajda, P.: The Role of Feature Selection in Building Pattern Recognizers for Computer-aided Diagnosis. In: Proceedings of SPIE. Medical Imaging 1998: Image Processing, vol. 3338, pp. 1434–1441 (1998)Google Scholar
- 15.Quinlan, J.R.: Induction of Decision Trees. Machine Learning 1(1), 81–106 (1986)Google Scholar
- 16.Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Series in Machine Learning (1993)Google Scholar
- 18.Breiman, L., Friedman, J.H., Olshen, R., Stone, C.J.: Classification and Regression Trees. Chapman & Hall/CRC, Boca RatonGoogle Scholar
- 21.Dempster, A.P.: A Generalization of Bayesian Inference. J. Royal Statistical Soc. Series B 30 (1968)Google Scholar
- 24.Rakotomalala, R.: TANAGRA: A free software for the education and the research. In: Actes de EGC 2005, RNTI-E-3, vol. 2, pp. 697–702 (2005)Google Scholar
- 26.Jolliffe, I.T.: Principal Component Analysis. Springer series in statistics (2002)Google Scholar