Multi-objective Feature Selection with NSGA II

  • Tarek M. Hamdani
  • Jin-Myung Won
  • Adel M. Alimi
  • Fakhri Karray
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4431)

Abstract

This paper deals with the multi-objective definition of the feature selection problem for different pattern recognition domains. We use NSGA II the latest multi-objective algorithm developed for resolving problems of multi-objective aspects with more accuracy and a high convergence speed. We define the feature selection as a problem including two competing objectives and we try to find a set of optimal solutions so called Pareto-optimal solutions instead of a single optimal solution. The two competing objectives are the minimization of both the number of used features and the classification error using 1-NN classifier. We apply our method to five databases selected from the UCI repository and we report the results on these databases. We present the convergence of the NSGA II on different problems and discuss the behavior of NSGA II on these different contexts.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Dash, M., Liu, H.: Feature Selection for Classification. Intelligent Data Analysis 1, 131–156 (1997)CrossRefGoogle Scholar
  2. 2.
    Kim, G., Kim, S.: Feature Selection Using Genetic Algorithms for Handwritten character Recognition. In: 7th Int’l Workshop on Frontiers in Handwriting Recognition, Amsterdam, pp. 103–112 (2000)Google Scholar
  3. 3.
    Oh, I.-S., Lee, J.-S., Moon, B.-R.: Hybrid Genetic Algorithms for Feature Selection. IEEE Transaction on Pattern Analysis and Machine Intelligence 26(11) (2004)Google Scholar
  4. 4.
    Hamdani, M.T., Alimi, M.A., Karray, F.: Distributed Genetic Algorithm with Bi-Coded Chromosomes and a New Evaluation Function for Features Selection. In: Proc. IEEE Congress on Evolutionary Computation, pp. 2596–2603 (2006)Google Scholar
  5. 5.
    Cvetkovic, D., Parmee, I.C.: Preferences and their application in evolutionary multiobjective optimization. IEEE Transactions on Evolutionary Computation 6(1), 42–57 (2002)CrossRefGoogle Scholar
  6. 6.
    Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., da Fonseca, V.G.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Transactions on Evolutionary Computation 7(2), 117–132 (2003)CrossRefGoogle Scholar
  7. 7.
    Morita, M., Oliveira, L.S., Sabourin, R.: Unsupervised feature selection for ensemble of classifiers. In: Ninth International Workshop on Frontiers in Handwriting Recognition, IWFHR-9 2004, 26-29 Oct. 2004, pp. 81–86 (2004)Google Scholar
  8. 8.
    Morita, M., Sabourin, R., Bortolozzi, F., Suen, C.Y.: Unsupervised feature selection using multi-objective genetic algorithms for handwritten word recognition. In: Proceedings of Seventh International Conference on Document Analysis and Recognition, 3-6 Aug. 2003, pp. 666–670 (2003)Google Scholar
  9. 9.
    Zio, E., Baraldi, P., Pedroni, N.: Selecting features nuclear transients classification by means of genetic algorithms. IEEE Transactions on Nuclear Science - Part 3 53(3), 1479–1493 (2006)CrossRefGoogle Scholar
  10. 10.
    Devogelaere, D., Rijckaert, M.: Scalars, a way to improve the multi-objective prediction of the GAdC-method. In: Proceedings of Sixth Brazilian Symposium on Neural Networks, 22-25 Nov. 2000, pp. 56–60 (2000)Google Scholar
  11. 11.
    Lac, H.C., Stacey, D.A.: Feature subset selection via multi-objective genetic algorithm. In: Proceedings of IEEE International Joint Conference on Neural Networks, IJCNN’05, 31 July - 4 Aug. 2005, vol. 3, pp. 1349–1354 (2005)Google Scholar
  12. 12.
    Oliveira, L.S., Sabourin, R., Bortolozzi, F., Suen, C.Y.: Feature selection using multi-objective genetic algorithms for handwritten digit recognition. In: Proceedings of 16th International Conference on Pattern Recognition, 2002, 11-15 Aug. 2002, vol. 1, pp. 568–571 (2002)Google Scholar
  13. 13.
    Srinivas, N., Deb, K.: Multiobjective Optimization Using Nondominated Sorting in Genetic Algorithms. Journal of Evolutionary Computation 2(3), 221–248 (1998)CrossRefGoogle Scholar
  14. 14.
    Oliveira, L.S., Sabourin, R., Bortolozzi, F., Suen, C.Y.: Feature selection for ensembles: a hierarchical multi-objective genetic algorithm approach. In: Proceedings of Seventh International Conference on Document Analysis and Recognition, 3-6 Aug. 2003, pp. 676–680 (2003)Google Scholar
  15. 15.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6(2), 182–197 (2002)CrossRefGoogle Scholar
  16. 16.
    Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases. University of California, Department of Information and Computer Science, Irvine, CA (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Tarek M. Hamdani
    • 1
  • Jin-Myung Won
    • 2
  • Adel M. Alimi
    • 1
  • Fakhri Karray
    • 2
  1. 1.REsearch Group on Intelligent Machines (REGIM), University of Sfax, ENIS, BP. W-3038 – SfaxTunisia
  2. 2.Pattern Analysis and Machine Intelligence Research Group, Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON N2L 3G1Canada

Personalised recommendations