The GA-Based Bayes-Optimal Feature Extraction Procedure Applied to the Supervised Pattern Recognition

  • Marek Kurzynski
  • Aleksander Rewak
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5097)


The paper deals with the extraction of features for statistical pattern recognition. Bayes probability of correct classification is adopted as the extraction criterion. The problem with complete probabilistic information is discussed and next the Bayes-optimal feature extraction procedure for the supervised classfication is presented in detail. As method of solution of optimal feature extraction a genetic algorithm is proposed. Several computer experiments for wide spectrum of cases were made and their results demonstrating capability of proposed approach to solve feature extraction problem are presented.


Genetic Algorithm Feature Extraction Linear Discriminant Analysis Principal Component Analysis Method Statistical Pattern Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Buturovic, L.: Toward Bayes-Optimal Linear Dimension Reduction. IEEE Trans. on PAMI 16, 420–424 (1994)Google Scholar
  2. 2.
    Choi, E., Lee, C.: Feature Extraction Based on the Bhattacharyya Distance. Pattern Recognition 36, 1703–1709 (2002)CrossRefGoogle Scholar
  3. 3.
    Devroye, L., Gyorfi, P., Lugossi, G.: A Probabilistic Theory of Pattern Recognition. Springer, New York (1996)zbMATHGoogle Scholar
  4. 4.
    Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley-Interscience, New York (2001)zbMATHGoogle Scholar
  5. 5.
    Fukunaga, K.: Introduciton to Statistical Pattern Recognition. Academic Press, London (1990)Google Scholar
  6. 6.
    Golub, G., Van Loan, C.: Matrix Computations. Johns Hopkins University Press (1996)Google Scholar
  7. 7.
    Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L.: Feature Extraction. Foundations and Applications. Springer, Heidelberg (2004)Google Scholar
  8. 8.
    Goldberg, D.: Genetic Algorithms in Search, Optimization and Machine Learning. Adison-Wesley, New York (1989)zbMATHGoogle Scholar
  9. 9.
    Herrera, F., Lozano, M.: Gradual Distributed Real-Coded Genetic Algorithm. IEEE Trans. on Evolutionary Computing 4, 43–63 (2000)CrossRefGoogle Scholar
  10. 10.
    Hsieh, P., Wang, D., Hsu, C.: A Linear Feature Extraction for Multiclass Classification Problems Based on Class Mean and Covariance Discriminant Information. IEEE Trans. on PAMI 28, 223–235 (2006)Google Scholar
  11. 11.
    Kubota, S., Mizutani, H., Yoshiaki, K.: A Discriminative Learning Criterion for the Overall Optimization of Error and Reject. In: Proc. 16th Int. Conf. on Pattern Recognition, vol. 4, pp. 498–502 (2002)Google Scholar
  12. 12.
    Kuo, B., Landgrebe, D.: A Robust Classification Procedure Based on Mixture Classifiers and Nonparametric Weighted Feature Extraction. IEEE Trans. on GRS 40, 2486–2494 (2002)Google Scholar
  13. 13.
    Puchala, E., Kurzynski, M.W., Rewak, A.: The Bayes-Optimal Feature Extraction Procedure for Pattern Recognition Using Genetic Algorithm. In: Kollias, S., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 21–30. Springer, Heidelberg (2006)Google Scholar
  14. 14.
    Loog, M., Duin, R., Haeb-Umbach, R.: Multiclass Linear Dimension Reduction by Meighted Pairwise Fisher Criteria. IEEE Trans. on PAMI 23, 762–766 (2001)Google Scholar
  15. 15.
    Michalewicz, Z.: Genetic Algorithms + Data Structure = Evolution Programs. Springer, New York (1996)Google Scholar
  16. 16.
    Park, H., Park, C., Pardalos, P.: Comparitive Study of Linear and Nonlinear Feature Extraction Methods - Technical Report. Minneapolis (2004)Google Scholar
  17. 17.
    Raymer, M., Punch, W., et al.: Dimensionality Reduction Using Genetic Algorithms. IEEE Trans. on EC 4, 164–168 (2002)Google Scholar
  18. 18.
    Rovithakis, G., Maniadakis, M., Zervakis, M.: A Hybrid Neural Network and Genetic Algorithm Approach to Optimizing Feature Extraction for Signal Classification. IEEE Trans. on SMC 34, 695–702 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Marek Kurzynski
    • 1
  • Aleksander Rewak
    • 1
  1. 1.Chair of Systems and Computer NetworksWroclaw University of TechnologyWroclawPoland

Personalised recommendations