A New Approach to Multi-class Linear Dimensionality Reduction

  • Luis Rueda
  • Myriam Herrera
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4225)


Linear dimensionality reduction (LDR) is quite important in pattern recognition due to its efficiency and low computational complexity. In this paper, we extend the two-class Chernoff-based LDR method to deal with multiple classes. We introduce the criterion, as well as the algorithm that maximizes such a criterion. The proof of convergence of the algorithm and a formal procedure to initialize the parameters of the algorithm are also given. We present empirical simulations on standard well-known multi-class datasets drawn from the UCI machine learning repository. The results show that the proposed LDR coupled with a quadratic classifier outperforms the traditional LDR schemes.


  1. 1.
    Aladjem, M.: Linear Discriminant Analysis for Two Classes Via Removal of Classification Structure. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(2), 187–192 (1997)CrossRefGoogle Scholar
  2. 2.
    Cooke, T.: Two Variations on Fisher’s Linear Discriminant for Pattern Recognition. IEEE Transations on Pattern Analysis and Machine Intelligence 24(2), 268–273 (2002)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Harville, D.: Matriz Algebra from a Statistician’s Perspective. Springer, New York (1997)Google Scholar
  4. 4.
    Du, Q., Chang, C.: A Linear Constrained Distance-based Discriminant Analysis for Hyperspectral Image Classification. Pattern Recognition 34(2), 361–373 (2001)zbMATHCrossRefGoogle Scholar
  5. 5.
    Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. John Wiley and Sons, Inc., New York (2000)Google Scholar
  6. 6.
    Gao, H., Davis, J.: Why Direct LDA is not Equivalent to LDA. Pattern Recognition 39, 1002–1006 (2006)zbMATHCrossRefGoogle Scholar
  7. 7.
    Herrera, M., Leiva, R.: Generalización de la Distancia de Mahalanobis para el Análisis Discriminante Lineal en Poblaciones con Matrices de Covarianza Desiguales. Revista de la Sociedad Argentina de Estadística 3(1-2), 64–86 (1999)Google Scholar
  8. 8.
    Loog, M., Duin, P.W.: Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA: The Chernoff Criterion. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(6), 732–739 (2004)CrossRefGoogle Scholar
  9. 9.
    Lotlikar, R., Kothari, R.: Adaptive Linear Dimensionality Reduction for Classification. Pattern Recognition 33(2), 185–194 (2000)CrossRefGoogle Scholar
  10. 10.
    Newman, D., Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases, University of California, Irvine, Dept. of Computer Science (1998)Google Scholar
  11. 11.
    Rao, A., Miller, D., Rose, K., Gersho, A.: A Deterministic Annealing Approach for Parsimonious Design of Piecewise Regression Models. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(2), 159–173 (1999)CrossRefGoogle Scholar
  12. 12.
    Raudys, S.: Evolution and Generalization of a Single Neurone: I. Single-layer Perception as Seven Statistical Classifiers. Neural Networks 11(2), 283–296 (1998)CrossRefGoogle Scholar
  13. 13.
    Rueda, L.: Selecting the Best Hyperplane in the Framework of Optimal Pairwise Linear Classifiers. Pattern Recognition Letters 25(2), 49–62 (2004)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Rueda, L., Herrera, M.: A New Linear Dimensionality Reduction Technique based on Chernoff Distance. In: Proceedings of the 10th Iberoamerican Conference on Artificial Intelligence, Ribeirao Prieto, Brazil, pp. 299–308 (October 2006)Google Scholar
  15. 15.
    Rueda, L., Herrera, M.: Linear Discriminant Analysis by Maximizing the Chernoff Distance in the Transformed Space (Submitted for Publication) (2006), Electronically available at,
  16. 16.
    Rueda, L., Oommen, B.J.: On Optimal Pairwise Linear Classifiers for Normal Distributions: The Two-Dimensional Case. IEEE Transations on Pattern Analysis and Machine Intelligence 24(2), 274–280 (2002)CrossRefGoogle Scholar
  17. 17.
    Rueda, L., Oommen, B.J.: On Optimal Pairwise Linear Classifiers for Normal Distributions: The d-Dimensional Case. Pattern Recognition 36(1), 13–23 (2003)zbMATHCrossRefGoogle Scholar
  18. 18.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 3rd edn. Elsevier, Amsterdam (2006)zbMATHGoogle Scholar
  19. 19.
    Tsujinishi, D., Abe, S.: Fuzzy Least Squares Support Vector Machines for Multi-class Problems. Neural Networks 16, 785–792 (2003)CrossRefGoogle Scholar
  20. 20.
    Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  21. 21.
    Webb, A.: Statistical Pattern Recognition, 2nd edn. John Wiley, New York (2002)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Luis Rueda
    • 1
  • Myriam Herrera
    • 2
  1. 1.Department of Computer Science and Center for BiotecnologyUniversity of ConcepciónConcepciónChile
  2. 2.Department and Institute of InformaticsNational University of San JuanSan JuanArgentina

Personalised recommendations