A New Linear Dimensionality Reduction Technique Based on Chernoff Distance

  • Luis Rueda
  • Myriam Herrera
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4140)


A new linear dimensionality reduction (LDR) technique for pattern classification and machine learning is presented, which, though linear, aims at maximizing the Chernoff distance in the transformed space. The corresponding two-class criterion, which is maximized via a gradient-based algorithm, is presented and initialization procedures are also discussed. Empirical results of this and traditional LDR approaches combined with two well-known classifiers, linear and quadratic, on synthetic and real-life data show that the proposed criterion outperforms the traditional schemes.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aladjem, M.: Linear Discriminant Analysis for Two Classes Via Removal of Classification Structure. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(2), 187–192 (1997)CrossRefGoogle Scholar
  2. 2.
    Chong, E., Zak, S.: An Introduction to Optimization, 2nd edn. John Wiley and Sons, Inc., New York (2001)MATHGoogle Scholar
  3. 3.
    Cooke, T.: Two Variations on Fisher’s Linear Discriminant for Pattern Recognition. IEEE Transations on Pattern Analysis and Machine Intelligence 24(2), 268–273 (2002)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Du, Q., Chang, C.: A Linear Constrained Distance-based Discriminant Analysis for Hyperspectral Image Classification. Pattern Recognition 34(2), 361–373 (2001)MATHCrossRefGoogle Scholar
  5. 5.
    Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. John Wiley and Sons, Inc., New York (2000)Google Scholar
  6. 6.
    Gao, H., Davis, J.: Why Direct LDA is not Equivalent to LDA. Pattern Recognition 39, 1002–1006 (2006)MATHCrossRefGoogle Scholar
  7. 7.
    Herrera, M., Leiva, R.: Generalización de la Distancia de Mahalanobis para el Análisis Discriminante Lineal en Poblaciones con Matrices de Covarianza Desiguales. Revista de la Sociedad Argentina de Estadística 3(1-2), 64–86 (1999)Google Scholar
  8. 8.
    Loog, M., Duin, P.W.: Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA: The Chernoff Criterion. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(6), 732–739 (2004)CrossRefGoogle Scholar
  9. 9.
    Lotlikar, R., Kothari, R.: Adaptive Linear Dimensionality Reduction for Classification. Pattern Recognition 33(2), 185–194 (2000)CrossRefGoogle Scholar
  10. 10.
    Newman, D., Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases, University of California, Irvine, Dept. of Computer Science (1998)Google Scholar
  11. 11.
    Rao, A., Miller, D., Rose, K., Gersho, A.: A Deterministic Annealing Approach for Parsimonious Design of Piecewise Regression Models. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(2), 159–173 (1999)CrossRefGoogle Scholar
  12. 12.
    Raudys, S.: Evolution and Generalization of a Single Neurone: I. Single-layer Perception as Seven Statistical Classifiers. Neural Networks 11(2), 283–296 (1998)CrossRefGoogle Scholar
  13. 13.
    Rueda, L.: Selecting the Best Hyperplane in the Framework of Optimal Pairwise Linear Classifiers. Pattern Recognition Letters 25(2), 49–62 (2004)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Rueda, L., Herrera, M.: Linear Discriminant Analysis by Maximizing the Chernoff Distance in the Transformed Space. Submitted for Publication (2006), Electronically available at http://www.inf.udec.cl/~lrueda/papers/ChernoffLDAJnl.pdf
  15. 15.
    Rueda, L., Oommen, B.J.: On Optimal Pairwise Linear Classifiers for Normal Distributions: The Two-Dimensional Case. IEEE Transations on Pattern Analysis and Machine Intelligence 24(2), 274–280 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Luis Rueda
    • 1
  • Myriam Herrera
    • 2
  1. 1.Department of Computer Science and Center for BiotechnologyUniversity of ConcepciónConcepciónChile
  2. 2.Department and Institute of InformaticsNational University of San JuanSan JuanArgentina

Personalised recommendations