Advertisement

On the Performance of Chernoff-Distance-Based Linear Dimensionality Reduction Techniques

  • Mohammed Liakat Ali
  • Luis Rueda
  • Myriam Herrera
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4013)

Abstract

We present a performance analysis of three linear dimensionality reduction techniques: Fisher’s discriminant analysis (FDA), and two methods introduced recently based on the Chernoff distance between two distributions, the Loog and Duin (LD) method, which aims to maximize a criterion derived from the Chernoff distance in the original space, and the one introduced by Rueda and Herrera (RH), which aims to maximize the Chernoff distance in the transformed space. A comprehensive performance analysis of these methods combined with two well-known classifiers, linear and quadratic, on synthetic and real-life data shows that LD and RH outperform FDA, specially in the quadratic classifier, which is strongly related to the Chernoff distance in the transformed space. In the case of the linear classifier, the superiority of RH over the other two methods is also demonstrated.

Keywords

Linear Discriminant Analysis Machine Intelligence Back Propagation Neural Network Original Space Lower Error Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aladjem, M.: Linear Discriminant Analysis for Two Classes Via Removal of Classification Structure. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(2), 187–192 (1997)CrossRefGoogle Scholar
  2. 2.
    Cooke, T.: Two Variations on Fisher’s Linear Discriminant for Pattern Recognition. IEEE Transations on Pattern Analysis and Machine Intelligence 24(2), 268–273 (2002)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Du, Q., Chang, C.: A Linear Constrained Distance-based Discriminant Analysis for Hyperspectral Image Classification. Pattern Recognition 34(2), 361–373 (2001)MATHCrossRefGoogle Scholar
  4. 4.
    Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. John Wiley and Sons, Inc., New York (2000)Google Scholar
  5. 5.
    Lippman, R.: An Introduction to Computing with Neural Nets. In: Neural Networks: Theoretical Foundations and Analsyis, pp. 5–24. IEEE Press, Los Alamitos (1992)Google Scholar
  6. 6.
    Loog, M., Duin, P.W.: Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA: The Chernoff Criterion. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(6), 732–739 (2004)CrossRefGoogle Scholar
  7. 7.
    Loog, M., Duin, R.: Non-iterative Heteroscedastic Linear Dimension Reduction for Two-Class Data. In: Caelli, T.M., Amin, A., Duin, R.P.W., Kamel, M.S., de Ridder, D. (eds.) SPR 2002 and SSPR 2002. LNCS, vol. 2396, pp. 508–517. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Lotlikar, R., Kothari, R.: Adaptive Linear Dimensionality Reduction for Classification. Pattern Recognition 33(2), 185–194 (2000)CrossRefGoogle Scholar
  9. 9.
    Murphy, O.: Nearest Neighbor Pattern Classification Perceptrons. In: Neural Networks: Theoretical Foundations and Analysis, pp. 263–266. IEEE Press, Los Alamitos (1992)Google Scholar
  10. 10.
    Newman, D., Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases, University of California, Irvine, Dept. of Computer Science (1998)Google Scholar
  11. 11.
    Rao, A., Miller, D., Rose, K., Gersho, A.: A Deterministic Annealing Approach for Parsimonious Design of Piecewise Regression Models. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(2), 159–173 (1999)CrossRefGoogle Scholar
  12. 12.
    Raudys, S.: On Dimensionality, Sample Size, and Classification Error of Nonparametric Linear Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(6), 667–671 (1997)CrossRefGoogle Scholar
  13. 13.
    Raudys, S.: Evolution and Generalization of a Single Neurone: I. Single-layer Perception as Seven Statistical Classifiers. Neural Networks 11(2), 283–296 (1998)CrossRefGoogle Scholar
  14. 14.
    Raudys, S.: Evolution and Generalization of a Single Neurone: II. Complexity of Statistical Classifiers and Sample Size Considerations. Neural Networks 11(2), 297–313 (1998)CrossRefGoogle Scholar
  15. 15.
    Rueda, L.: Selecting the Best Hyperplane in the Framework of Optimal Pairwise Linear Classifiers. Pattern Recognition Letters 25(2), 49–62 (2004)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Rueda, L., Herrera, M.: Linear Discriminant Analysis by Maximizing the Chernoff Distance in the Transformed Space (submitted for Publication) (2006)Google Scholar
  17. 17.
    Rueda, L., Oommen, B.J.: On Optimal Pairwise Linear Classifiers for Normal Distributions: The Two-Dimensional Case. IEEE Transations on Pattern Analysis and Machine Intelligence 24(2), 274–280 (2002)CrossRefGoogle Scholar
  18. 18.
    Webb, A.: Statistical Pattern Recognition, 2nd edn. John Wiley & Sons, New York (2002)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Mohammed Liakat Ali
    • 1
  • Luis Rueda
    • 2
  • Myriam Herrera
    • 3
  1. 1.School of Computer ScienceUniversity of WindsorWindsorCanada
  2. 2.Department of Computer ScienceUniversity of ConcepciónConcepciónChile
  3. 3.Institute of InformaticsNational University of San Juan, Cereceto y MeglioliSan JuanArgentina

Personalised recommendations