Chernoff-Based Multi-class Pairwise Linear Dimensionality Reduction

  • Luis Rueda
  • Claudio Henríquez
  • B. John Oommen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5197)


Linear dimensionality reduction techniques have been studied very well for the two-class problem, while the corresponding issues encountered when dealing with multiple classes are far from trivial. In this paper, we show that dealing with multiple classes, it is not expedient to treat it as a multi-class problem, but it is better to treat it as an ensemble of Chernoff-based two-class reductions onto different subspaces. The solution is achieved by resorting to either Voting, Weighting, or a Decision Tree combination scheme. The ensemble methods were tested on benchmark datasets demonstrating that the proposed method is not only efficient, but also yields an accuracy comparable to that obtained by the optimal Bayes classifier.


Linear Dimensionality Reduction Fisher’s Discriminant Analysis Heteroscedastic Discriminant Analysis Chernoff Distance 


  1. 1.
    Abe, S.: Support Vector Machines for Pattern Classification. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  2. 2.
    Asuncion, A., Newman, D.J.: UCI machine learning repository (2007),
  3. 3.
    Kim, S., Magnani, A., Boyd, S.: Optimal Kernel Selection in Kernel Fisher Discriminant Analysis. In: Proc. of the 23rd International Conference on Machine Learning, Pittsburgh, USA, pp. 465–472 (2006)Google Scholar
  4. 4.
    Loog, M., Duin, P.W.: Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA: The Chernoff Criterion. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(6), 732–739 (2004)CrossRefGoogle Scholar
  5. 5.
    Martinez, A., Zhu, M.: Where Are Linear Feature Extraction Methods Applicable? IEEE Trans. on Pattern Analysis and Machine Intelligence 27(12), 1934–1944 (2005)CrossRefGoogle Scholar
  6. 6.
    Rohl, M., Weihs, C.: Optimal vs. classical linear dimension reduction. In: Information Age, Studies in Classification, Data Analysis, and Knowledge Organization, pp. 252–259. Springer, Heidelberg (1999)Google Scholar
  7. 7.
    Rueda, L.: Selecting the Best Hyperplane in the Framework of Optimal Pairwise Linear Classifiers. Pattern Recognition Letters 25(2), 49–62 (2004)CrossRefGoogle Scholar
  8. 8.
    Rueda, L., Herrera, M.: A New Approach to Multi-class Linear Dimensionality Reduction. In: Martínez-Trinidad, J.F., Carrasco Ochoa, J.A., Kittler, J. (eds.) CIARP 2006. LNCS, vol. 4225, pp. 634–643. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Rueda, L., Oommen, B.J.: On Optimal Pairwise Linear Classifiers for Normal Distributions: The Two-Dimensional Case. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(2), 274–280 (2002)CrossRefGoogle Scholar
  10. 10.
    Tax, D.M.J., Duin, R.P.W.: Using Two-Class Classifiers for Multiclass Classification. In: Proceedings of the 16th International Conference on Pattern Recognition, Quebec, Canada, vol. 2, pp. 124–127 (2002)Google Scholar
  11. 11.
    Zhu, M., Martinez, A.: Subclass Discriminant Analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(8), 1262–1273 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Luis Rueda
    • 1
  • Claudio Henríquez
    • 2
  • B. John Oommen
    • 3
  1. 1.School of Computer ScienceUniversity of WindsorWindsorCanada
  2. 2.Department of Computer ScienceUniversity of ConcepciónConcepciónChile
  3. 3.School of Computer ScienceCarleton UniversityOttawaCanada

Personalised recommendations