On Using a Pre-clustering Technique to Optimize LDA-Based Classifiers for Appearance-Based Face Recognition

  • Sang-Woon Kim
  • Robert Duin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4756)


Fisher’s Linear Discriminant Analysis (LDA) is a traditional dimensionality reduction method that has been proven to be successful for decades. To enhance the LDA’s power for high-dimensional pattern classification, such as face recognition, numerous LDA-extension approaches have been proposed in the literature. This paper proposes a new method that improves the performance of LDA-based classification by simply increasing the number of (sub)-classes through clustering a few of classes of the training set prior to the execution of LDA. This is based on the fact that the eigen space of the training set consists of the range space and the null space, and that the dimensionality of the range space increases as the number of classes increases. Therefore, when constructing the transformation matrix, through minimizing the null space, the loss of discriminative information resulted from this space can be minimized. To select the classes to be clustered, in the present paper, the intraset distance is employed as a criterion and the k-means clustering is performed to divide them. Our experimental results for an artificial data set of XOR-type samples and a well-known benchmark face database of Yale demonstrate that the classification efficiency of the proposed method could be improved.


Face Recognition Linear Discriminant Analysis (LDA) Pre-clustered LDA (PLDA) 


  1. 1.
    Zhao, W., Chellappa, R., Rosenfeld, A., Phillips, P.J.: Face recognition: a liter- ature survey. ACM Compt. Surveys 35(4), 399–458 (2003)CrossRefGoogle Scholar
  2. 2.
    Ruiz-del-Solar, J., Navarrete, P.: Eigenspace-based face recognition: a compar- ative study of different approaches. IEEE Trans. Systems, Man, and Cybernetics- Part C 35(3), 315–325 (2005)CrossRefGoogle Scholar
  3. 3.
    Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. and Machine Intell. 19(7), 711–720 (1997)CrossRefGoogle Scholar
  4. 4.
    Yu, H., Yang, J.: A direct LDA algorithm for high-dimensional data - with application to face recognition. Pattern Recognition 34, 2067–2070 (2001)zbMATHCrossRefGoogle Scholar
  5. 5.
    Yang, M.-H.: Kernel eigenfaces vs. kernel Fisherfaces: Face recognition using kernel methods. In: Proceedings of Fifth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 215–220 (2002)Google Scholar
  6. 6.
    Roweis, S., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  7. 7.
    Lotlikar, R., Kothari, R.: Adaptive linear dimensionality reduction for classifi cation. Pattern Recognition 33, 185–194 (2000)CrossRefGoogle Scholar
  8. 8.
    Loog, M., Duin, P.W.: Linear dimensionality reduction via a heteroscedastic extension of LDA: The Chernoff Criterion. IEEE Trans. Pattern Anal. and Machine Intell. 26(6), 732–739 (2004)CrossRefGoogle Scholar
  9. 9.
    Rueda, L., Herrera, M.: A new approach to multi-class linear dimensionality reduction. In: Martínez-Trinidad, J.F., Carrasco Ochoa, J.A., Kittler, J. (eds.) CIARP 2006. LNCS, vol. 4225, pp. 634–643. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Kim, T.-K., Kittler, J.: Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image. Pattern Anal. and Machine Intell. 27(3), 318–327 (2005)CrossRefGoogle Scholar
  11. 11.
    Zhu, M., Martinez, A.M.: Subclass discriminant analysis. IEEE Trans. Pattern Anal. and Machine Intell. 28(8), 1274–1286 (2006)CrossRefGoogle Scholar
  12. 12.
    Kim, S.-W.: On solving the small sample size problem using a dissimilarity rep- resentation for face recognition. In: Blanc-Talon, J., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2006. LNCS, vol. 4179, pp. 1174–1185. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, San Diego (1990)zbMATHGoogle Scholar
  14. 14.
    Friedman, M., Kandel, A.: Introduction to Pattern Recognition - Statistical, Structural, Neural and Fuzzy Logic Approaches. World Scientific, New Jersey (1999)Google Scholar
  15. 15.
    Maulik, U., Bandyopadhyay, S.: Performance evaluation of some clustering algorithms and validity indices. IEEE Trans. Pattern Anal. and Machine Intell. 24(12), 1650–1654 (2002)CrossRefGoogle Scholar
  16. 16.
    Howland, P., Wang, J., Park, H.: Solving the small sample size problem in face reognition using generalized discriminant analysis. Pattern Recognition 39, 277–287 (2006)CrossRefGoogle Scholar
  17. 17.
    Ye, J., Li, Q.: A two-stage linear discriminant analysis via QR-decomposition. IEEE Trans. Pattern Anal. and Machine Intell. 27(6), 929–941 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Sang-Woon Kim
    • 1
  • Robert Duin
    • 2
  1. 1.Senior Member, IEEE. Dept. of Computer Science and Engineering, Myongji University, Yongin, 449-728South Korea
  2. 2.Information and Communication Theory group, Delft University of TechnologyThe Netherlands

Personalised recommendations