Regularization of LDA for Face Recognition: A Post-processing Approach

  • Wangmeng Zuo
  • Kuanquan Wang
  • David Zhang
  • Jian Yang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3723)


When applied to high-dimensional classification task such as face recognition, linear discriminant analysis (LDA) can extract two kinds of discriminant vectors, those in the null space (irregular) and those in the range space (regular) of the within-class scatter matrix. Recently, regularization techniques, which alleviate the over-fitting to the training set, have been used to further improve the recognition performance of LDA. Most current regularization techniques, however, are pre-processing approaches and can’t be used to regularize irregular discriminant vectors. This paper proposes a post-processing method, 2D-Gaussian filtering, for regularizing both regular and irregular discriminant vectors. This method can also be combined with other regularization techniques. We present two LDA methods, regularization of subspace LDA (RSLD) and regularization of complete Fisher discriminant framework (RCFD) and test them on the FERET face database. Post-processing is shown to improve the recognition accuracy in face recognition.


Face Recognition Linear Discriminant Analysis Recognition Rate Regularization Technique Average Recognition Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Computing Surveys 35, 399–458 (2003)CrossRefGoogle Scholar
  2. 2.
    Swets, D.L., Weng, J.: Using discriminant Eigenfeatures for image retrieval. IEEE Trans. Pattern Analysis and Machine Intelligence 18, 831–836 (1996)CrossRefGoogle Scholar
  3. 3.
    Belhumeour, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces versus Fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Analysis and Machine Intelligence 19, 711–720 (1997)CrossRefGoogle Scholar
  4. 4.
    Torkkola, K.: Linear discriminant analysis in document classification. In: Proc. IEEE ICDM Workshop Text Mining (2001)Google Scholar
  5. 5.
    Baeka, J., Kimb, M.: Face recognition using partial least squares components. Pattern Recognition 37, 1303–1306 (2004)CrossRefGoogle Scholar
  6. 6.
    Chien, J.T., Wu, C.C.: Discriminant waveletfaces and nearest feature classifiers for face recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 24, 1644–1649 (2002)CrossRefGoogle Scholar
  7. 7.
    Chen, L.F., Mark Liao, H.Y., Ko, M.T., Lin, J.C., Yu, G.J.: A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognition 33, 1713–1726 (2000)CrossRefGoogle Scholar
  8. 8.
    Yu, H., Yang, J.: A direct LDA algorithm for high-dimensional data with application to face recognition. Pattern Recognition 34, 2067–2070 (2001)zbMATHCrossRefGoogle Scholar
  9. 9.
    Lu, J., Plataniotis, K.N., Venetsanopoulos, A.N.: Face recognition using LDA-based algorithms. IEEE Trans. Neural Networks 14, 195–200 (2003)CrossRefGoogle Scholar
  10. 10.
    Yang, J., Zhang, D., Yang, J.Y.: A generalized K-L expansion method which can deal with Small Smaple Size and high-dimensional problems. Pattern Analysis and Applications 6, 47–54 (2003)zbMATHCrossRefGoogle Scholar
  11. 11.
    Cevikalp, H., Neamtu, M., Wilkes, M., Barkana, A.: Discriminative common vectors for face recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 27, 4–13 (2005)CrossRefGoogle Scholar
  12. 12.
    Yang, J., Yang, J.Y.: Why can LDA be performed in PCA transformed space. Pattern Recognition 36, 563–566 (2003)CrossRefGoogle Scholar
  13. 13.
    Yang, J., Frangi, A.F., Yang, J.Y., Zhang, D., Jin, Z.: KPCA plus LDA: a complete Kernel Fisher discriminant framework for feature extraction and recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 27, 230–244 (2005)CrossRefGoogle Scholar
  14. 14.
    Zhao, W., Chellappa, R., Krishnaswamy, A.: Discriminant analysis of principal components for face recognition. In: Proc. Int’l Conf. Automatic Face and Gesture Recognition, pp. 336–341 (1998)Google Scholar
  15. 15.
    Bensmail, H., Celeux, G.: Regularized Gaussian discriminant analysis through eigenvalue decomposition. Journal of the American Statistics Association 91, 1743–1748 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Liu, C., Wechsler, H.: Enhanced Fisher linear discriminant models for face recognition. In: Proc. 14th Int’l Conf. Pattern Recognition, vol. 2, pp. 1368–1372 (1998)Google Scholar
  17. 17.
    Wang, X., Tang, X.: A unified framework for subspace face recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 26, 1222–1228 (2004)CrossRefGoogle Scholar
  18. 18.
    Dai, D.Q., Yuen, P.C.: Regularized discriminant analysis and its application to face recognition. Pattern Recognition 36, 845–847 (2003)zbMATHCrossRefGoogle Scholar
  19. 19.
    Chen, W.S., Yuen, P.C., Huang, J., Dai, D.Q.: Kernel machine-based one-parameter regularized Fisher Discriminant method for face recognition. IEEE Trans. Systems, Man, and Cybernetics-B 35, 659–669 (2005)CrossRefGoogle Scholar
  20. 20.
    Pima, I., Aladjem, M.: Regularized discriminant analysis for face recognition. Pattern Recognition 37, 1945–1948 (2004)CrossRefGoogle Scholar
  21. 21.
    Hoffbeck, J.P., Landgrebe, D.A.: Covariance matrix estimation and classification with limited training data. IEEE Trans. Pattern Analysis and Machine Intelligence 18, 763–767 (1996)CrossRefGoogle Scholar
  22. 22.
    Thomaz, C.E., Gillies, D.F., Feitosa, R.Q.: A new covariance estimate for Bayesian classifier in biometrics recognition. IEEE Trans. Circuits and Systems for Video Technology 14, 214–223 (2004)CrossRefGoogle Scholar
  23. 23.
    Pratt, W.K.: Digital Image Processing, 2nd edn. Wiley, New York (1991)zbMATHGoogle Scholar
  24. 24.
    Yang, J., Yang, J.Y., Zhang, D.: What’s wrong with Fisher criterion. Pattern recognition 35, 2665–2668 (2002)zbMATHCrossRefGoogle Scholar
  25. 25.
    Xu, Y., Yang, J.Y., Jin, Z.: Theory analysis on FSLDA and ULDA. Pattern recognition 36, 3031–3033 (2003)zbMATHCrossRefGoogle Scholar
  26. 26.
    Jin, Z., Yang, J.Y., Hu, Z.S., Lou, Z.: Face recognition based on the uncorrelated discriminant transform. Pattern recognition 34, 1405–1416 (2001)zbMATHCrossRefGoogle Scholar
  27. 27.
    Philips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Analysis and Machine Intelligence 22, 1090–1104 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Wangmeng Zuo
    • 1
  • Kuanquan Wang
    • 1
  • David Zhang
    • 2
  • Jian Yang
    • 2
  1. 1.School of Computer Science and TechnologyHarbin Institute of TechnologyHarbinChina
  2. 2.Department of ComputingThe Hong Kong Polytechnic UniversityKowloon, Hong Kong

Personalised recommendations