Pattern Analysis and Applications

, Volume 18, Issue 3, pp 639–650 | Cite as

Graph regularized linear discriminant analysis and its generalization

Short Paper

Abstract

Linear discriminant analysis (LDA) is a powerful dimensionality reduction technique, which has been widely used in many applications. Although, LDA is well-known for its discriminant capability, it clearly does not capture the geometric structure of the data. However, from the geometric perspective, the high-dimensional data resides on some low-dimensional manifolds in the sample space and these manifold structures are essential for data clustering and classification. In this paper, we propose a novel LDA algorithm named graph regularized linear discriminant analysis (GRLDA) to further improve the conventional LDA by incorporating such geometric information of data. GRLDA is achieved by penalizing the LDA with a Graph regularization, which is an affinity matrix encoding the geometric relationship of the data points. To take high-order geometric relationship among samples into consideration, we generalize GRLDA via using the hypergraph regularization instead of the graph regularization. We name this new version as hyper graph regularized linear discriminant analysis. Moreover, we exploit the null space of LDA via using an identity matrix to regularize the between-class scatter matrix. This strategy can further improve the discriminating power of LDA algorithms. Four popular face databases are used to evaluate our proposed LDA algorithms and the results of experiments demonstrate that they outperform the state-of-the-art dimensionality reduction algorithms.

Keywords

Linear discriminant analysis Locality preserving projections Face recognition Dimensionality reduction Hypergraph learning 

References

  1. 1.
    Jimenez LO, Landgrebe DA (1998) Supervised classification in high-dimensional space: geometrical, statistical, and asymptotical properties of multivariate data. IEEE Trans Syst Man Cybern Part C Appl Rev 28(1):39–54CrossRefGoogle Scholar
  2. 2.
    Van der Maaten LJP, Postma EO, Van Den Herik HJ (2009) Dimensionality reduction: a comparative review. J Mach Learn Res 10:1–41Google Scholar
  3. 3.
    Jolliffe I (2005) Principal component analysis. Wiley Online LibraryGoogle Scholar
  4. 4.
    Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791CrossRefGoogle Scholar
  5. 5.
    Seung D, Lee L (2001) Algorithms for non-negative matrix factorization. Adv Neural Inf Process Syst (NIPS) 13:556–562Google Scholar
  6. 6.
    Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720CrossRefGoogle Scholar
  7. 7.
    Kosinov S, Pun T (2008) Distance-based discriminant analysis method and its applications. Pattern Anal Appl 11(3–4):227–246MathSciNetCrossRefGoogle Scholar
  8. 8.
    Ching Wai-Ki, Chu Delin, Liao Li-Zhi, Wang Xiaoyan (2012) Regularized orthogonal linear discriminant analysis. Pattern Recogn 45(7):2719–2732CrossRefMATHGoogle Scholar
  9. 9.
    Sharma A, Paliwal KK (2012) A two-stage linear discriminant analysis for face-recognition. Pattern Recogn Lett 33(9):1157–1162CrossRefGoogle Scholar
  10. 10.
    Hyvärinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13(4):411–430CrossRefGoogle Scholar
  11. 11.
    Tenenbaum JB, De Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323CrossRefGoogle Scholar
  12. 12.
    Zheng Y, Tang YY, Fang B, Zhang T (2012) Orthogonal isometric projection. In: International conference on pattern recognition (ICPR), IEEE, pp 405–408Google Scholar
  13. 13.
    Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRefGoogle Scholar
  14. 14.
    Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396CrossRefMATHGoogle Scholar
  15. 15.
    Jerome H (1989) Friedman. Regularized discriminant analysis. J Am Stat Assoc 84(405):165–175CrossRefGoogle Scholar
  16. 16.
    Ye J, Yu B (2005) Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems. J Mach Learn Res 6(4)Google Scholar
  17. 17.
    Zhang T, Fang B, Tang YY, Shang Z, Xu B (2010) Generalized discriminant analysis: a matrix exponential approach. IEEE Trans Syst Man Cybern Part B Cybern 40(1):186–197CrossRefGoogle Scholar
  18. 18.
    Zafeiriou Stefanos, Tzimiropoulos Georgios, Petrou Maria, Stathaki Tania (2012) Regularized kernel discriminant analysis with a robust kernel for face recognition and verification. IEEE Trans Neural Netw Learn Syst 23(3):526–534CrossRefGoogle Scholar
  19. 19.
    Yin Xuesong, Chen Songcan, Enliang Hu (2013) Regularized soft k-means for discriminant analysis. Neurocomputing 103:29–42CrossRefGoogle Scholar
  20. 20.
    He X, Yan S, Hu Y, Niyogi P, Zhang H (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340CrossRefGoogle Scholar
  21. 21.
    He X, Cai D, Yan S, Zhang H (2005) Neighborhood preserving embedding. In: IEEE international conference on computer vision (ICCV), IEEE, vol 2. pp 1208–1213Google Scholar
  22. 22.
    Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560CrossRefGoogle Scholar
  23. 23.
    Liu W, Tao D (2013) Multiview hessian regularization for image annotation. IEEE Trans Image Process 22(7):2676MathSciNetCrossRefGoogle Scholar
  24. 24.
    Yu W, Teng X, Liu C (2006) Face recognition using discriminant locality preserving projections. Image Vision Comput 24(3):239–248CrossRefGoogle Scholar
  25. 25.
    Lu J, Tan Y-P (2010) Regularized locality preserving projections and its extensions for face recognition. IEEE Trans Syst Man Cybern Part B Cybern 40(3):958–963CrossRefGoogle Scholar
  26. 26.
    Huang S, Yang D (2013) Collaborative discriminant locality preserving projections with its application to face recognition. arXiv:1312.7469
  27. 27.
    Huang S, Elgammal A, Luwen H, Dan Y, Xiaohong Z (2014) Globality-locality preserving projections for biometric data dimensionality reduction. In: IEEE conference on computer vision and pattern recognition workshop on biometrics (CVPRW)Google Scholar
  28. 28.
    Zhou D, Huang J, Schölkopf B (2007) Learning with hypergraphs: clustering, classification, and embedding. Adv Neural Inf Process Syst (NIPS), 19:1601Google Scholar
  29. 29.
    Agarwal S, Branson K, Belongie S (2006) Higher order learning with graphs. In: International conference on machine learning (ICML)Google Scholar
  30. 30.
    Yuchi H, Qingshan L, Shaoting Z, Dimitris N (2010) Metaxas. Image retrieval via probabilistic hypergraph ranking, In: IEEE conference on computer vision and pattern recognition (CVPR)Google Scholar
  31. 31.
    Gao Y, Wang M, Tao D, Ji R, Dai Q (2012) 3-d object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process, p 21Google Scholar
  32. 32.
    Chung FRK (1997) Spectral graph theory, vol. 92. AMS BookstoreGoogle Scholar
  33. 33.
    Sun L, Ji S, Ye J (2008) Hypergraph spectral learning for multi-label classification. In: ACM international conference on knowledge discovery and data mining (ACM SIGKDD), pp 668–676Google Scholar
  34. 34.
    Sheng H, Dan Y, Yongxin G, Dengyang Z, Feng X (2014) Discriminant hyper-laplacian projections with its applications to face recognition, In: IEEE conference on multimedia and expo workshop on HIM (ICMEW)Google Scholar
  35. 35.
    Juwei Lu, Plataniotis KN, Venetsanopoulos AN (2005) Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition. Pattern Recogn Lett 26(2):181–191CrossRefGoogle Scholar
  36. 36.
    Vauhkonen Marko, Vadasz D, Karjalainen PA, Somersalo E, Kaipio JP (1998) Tikhonov regularization and prior information in electrical impedance tomography. IEEE Trans Med Imag 17(2):285–293CrossRefGoogle Scholar
  37. 37.
    Martínez A, Benavente R (1998) The ar face databaseGoogle Scholar
  38. 38.
    Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The feret database and evaluation procedure for face-recognition algorithms. Imag Vision Comput 16(5):295–306CrossRefGoogle Scholar
  39. 39.
    Wolf L, Hassner T, Taigman Y (2010) Similarity scores based on background samples. In: Asian conference on computer vision (ACCV), Springer, pp 88–97Google Scholar
  40. 40.
    Naseem Imran, Togneri Roberto, Bennamoun Mohammed (2010) Linear regression for face recognition. IEEE Trans Pattern Anal Mach Intell 32(11):2106–2112CrossRefGoogle Scholar
  41. 41.
    Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041CrossRefGoogle Scholar
  42. 42.
    Liu W, Tao D, Cheng J, Tang Y (2014) Multiview hessian discriminative sparse coding for image annotation. Comput Vis Image Underst 118:50–60CrossRefGoogle Scholar
  43. 43.
    Tao Dapeng, Jin Lianwen, Liu Weifeng, Li Xuelong (2013) Hessian regularized support vector machines for mobile image annotation on the cloud. IEEE Trans Multimed 15(4):833–844CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • Sheng Huang
    • 1
  • Dan Yang
    • 1
  • Jia Zhou
    • 1
  • Xiaohong Zhang
    • 2
  1. 1.College of Computer ScienceChongqing UniversityChongqingPeople’s Republic of China
  2. 2.School of Software Engineering, Ministry of Education Key Laboratory, Dependable Service Computing in Cyber Physical SocietyChongqing UniversityChongqingPeople’s Republic of China

Personalised recommendations