Skip to main content
SpringerLink
Log in

Projection-optimal local Fisher discriminant analysis for feature extraction

  • Advances in Intelligent Data Processing and Analysis
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this paper, a novel dimensionality reduction algorithm called projection-optimal local Fisher discriminant analysis (PoLFDA) is proposed in order to address the multimodal problem. Novel weight matrices defined on the projected space can represent the intraclass compactness and the interclass separability. Based on the novel weighted matrices, the local between-class scatter matrix and the local within-class scatter matrix are defined such that the local structure can be preserved. In order to enhance the discriminant ability, we impose an orthogonal constraint on the objective function, which can be regarded as a trace ratio problem. In general, a trace ratio problem does not have a closed-form solution; however, it can be solved using some efficient iterative algorithms. Therefore, we optimize the projection matrix by solving the trace ratio problem iteratively. Experiments on toy data, face, and handwritten digit data sets are conducted to evaluate the performance of PoLFDA; the results and comparisons verify the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Turk MA, Pentland AP (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–96

    Article  Google Scholar 

  2. Fukunaga K (1991) Introduction to statistical pattern recognition, 2nd edn. Academic Press, New York

    Google Scholar 

  3. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenface vs. Fisherface:recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  4. Yu H, Yang J (2001) A direct LDA algorithm for high-dimensional data with application to face recognition. Pattern Recognit 34(10):2067–2070

    Article  MATH  Google Scholar 

  5. Chen LF, Liao HY, Lin JC, Ko MT, Yu GJ (2000) A new LDA based face recognition system which can solve the small sample size problem. Pattern Recognit 33(10):1713–1726

    Article  Google Scholar 

  6. Chu D, Thye ST (2010) A new and fast implement for null space based linear discriminant analysis. Pattern Recognit 43(4):1373–1379

    Article  MATH  Google Scholar 

  7. Tenenbaum J, Silva V, Langford J (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Article  Google Scholar 

  8. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  9. Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396

    Article  MATH  Google Scholar 

  10. Zhang Z, Zha H (2004) Principle manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J Sci Comput 26(1):313–338

    Article  MATH  MathSciNet  Google Scholar 

  11. He XF, Yan SC, Hu Y, Niyogi P, Zhang HJ (2005) Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340

    Article  Google Scholar 

  12. Cai D, He XF, Han JW, Zhang HJ (2006) Orthogonal Laplacianfaces for face recognition. IEEE Trans Image Process 15(11):3608–3614

    Article  Google Scholar 

  13. He XF, Cai D, Yan SC, Zhang H (2005) Neighborhood preserving embedding. In: Tenth IEEE international conference on computer vision, Beijing, China, pp 1208–1213

  14. Zhang T, Yang J, Zhao D, Ge X (2007) Linear local tangent space alignment and application to face recognition. Neurocomputing 70:1547–1553

    Article  Google Scholar 

  15. Li X, Lin S, Yan S, Xu D (2008) Discriminant locally linear embedding with high-order tensor data. IEEE Trans Syst Man Cybern Part B 38(2):342–352

    Article  Google Scholar 

  16. Yu W, Teng X, Liu C (2006) Face recognition using discriminant locality preserving projections. Image Vis Comput 24(3):239–248

    Article  Google Scholar 

  17. Zhang TH, Huang KQ, Li XL, Yang J, Tao DC (2010) Discriminative orthogonal neighborhood-preserving projections for classification. IEEE Trans Syst Man Cybern Part B 40(1):253–263

    Article  Google Scholar 

  18. Yan S, Xu D, Zhang B, Zhang H, Yang Q, Lin S (2007) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–50

    Article  Google Scholar 

  19. Cai D, He X, Zhou K, Han J, Bao H (2007) Locality sensitive discriminant analysis. In: Proceedings of international joint conference on artificial intelligence, Hyderabad, India, pp 708–713

  20. Chen HT, Chang HW, Liu TL (2005) Local discriminant embedding and its variants. In: IEEE computer society conference on computer vision and pattern recognition, vol 2, pp 846–853

  21. Sugiyama M (2007) Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. J Mach Learn Res 8:1027–1061

    MATH  Google Scholar 

  22. Wang H, Yan S, Xu D, Tang X, Huang T (2007) Trace ratio versus ratio trace for dimensionality reduction. In: IEEE computer society conference on computer vision and pattern recognition. Minneapolis, pp 1–8

  23. Jia Y, Nie F, Zhang C (2009) Trace ratio problem revisited. IEEE Trans Neural Netw 20(4):729–735

    Article  Google Scholar 

  24. Zhang LH, Liao LZ, Ng MK (2010) Fast algorithms for the generalized Foley–Sammon discriminant analysis. SIAM J Matrix Anal Appl 31:1584–1605

    Article  MATH  MathSciNet  Google Scholar 

  25. Zhang LH, Liao LZ, Ng MK (2013) Superlinear convergence of a general algorithm for the generalized Foley–Sammon discriminant analysis. J Optim Theory Appl 157(3):853–865

    Article  MATH  MathSciNet  Google Scholar 

  26. Zhao MB, Zhang Z, Chow WS (2012) Trace ratio criterion based generalized discriminative learning for semi-supervised dimensionality reduction. Pattern Recognit 45(4):1482–1499

    Article  MATH  Google Scholar 

  27. Ngo TT, Bellalij M, Saad Y (2010) The trace ratio optimization problem for dimensionality reduction. SIAM J Matrix Anal Appl 31:2950–2971

    Article  MATH  MathSciNet  Google Scholar 

  28. Zhang Z, Chow WS, Zhao MB (2013) Trace ratio optimization-based semi-supervised nonlinear dimensionality reduction for marginal manifold visualization. IEEE Trans Knowl Data Eng 23:1148–1161

    Article  Google Scholar 

  29. Zelnik-Manor L, Perona P (2005) Self-tuning spectral clustering. In: Advances in neural information processing systems, pp 1601–1608

  30. Nocedal J, Wright S (2006) Numerical optimization, 2nd edn. Springer, Berlin

    MATH  Google Scholar 

  31. Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: Second IEEE workshop on applications of computer vision, pp 138–142

  32. Lee KC, Ho J, Kriegman DJ (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698

    Article  Google Scholar 

  33. Hull J (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554

    Article  Google Scholar 

  34. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported partly by the National Natural Science Foundation of China (61172128, 61003114), National Key Basic Research Program of China (2012CB316304), the fundamental research funds for the central universities (2013JBM020, 2013JBZ003), Program for Innovative Research Team in University of Ministry of Education of China (IRT201206), and Doctoral Foundation of China Ministry of Education (20120009120009).

Author information

Authors and Affiliations

  1. Institute of Information Science, Beijing Jiaotong University, Beijing, 100044, China

    Zhan Wang, Qiuqi Ruan & Gaoyun An

  2. Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing, 100044, China

    Zhan Wang, Qiuqi Ruan & Gaoyun An

Authors
  1. Zhan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiuqi Ruan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gaoyun An
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhan Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Ruan, Q. & An, G. Projection-optimal local Fisher discriminant analysis for feature extraction. Neural Comput & Applic 26, 589–601 (2015). https://doi.org/10.1007/s00521-014-1768-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-014-1768-9

Keywords

Search

Navigation