Skip to main content

Principal component analysis based on graph embedding

Abstract

Dimensionality reduction plays an important role in image recognition and data mining. Traditional methods extract features from data itself and ignore the structure information of data even though it is crucial for effective representation. Considering graph embedding method can capture and model the complicated relationships among data, therefore, we consider to incorporate graph convolution learning into principal component analysis (GCPCA) to abstract more effective features in this paper. The key idea of the proposed model is embedding graph convolutional to realize linear representation by fusing the relationship of data points. Then PCA is operated on projected data to extract effective features. The model can be solved to obtain a globally optimal closed-form solution, which is convenient for implementation and practical application. Experiments on some publicly available datasets demonstrate that the proposed GCPCA model show the better performance than the existing classical algorithms in terms of classification accuracy.

This is a preview of subscription content, access via your institution.

Fig. 1

References

  1. Abraham R, Marsden JE, Ratiu T (2012) Manifolds, tensor analysis, and applications springer science & business media

  2. Benamira A, Devillers B, Lesot E, Ray A, Saadi M, Malliaros FD (2019) Semi-supervised learning and graph neural networks for fake news detection. In: IEEE/ACM International conference on advances in social networks analysis and mining

  3. Bishop C (2006) Pattern recognition and machine learning. Inf Sci Stat Springer

  4. Celik S, Logsdon B, Aase S (2014) Efficient dimensionality reduction for high-dimensional network estimation. In: International conference on machine learning, pp 1953–1961

  5. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in neural information processing systems, pp 3844–3852

  6. Feng G, Hu D, Zhang D, Zhou Z (2006) Letters: an alternative formulation of kernel lpp with application to image recognition. Neurocomputing 69 (13):1733–1738

    Article  Google Scholar 

  7. He X, Niyogi P (2003) Locality preserving projections. In: Advances in neural information processing systems

  8. He X, Yan S, Hu Y, Niyogi P, Zhang H (2005) Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340

    Article  Google Scholar 

  9. Hoyer P, Hyvärinen A. (2000) Independent component analysis applied to feature extraction from colour and stereo images. Netw Comput Neural Syst 11 (3):191–210

    Article  Google Scholar 

  10. Hu D, Feng G, Zhou Z (2007) Two-dimensional locality preserving projections (2dlpp) with its application to palmprint recognition. Pattern Recogn 40 (1):339–342

    Article  Google Scholar 

  11. Jenatton R, Obozinski G, Bach F (2010) Structured sparse principal component analysis. In: International conference on artificial intelligence and statistics, pp 366–373

  12. Jolliffe IT (2002) Principal component analysis springer verlag

  13. Ju F, Sun Y, Gao J, Hu Y, Yin B (2015) Image outlier detection and feature extraction via L1-norm based 2D probabilistic PCA. IEEE Trans Image Process 24(12):4834–4846

    MathSciNet  Article  Google Scholar 

  14. Ke Y, Sukthankar R (2004) PCA-SIFT: a more distinctive representation for local image descriptors. In: IEEE International conference on computer vision and pattern recognition, pp 506–513

  15. Kim Y, Toh KA (2008) Sparse random projection for efficient cancelable face feature extraction. In: IEEE Conference on industrial electronics & applications, pp 2139–2144

  16. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International conference on machine learning

  17. Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500

    MathSciNet  Article  Google Scholar 

  18. Kuncheva L, Faithfull WJ (2014) PCA Feature extraction for change detection in multidimensional unlabeled data. IEEE Trans Neural Netw Learn Syst 25(1):69–80

    Article  Google Scholar 

  19. Leng L, Zhang J, Chen G, Khan MK, Alghathbar K (2011) Two-directional two-dimensional random projection and its variations for face and palmprint recognition springer berlin heidelberg

  20. Leng L, Zhang S, Bi X, Khan MK (2012) Two-dimensional cancelable biometric scheme. In: International conference on wavelet analysis & pattern recognition, pp 164–169

  21. Li Q, Han Z, Wu XM (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the thirty-second AAAI conference on artificial intelligence

  22. Li X, Pang Y, Yuan Y (2010) L1-norm-based 2DPCA. IEEE Trans Syst Man Cybern Part B Cybern 40(4):1170–1175

    Article  Google Scholar 

  23. Li X, Zhang H, Zhang R (2020) Embedding graph auto-encoder with joint clustering via adjacency sharing. arXiv:2002.08643

  24. Liu X, You X, Zhang X, Wu J, Lv P (2019) Tensor graph convolutional networks for text classification. In: Conference on artificial intelligence, p online

  25. Lu H, Plataniotis K, Venetsanopoulos A (2008) MPCA: multilinear principal component analysis of tensor objects. IEEE Trans Neural Netw 19(1):18–39

    Article  Google Scholar 

  26. Lv J, Kang Z, Lu X, Xu Z (2021) Pseudo-supervised deep subspace clustering. IEEE Trans Image Process 30:5252–5263

    Article  Google Scholar 

  27. Pan S, Hu R, Fung S, Long G, Jiang J, Zhang C (2020) Learning graph embedding with adversarial training methods. IEEE Trans Cybern 50(6):2475–2487

    Article  Google Scholar 

  28. Pang Y, Yuan Y (2010) Outlier-resisting graph embedding. Neurocomputing 73(4):968–974

    Article  Google Scholar 

  29. Shen XJ, Liu SX, Bao BK, Pan CH, Fan J (2019) A generalized least-squares approach regularized with graph embedding for dimensionality reduction. Pattern Recogn 98:107023

    Article  Google Scholar 

  30. Song T, Chowdhury SR, Yang F, Jacobs H, Fakhri GE, Li Q, Johnson K, Dutta J (2019) Graph convolutional neural networks for alzheimer’s disease classification. In: IEEE International symposium on biomedical imaging, pp 414–417

  31. T TZ, Hu Y, Cheng L (2021) Deep-DRM: a computational method for identifying disease-related metabolites based on graph deep learning approaches. Brief Bioinform 22(4):1–9

    Google Scholar 

  32. Torki M, Elgammal A, Lee CS (2010) Learning a joint manifold representation from multiple data sets. In: IEEE international conference on pattern recognition, pp 1068–1071

  33. van den Berg R, Kipf TN, Welling M (2018) Graph convolutional matrix completion. In: ACM SIGKDD Conference on knowledge discovery and data mining

  34. Vasilescu M A O, Terzopoulos D (2003) Multilinear subspace analysis of image ensembles. In: IEEE international conference on computer vision and pattern recognition, pp 93–99

  35. Wang C, Pan S, Hu R, Long G (2019) Attributed graph clustering: a deep attentional embedding approach. arXiv:1906.06532

  36. Wang H, Wang J (2013) 2DPCA with L1-norm for simultaneously robust and sparse modelling. Neural Netw 46(10):190–198

    Article  Google Scholar 

  37. Wang J (2012) Random projection springer berlin heidelberg

  38. Wu M, Pan S, Zhu X, Zhou C, Pan L (2019) Domain-adversarial graph neural networks for text classification. In: Industrial conference on data mining, pp 648–657

  39. Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2020) A comprehensive survey on graph neural networks. IEEE Trans Neual Netw Learn Syst 32 (1):4–24

    MathSciNet  Article  Google Scholar 

  40. Wu Z, Pan S, Long G, Jiang J, Zhang C (2019) Graph wavenet for deep spatial-temporal graph modeling. In: International joint conference on artificial intelligence, pp 1907–1913

  41. Xie X, Yan S, Kwok J, Huang T (2008) Matrix-variate factor analysis and its applications. IEEE Trans Neural Netw 19(10):1821–1826

    Article  Google Scholar 

  42. Xu Y, Zhong A, Yang J, Zhang D (2010) Lpp solution schemes for use with face recognition. Pattern Recogn 43(12):4165Ċ4176

    Article  Google Scholar 

  43. Yang J, Zhang D, Frangi AF, Yang J (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137

    Article  Google Scholar 

  44. Yang M (2002) Kernel eigenfaces vs. kernel fisherfaces: face recognition using kernel methods. In: IEEE International conference on automatic face gesture recognition, pp 215–220

  45. Ye J (2005) Generalized low rank approximations of matrices. Mach Learn 61(1–3):167–191

    Article  Google Scholar 

  46. Yuan Y, Yu B, Wang W, Yu B H (2021) Multi-filter dynamic graph convolutional networks for skeleton-based action recognition. Procedia Comput Sci 183:572–578

    Article  Google Scholar 

  47. Zhang J, Huang H, Wang J (2010) Manifold learning for visualizing and analyzing high-dimensional data. IEEE Intell Syst 25(4):54–61

    Google Scholar 

  48. Zhang Z, Cui P, Zhu W (2020) Deep learning on graph: a survey. IEEE Trans Knowl Data Eng 34(1):249–270

    Article  Google Scholar 

  49. Zhao H, Xing H, Wang X, Chen J (2011) L1-norm-based 2dlpp. In: Control and decision conference, pp 1259–1264

Download references

Acknowledgments

This research was supported by Beijing Municipal Science and Technology Project with no. Z191100009119013, in part by National Natural Science Foundation of China under Grant 61772048, 61672071 and 61902053, in part by the Beijing Natural Science Foundation under Grant 4172003, in part by Beijing Municipal Science and Technology Project KM201910005028 and KM202011417004.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yanfeng Sun.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ju, F., Sun, Y., Li, J. et al. Principal component analysis based on graph embedding. Multimed Tools Appl (2022). https://doi.org/10.1007/s11042-022-13620-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-022-13620-6

Keywords

  • Dimensionality reduction
  • Graph embedding
  • Principal component analysis
  • Feature extraction