Skip to main content
SpringerLink
Log in

A Simple Review of Sparse Principal Components Analysis

  • Conference paper
  • First Online:
Intelligent Computing Theories and Application (ICIC 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9772))

Included in the following conference series:

Abstract

Principal Component Analysis (PCA) is a common tool for dimensionality reduction and feature extraction, which has been applied in many fields, such as biology, medicine, machine learning and bioinformatics. But PCA has two obvious drawbacks: each principal component is line combination and loadings are non-zero which is hard to interpret. Sparse Principal Component Analysis (SPCA) was proposed to overcome these two disadvantages of PCA under the circumstances. This review paper will mainly focus on the research about SPCA, where the basic models of PCA and SPCA, various algorithms and extensions of SPCA are summarized. According to the difference of objective function and the constraint conditions, SPCA can be divided into three groups as it shown in Fig. 1. We also make a comparison among the different kind of sparse penalties. Besides, brief statements and other different classifications are summarized at last.

The classification of SPCA algorithms in this paper.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Liu, J.-X., Xu, Y., Gao, Y.-L., Zheng, C.-H., Wang, D., Zhu, Q.: A Class-Information-based Sparse Component Analysis Method to Identify Differentially Expressed Genes on RNA-Seq Data. 13(2), 392–398 (2015)

    Google Scholar 

  2. Richtárik, P., Takáč, M., Ahipaşaoğlu, S.D.: Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes. arXiv preprint arXiv:1212.4137 (2012)

  3. Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24(6), 417 (1933)

    Article  MATH  Google Scholar 

  4. Jolliffe, I.T.: Rotation of principal components: choice of normalization constraints. J. Appl. Stat. 22(1), 29–35 (1995)

    Article  MathSciNet  Google Scholar 

  5. Ma, Z.: Sparse principal component analysis and iterative thresholding. Ann. Stat. 41(2), 772–801 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  6. Moghaddam, B., Weiss, Y., Avidan, S.: Spectral bounds for sparse PCA: Exact and greedy algorithms. In: Advances in neural information processing systems, pp. 915–922 (2005)

    Google Scholar 

  7. d’Aspremont, A., Bach, F.R., Ghaoui, L.E.: Full regularization path for sparse principal component analysis. In: Proceedings of the 24th international conference on Machine learning, pp. 177–184. ACM (2007)

    Google Scholar 

  8. Farcomeni, A.: An exact approach to sparse principal component analysis. Comput. Stat. 24(4), 583–604 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  9. Mackey, L.W.: Deflation methods for sparse PCA. In: Advances in Neural Information Processing Systems, pp. 1017–1024 (2009)

    Google Scholar 

  10. Wang, Y., Wu, Q.: Sparse PCA by iterative elimination algorithm. Adv. Comput. Math. 36(1), 137–151 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. Kuleshov, V.: Fast algorithms for sparse principal component analysis based on Rayleigh quotient iteration. In: Proceedings of the 30th International Conference on Machine Learning (ICML-13), pp. 1418–1425 (2013)

    Google Scholar 

  12. Journée, M., Nesterov, Y., Richtárik, P., Sepulchre, R.: Generalized power method for sparse principal component analysis. J. Mach. Learn. Res. 11, 517–553 (2010)

    MathSciNet  MATH  Google Scholar 

  13. Yuan, X.-T., Zhang, T.: Truncated power method for sparse eigenvalue problems. J. Mach. Learn. Res. 14(1), 899–925 (2013)

    MathSciNet  MATH  Google Scholar 

  14. Zhao, Q., Meng, D., Xu, Z.: A recursive divide-and-conquer approach for sparse principal component analysis. arXiv preprint, arXiv:1211.7219 (2012)

  15. Zass, R., Shashua, A.: Nonnegative sparse PCA. In: Advances in Neural Information Processing Systems, pp. 1561–1568 (2006)

    Google Scholar 

  16. Ulfarsson, M.O., Solo, V.: Vector sparse variable PCA. IEEE Trans. Sig. Process. 59(5), 1949–1958 (2011)

    Article  MathSciNet  Google Scholar 

  17. Jolliffe, I.T., Trendafilov, N.T., Uddin, M.: A modified principal component technique based on the LASSO. J. Comput. Graph. Stat. 12(3), 531–547 (2003)

    Article  MathSciNet  Google Scholar 

  18. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B (Methodological) 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  19. d’Aspremont, A., El Ghaoui, L., Jordan, M.I., Lanckriet, G.R.: A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49(3), 434–448 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  20. d’Aspremont, A., Bach, F., Ghaoui, L.E.: Optimal solutions for sparse principal component analysis. J. Mach. Learn. Res. 9, 1269–1294 (2008)

    MathSciNet  MATH  Google Scholar 

  21. Lu, Z., Zhang, Y.: An augmented Lagrangian approach for sparse principal component analysis. Math. Program. 135(1–2), 149–193 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Guo, J., James, G., Levina, E., Michailidis, G., Zhu, J.: Principal component analysis with sparse fused loadings. J. Comput. Graph. Stat. 19(4), 930–946 (2012)

    Article  MathSciNet  Google Scholar 

  23. Qi, X., Luo, R., Zhao, H.: Sparse principal component analysis by choice of norm. J. Multivar. Anal. 114, 127–160 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  24. Hein, M., Bühler, T.: An inverse power method for nonlinear eigenproblems with applications in 1-spectral clustering and sparse PCA. In: Advances in Neural Information Processing Systems, pp. 847–855 (2010)

    Google Scholar 

  25. Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. Comput. Graph. Stat. 15(2), 265–286 (2006)

    Article  MathSciNet  Google Scholar 

  26. Leng, C., Wang, H.: On general adaptive sparse principal component analysis. J. Comput. Graph. Stat. 18, 201–215 (2012)

    Article  MathSciNet  Google Scholar 

  27. Xiaoshuang, S., Zhihui, L., Zhenhua, G., Minghua, W., Cairong, Z., Heng, K.: Sparse Principal Component Analysis via Joint L 2, 1-Norm Penalty. In: Cranefield, S., Nayak, A. (eds.) AI 2013. LNCS, vol. 8272, pp. 148–159. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  28. Witten, D.M., Tibshirani, R., Hastie, T.: A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics 10, 515–534 (2009). kxp008

    Article  Google Scholar 

  29. Shen, H., Huang, J.Z.: Sparse principal component analysis via regularized low rank matrix approximation. J. Multivar. Anal. 99(6), 1015–1034 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  30. Papailiopoulos, D.S., Dimakis, A.G., Korokythakis, S.: Sparse PCA through low-rank approximations. arXiv preprint, arXiv:1303.0551 (2013)

  31. Lee, D., Lee, W., Lee, Y., Pawitan, Y.: Super-sparse principal component analyses for high-throughput genomic data. BMC Bioinformatics 11(1), 1 (2010)

    Article  Google Scholar 

  32. Meng, D., Zhao, Q., Xu, Z.: Improve robustness of sparse PCA by L 1-norm maximization. Pattern Recogn. 45(1), 487–497 (2012)

    Article  MATH  Google Scholar 

  33. Jenatton, R., Obozinski, G., Bach, F.: Structured sparse principal component analysis. arXiv preprint arXiv:0909.1440 (2009)

  34. Tipping, M.E., Nh, C.C.: Sparse kernel principal component analysis (2001)

    Google Scholar 

  35. Lounici, K.: Sparse principal component analysis with missing observations. In: Houdré, C., Mason, D.M., Rosiński, J., Wellner, J.A. (eds.) High dimensional probability VI, vol. 66, pp. 327–356. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  36. Asteris, M., Papailiopoulos, D.S., Karystinos, G.N.: Sparse principal component of a rank-deficient matrix. In: IEEE International Symposium on 2011 Information Theory Proceedings (ISIT), pp. 673–677. IEEE (2011)

    Google Scholar 

  37. Liu, W., Zhang, H., Tao, D., Wang, Y., Lu, K.: Large-scale paralleled sparse principal component analysis. Multimedia Tools Appl. 1–13 (2014)

    Google Scholar 

  38. Xiao, C.: Two-dimensional sparse principal component analysis for face recognition. In: 2010 2nd International Conference on Future Computer and Communication (ICFCC), pp. V2–561-V562-565. IEEE (2010)

    Google Scholar 

  39. Lai, Z., Xu, Y., Chen, Q., Yang, J., Zhang, D.: Multilinear sparse principal component analysis. IEEE Trans. Neural Netw. Learn. Syst. 25(10), 1942–1950 (2014)

    Article  Google Scholar 

  40. Sharp, K., Rattray, M.: Dense message passing for sparse principal component analysis. In: International Conference on Artificial Intelligence and Statistics, pp. 725–732 (2010)

    Google Scholar 

  41. Wang, S.-J., Sun, M.-F., Chen, Y.-H., Pang, E.-P., Zhou, C.-G.: STPCA: sparse tensor principal component analysis for feature extraction. In: 21st International Conference on 2012 Pattern Recognition (ICPR), pp. 2278–2281. IEEE (2012)

    Google Scholar 

  42. Jiang, R., Fei, H., Huan, J.: Anomaly localization for network data streams with graph joint sparse PCA. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 886–894. ACM (2011)

    Google Scholar 

  43. Grbovic, M., Li, W., Xu, P., Usadi, A.K., Song, L., Vucetic, S.: Decentralized fault detection and diagnosis via sparse PCA based decomposition and Maximum Entropy decision fusion. J. Process Control 22(4), 738–750 (2012)

    Article  Google Scholar 

  44. Johnstone, I.M., Lu, A.Y.: On consistency and sparsity for principal components analysis in high dimensions. J. Am. Stat. Assoc. 104 (2012)

    Google Scholar 

  45. Shen, D., Shen, H., Marron, J.S.: Consistency of sparse PCA in high dimension, low sample size contexts. J. Multivar. Anal. 115, 317–333 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  46. Cai, T.T., Ma, Z., Wu, Y.: Sparse PCA: Optimal rates and adaptive estimation. Ann. Stat. 41(6), 3074–3110 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  47. Luss, R., Teboulle, M.: Convex approximations to sparse PCA via Lagrangian duality. Oper. Res. Lett. 39(1), 57–61 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  48. Zhang, Y., d’Aspremont, A., El Ghaoui, L.: Sparse PCA: Convex relaxations, algorithms and applications. In: Anjos, M.F., Lasserre, J.B. (eds.) Handbook on Semidefinite, Conic and Polynomial Optimization, vol. 166, pp. 915–940. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  49. Wang, Z., Lu, H., Liu, H.: Tighten after relax: Minimax-optimal sparse PCA in polynomial time. Adv. Neural Inf. Process. Syst. 2014, 3383–3391 (2014)

    Google Scholar 

  50. Gao, C., Zhou, H.H.: Rate-optimal posterior contraction for sparse PCA. Ann. Stat. 43(2), 785–818 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  51. Johnstone, I.M., Lu, A.Y.: Sparse principal components analysis. Unpublished manuscript 7 (2004)

    Google Scholar 

  52. Ulfarsson, M.O., Solo, V.: Sparse variable PCA using geodesic steepest descent. IEEE Trans. Sig. Process. 56(12), 5823–5832 (2008)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the NSFC under grant Nos. 61572284, 61502272, 61572283, 61370163, 61373027 and 61272339; the Shandong Provincial Natural Science Foundation, under grant No. BS2014DX004; Shenzhen Municipal Science and Technology Innovation Council (Nos. JCYJ20140904154645958, JCYJ20140417172417174 and CXZZ20140904154910774).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Qufu Normal University, Rizhao, 276826, China

    Chun-Mei Feng, Jin-Xing Liu, Chun-Hou Zheng, Sheng-Jun Li & Dong Wang

  2. Library of Qufu Normal University, Qufu Normal University, Rizhao, 276826, China

    Ying-Lian Gao

  3. Bio-Computing Research Center, Shenzhen Graduate School, Harbin Institute of Technology, Shenzhen, 518055, China

    Jin-Xing Liu & Chun-Hou Zheng

Authors
  1. Chun-Mei Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying-Lian Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin-Xing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chun-Hou Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sheng-Jun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin-Xing Liu .

Editor information

Editors and Affiliations

  1. Tongji University , Shanghai, China

    De-Shuang Huang

  2. University of Ulsan , Ulsan, Korea (Republic of)

    Kang-Hyun Jo

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Feng, CM., Gao, YL., Liu, JX., Zheng, CH., Li, SJ., Wang, D. (2016). A Simple Review of Sparse Principal Components Analysis. In: Huang, DS., Jo, KH. (eds) Intelligent Computing Theories and Application. ICIC 2016. Lecture Notes in Computer Science(), vol 9772. Springer, Cham. https://doi.org/10.1007/978-3-319-42294-7_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-42294-7_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-42293-0

  • Online ISBN: 978-3-319-42294-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation