Skip to main content

Survey of Popular Linear Dimensionality Reduction Techniques

  • Conference paper
  • First Online:
Proceedings of International Conference on Deep Learning, Computing and Intelligence

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1396))

  • 623 Accesses

Abstract

Big Data analytics related solutions are one of the prime industrial focuses across all domains. In this digital era, the volumes of the data being generated by both machine and man are humongous. The key challenges are to store the data which requires more space and also to retrieve the data in an optimized way that saves time and money. The number of features for any given dataset is another problem statement. As in most real-life datasets, the number of features present is more and we are not sure, which features or dimensions are good enough to be considered. Visualization of the high dimensional data is also one of the most complex issues that impair meaningful insights since the visuals are not precise anymore due to feature explosion. In this paper, we have done a comprehensive survey on some of the most popular dimensionality reduction approaches broadly categorized under linear dimensionality reduction techniques. We have also done a critical comparative analysis focused on the utility of these algorithms. Finally concluded the survey with few notes on future work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. L. Gao, J. Song, X. Liu, J. Shao, J. Liu, J. Shao, Learning in high-dimensional multimedia data: the state of the art. Multimedia Syst. 23(3), 303–313 (2017)

    Article  Google Scholar 

  2. L.V.D. Maaten, E. Postma, J.V. Herik, Dimensionality reduction: a comparative. J. Mach. Learn. Res. 10, 66–71 (2009). G. Chandrashekar, F. Sahin, A survey

    Google Scholar 

  3. A.N. Escalante-B, L. Wiskott, How to solve classification and regression problems on high dimensional data with a supervised extension of slow feature analysis. JMLR 14, 3683–3719 (2013)

    Google Scholar 

  4. D. Amaratunga, J. Cabrera, High-dimensional data, J. Natl. Sci. Found. 44

    Google Scholar 

  5. S. Ayesha, M.K. Hanif, R. Talib, Overview and comparative study of dimensionality reduction techniques for high dimensional data. Information Fusion 59, 44–58

    Google Scholar 

  6. P. Comon, Independent component analysis, a new concept? Signal Process. 36(3), 287–314 (1994)

    Article  Google Scholar 

  7. B. Tang, M. Shepherd, E. Milios, M.I. Heywood, Comparing and combining dimension reduction techniques for efficient text clustering, in International Workshop on Feature Selection for Data Mining, vol. 39 (2005), pp. 81–88

    Google Scholar 

  8. M. Holmes, A. Gray, C. Isbell, Fast SVD for large-scale matrices. In Workshop on Efficient Machine Learning at NIPS 58, 249–252 (2007)

    Google Scholar 

  9. Z. Zhang, F. Yang, K. Xia, R. Yang, A supervised lpp algorithm and its application to face recognition [j]. J. Electron. Inf. Technol. 3, 8 (2008)

    Google Scholar 

  10. S. Buchala, N. Davey, T.M. Gale, R.J. Frank, Analysis of linear and nonlinear dimensionality reduction methods for gender classification of face images. Int. J. Syst. Sci. 36(14), 931–942 (2005)

    Article  MathSciNet  Google Scholar 

  11. X.L. Zhang, Nonlinear dimensionality reduction of data by deep distributed random samplings, in Asian Conference on Machine Learning, vol. 2015, pp. 221–233

    Google Scholar 

  12. F. Namugera, Dimensionality Reduction of High-Dimensional Noisy Data. African Institute of Mathematical Sciences (AIMS), Senegal (2017)

    Google Scholar 

  13. T. Lesort, N. Díaz-Rodríguez, J.-F. Goudou, D. Filliat, State representation learning for control: an overview. Neural Netw. 108, 379–392 (2018)

    Article  Google Scholar 

  14. E. Shchurenkova, Dimension Reduction Using Independent Component Analysis with an Application in Business Psychology, University of British Columbia, 2017 PhD. thesis

    Google Scholar 

  15. J. Rahmanishamsi, A. Donati, M.R. Aghabozorgi, A copula-based ica algorithm and its application to time series clustering. J. Classif. 35(2), 230–249 (2018)

    Article  MathSciNet  Google Scholar 

  16. Y. Xin, Q. Wu, Q. Zhao, Q. Wu, Semi-supervised regularized discriminant analysis for Eeg-based Bci system, in International Conference on Intelligent Data Engineering and Automated Learning (Springer, 2017), pp. 516–523

    Google Scholar 

  17. M. Verleysen, D. François, The curse of dimensionality in data mining and time series prediction, in International Work-Conference on Artificial Neural Networks (Springer, 2005), pp. 758–770

    Google Scholar 

  18. J.P. Cunningham, Z. Ghahramani, Linear dimensionality reduction: survey, in-sights, and generalizations. J. Mach. Learn. Res. 16(1), 2859–2900 (2015)

    MathSciNet  MATH  Google Scholar 

  19. A. Gisbrecht, B. Hammer, Data visualization by nonlinear dimensionality reduction. Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 5(2), 51–73 (2015)

    Google Scholar 

  20. B. Kuster, A.M. Gholami, A.C. Culhane, Dimension reduction techniques for the integrative analysis of multi-omics data. Brief. Bioinform. 17(4), 628–641 (2016)

    Article  Google Scholar 

  21. C.K. Chandrasekhar, H. Bagyalakshmi, M.R. Srinivasan, M. Gallo, Partial ridge regression under multicollinearity. J. Appl. Statistics 43 (2016)

    Google Scholar 

  22. H.H. Haeman, Modern Factor Analysis, 3rd Revision Published 1976 by The University of Chicago

    Google Scholar 

  23. N.B. Erichson, P. Zheng, K. Manohar, S.L. Brunton, J.N. Kutz, A.Y. Aravkin, Sparse Principal Component Analysis Via Variable Projection. arXiv preprint arXiv:1804.00341

  24. R.P. McDonald, Factor Analysis and Related Methods. Lawrence Erlaum Associates

    Google Scholar 

  25. H. Hotelling, Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24(6), 417 (1933)

    Article  Google Scholar 

  26. S. Deegalla, H. Boström, K. Walgama, Choice of dimensionality reduction methods for feature and classifier fusion with nearest neighbor classifiers, in 15th International Conference on Information Fusion (FUSION) (IEEE, 2012), pp. 875–881

    Google Scholar 

  27. I.T. Jolliffe, J. Cadima, Principal component analysis: a review and recent developments. Philos. Trans. R. Soc. A: Math. Phys. Eng. Sci. 374(2065), 20150202 (2016)

    Article  MathSciNet  Google Scholar 

  28. T. Radüntz, J. Scouten, O. Hochmuth, B. Meffert, Automated EEG artifact elimination by applying machine learning algorithms to ica-based features. J. Neural Eng. 14(4), 46004 (2017)

    Article  Google Scholar 

  29. M.F. Glasser, T.S. Coalson, J.D. Bijsterbosch, S.J. Harrison, M.P. Harms, A. Anticevic, D.C.V. Essen, S.M. Smith, Using temporal ica to selectively remove global noise while preserving global signal in functional MRI data. Neuroimage 181, 692–717 (2018)

    Article  Google Scholar 

  30. K. Pearson, Liii. on lines and planes of closest fit to systems of points in space, London Edinburgh Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901)

    Google Scholar 

  31. J.C. Loehlin, Latent Variable Models: An Introduction to Factor, Path, and Structural Analysis (American Psychological Association Press, Washington, 1998)

    MATH  Google Scholar 

  32. C. Ding, H. Xiaofeng, K-means clustering via principal component analysis, in ICML ‘04 Proceedings of the Twenty-First International Conference on Machine learning (2004), p. 29

    Google Scholar 

  33. T. Bruce, Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications (American Psychological Association Press, Washington, 2004)

    Google Scholar 

  34. M. Nascimento, F.F.e. Silva, T. Sáfadi, A.C.C. Nascimento, T.E.M. Ferreira, L.M.A. Barroso, C.F. Azevedo, S.E.F. Guimarães, N.V.L. Serão, Independent component analysis (ica) based-clustering of temporal rna-seq data. PloS one 12(7), e0181195 (2017)

    Google Scholar 

  35. C.F. Beckmann, S.M. Smith, Probabilistic independent component analysis for functional magnetic resonance imaging. IEEE Trans. Med. Imaging 23(2), 137–152 (2004)

    Article  Google Scholar 

  36. P. Ablin, J.-F. Cardoso, A. Gramfort, Faster ICA Under Orthogonal Constraint. arXiv, preprint rXiv:1711.10873

    Google Scholar 

  37. N. Abrahamsen, P. Rigollet, Sparse Gaussian ICA, arXiv preprint arXiv:1804.00408

  38. S.J. Press, S. Wilson, Choosing between logistic regression and discriminant analysis. J. Am. Stat. Assoc. 73(364), 699–705 (1978)

    Article  Google Scholar 

  39. C. Ecse, Dimensionality Reduction. pca. kernel pca, Lecture slides: COMP-652 and ECSE-608

    Google Scholar 

  40. L. Wiskott, Lecture Notes on Principal Component Analysis (2013)

    Google Scholar 

  41. M. Wan, G. Yang, C. Sun, M. Liu, Sparse two-dimensional discriminant locality-preserving projection (s2ddlpp) for feature extraction. Soft. Comput. 1–8 (2018)

    Google Scholar 

  42. D.T. Pham, P. Garat, Blind separation of the mixture of independent sources through a quasi-maximum likelihood approach. IEEE Trans. Signal Process. 45(7), 1712–1725 (1997)

    Article  Google Scholar 

  43. X.-s. He, F. He, A.l. He, Super-gaussian bss using fast-ica with chebyshev–pade approximant. Circuits Syst. Signal Process. 37(1), 305–341 (2018)

    Google Scholar 

  44. Z. Yang, S. La Conte, X. Weng, X. Hu, Ranking and averaging independent component analysis by reproducibility (raicar). Hum. Brain Mapp. 29(6), 711–725 (2008)

    Article  Google Scholar 

  45. H. Ince, T.B. Trafalis, A hybrid forecasting model for stock market prediction. Econ. Comput. Econ. Cybern. Stud. Res. 51(3), 263–280 (2017)

    Google Scholar 

  46. N. Kambhatla, T.K. Leen, Dimension Reduction by Local Principal Component

    Google Scholar 

  47. B. Wang, Y. Hu, J. Gao, Y. Sun, H. Chen, B. Yin, Locality Preserving Projections for Grassmann Manifold, arXiv preprint arXiv:1704.08458

  48. S. Ahmadkhani, P. Adibi, Face recognition using supervised probabilistic principal component analysis mixture model in dimensionality reduction without loss framework. IET Comput. Vision 10(3), 193–201 (2016)

    Article  Google Scholar 

  49. H. Zhao, S. Sun, Z. Jing, Local-information-based uncorrelated feature extraction. Opt. Eng. 45(2), 20505 (2006)

    Article  Google Scholar 

  50. S. Chen, H. Zhao, M. Kong, B. Luo, 2D-lpp: a two-dimensional extension of locality preserving projections. Neurocomputing 70(4–6), 912–921 (2007)

    Article  Google Scholar 

  51. M. Balaanand, N. Karthikeyan, S. Karthik, R. Varatharajan, G. Manogaran, C.B. Sivaparthipan, An enhanced graph-based semi-supervised learning algorithm to detect fake users on Twitter. J. Supercomput. 75(9), 6085–6105 (2019). https://doi.org/10.1007/s11227-019-02948-w

    Article  Google Scholar 

  52. T.N. Nguyen, B. Liu, S. Chu, D. Do, T.D. Nguyen, WRSNs: toward an efficient scheduling for mobile chargers. IEEE Sensors J. 20(12), 6753–6761, 15 June 15, 2020. https://doi.org/10.1109/JSEN.2020.2974255

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Anne Lourdu Grace .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Grace, A.L., Thenmozhi, M. (2022). Survey of Popular Linear Dimensionality Reduction Techniques. In: Manogaran, G., Shanthini, A., Vadivu, G. (eds) Proceedings of International Conference on Deep Learning, Computing and Intelligence. Advances in Intelligent Systems and Computing, vol 1396. Springer, Singapore. https://doi.org/10.1007/978-981-16-5652-1_53

Download citation

Publish with us

Policies and ethics