Skip to main content
SpringerLink
Log in

Sparse Versus Simple Structure Loadings

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

The component loadings are interpreted by considering their magnitudes, which indicates how strongly each of the original variables relates to the corresponding principal component. The usual ad hoc practice in the interpretation process is to ignore the variables with small absolute loadings or set to zero loadings smaller than some threshold value. This, in fact, makes the component loadings sparse in an artificial and a subjective way. We propose a new alternative approach, which produces sparse loadings in an optimal way. The introduced approach is illustrated on two well-known data sets and compared to the existing rotation methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Absil, P.-A., Mahony, R., & Sepulchre, R. (2008). Optimization algorithms on matrix manifolds. Princeton, NJ: Princeton University Press.

    Book  Google Scholar 

  • Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150.

    Article  Google Scholar 

  • Cadima, J., & Jolliffe, I. T. (1995). Loadings and correlations in the interpretations of principal components. Journal of Applied Statistics, 22, 203–214.

    Article  Google Scholar 

  • Candès, E. J., & Tao, T. (2007). The Dantzig selector: Statistical estimation when \(p\) is much larger than \(n\). Annals of Statistics, 35, 2313–2351.

    Article  Google Scholar 

  • Harman, H. H. (1976). Modern factor analysis (3rd ed.). Chicago, I: University of Chicago Press.

    Google Scholar 

  • Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(417–441), 498–520.

    Article  Google Scholar 

  • Jennrich, R. I. (2001). A simple general procedure for orthogonal rotation. Psychometrika, 66, 289–309.

    Article  Google Scholar 

  • Jennrich, R. I. (2004). Rotation to simple loadings using component loss functions: The orthogonal case. Psychometrika, 69, 257–273.

    Article  Google Scholar 

  • Jennrich, R. I. (2006). Rotation to simple loadings using component loss functions: The oblique case. Psychometrika, 71, 173–191.

    Article  Google Scholar 

  • Jennrich, R. I. (2007). Rotation methods, algorithms, and standard errors. In R. Cudeck & R. C. MacCallum (Eds.), Factor analysis at 100 (pp. 315–335). Mahwah, NJ: Lawrens Erlbaum Associates.

    Google Scholar 

  • Jolliffe, I. T. (2002). Principal component analysis (2nd ed.). New York: Springer.

    Google Scholar 

  • Jolliffe, I. T., Trendafilov, N. T., & Uddin, M. (2003). A modified principal component technique based on the LASSO. Journal of Computational and Graphical Statistics, 12, 531–547.

    Article  Google Scholar 

  • Mackey, L. (2009). Deflation methods for sparse PCA. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou (Eds.), Advances in neural information processing systems, 21, 1017–1024.

  • MATLAB. (2011). MATLAB R2011a. New York: The MathWorks Inc.

  • Mulaik, S. A. (2010). The foundations of factor analysis (2nd ed.). Boca Raton, FL: Chapman and Hall/CRC.

    Google Scholar 

  • Shen, H., & Huang, J. Z. (2008). Sparse principal component analysis via regularized low-rank matrix approximation. Journal of Multivariate Analysis, 99, 1015–1034.

    Article  Google Scholar 

  • Thurstone, L. L. (1935). The vectors of mind. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Thurstone, L. L. (1947). Multiple factor analysis. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Trendafilov, N. T. (2006). The dynamical system approach to multivariate data analysis, a review. Journal of Computational and Graphical Statistics, 50, 628–650.

    Article  Google Scholar 

  • Trendafilov, N. T. (2013). From simple structure to sparse components: A review. Computational Statistics. doi:10.1007/s00180-013-0434-5. Special Issue: Sparse Methods in Data Analysis.

  • Zou, H., Hastie, T., & Tibshirani, R. (2006). Sparse principal component analysis. Journal of Computational and Graphical Statistics, 15, 265–286.

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by a grant RPG-2013-211 from The Leverhulme Trust, UK.

Author information

Authors and Affiliations

  1. Department of Mathematics and Statistics, Open University, Walton Hall, Milton Keynes, MK7 6AA, UK

    Nickolay T. Trendafilov

  2. Graduate School of Human Sciences, Osaka University, Suita, Japan

    Kohei Adachi

Authors
  1. Nickolay T. Trendafilov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kohei Adachi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nickolay T. Trendafilov.

Additional information

Correspondence should be sent to Nickolay T. Trendafilov, Department of Mathematics and Statistics, Open University, Walton Hall, Milton Keynes MK7 6AA, UK. Email: Nickolay.Trendafilov@open.ac.uk

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Trendafilov, N.T., Adachi, K. Sparse Versus Simple Structure Loadings. Psychometrika 80, 776–790 (2015). https://doi.org/10.1007/s11336-014-9416-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-014-9416-y

Keywords

Search

Navigation