Skip to main content
SpringerLink
Log in

Graphical tool of sparse factor analysis

  • Invited Paper
  • Published:
Behaviormetrika Aims and scope Submit manuscript

An Erratum to this article was published on 01 January 2017

Abstract

Factor analysis is a popular statistical model for analyzing correlation structures among observed variables. It is well known that this model has a rotational indeterminacy. Traditionally, the model parameters are estimated by the maximum likelihood method; then, factor rotation methods are applied to obtain an interpretable factor loading matrix. Recently, several sparse estimation procedures via penalization have been developed. Sparse estimation via penalization is an alternative to the factor rotation; it leads to an interpretable and sufficiently sparse solution. In this paper, we give an overview of several sparse factor analysis models, followed by a discussion of a relation between ordinary factor rotation and penalized maximum likelihood approaches. Then, we introduce a novel analyzing tool wherein a user can select a model that is easy to interpret and also possesses desirable values of goodness-of-fit indices based on the graphical representation of solution path.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. The definition of BIC for the penalized FA model is given in Section 4.

  2. BFRM is a comprehensive software implementation of sparse statistical models for high-dimensional data analysis, structure discovery and prediction. It is available at https://www2.stat.duke.edu/research/software/west/bfrm/index.html.

  3. The current version is 2.2.

  4. Since there is indeterminacy of the order of factors, the figure made by the above code can be different in the order of factors. This is true for other figures shown in this paper.

References

  • Amemiya Y, Anderson TW (1990) Asymptotic chi-square tests for a large class of factor analysis models. Ann Stat 18:1453–1463

    Article  MathSciNet  MATH  Google Scholar 

  • Anderson TW (2003) An introduction to multivariate statistical analysis. 3rd ed. Wiley

  • Anderson TW, Amemiya Y (1988) The asymptotic normal distribution of estimators in factor analysis under general conditions. Ann Stat 16:759–771

    Article  MathSciNet  MATH  Google Scholar 

  • Asparouhov T, Muthén B (2009) Exploratory structural equation modeling. Struct Equ Model Multidiscip J 16:397–438

    Article  MathSciNet  Google Scholar 

  • Bai J, Li K (2012) Statistical analysis of factor models of high dimension. Ann Stat 40:436–465

    Article  MathSciNet  MATH  Google Scholar 

  • Bentler P (2007) Covariance structure models for maximal reliability of unit-weighted composites. In Handbook of latent variable and related models, North Holland, pp 1–17

  • Bernaards CA, Jennrich RI (2003) Orthomax rotation and perfect simple structure. Psychometrika 68:585–588

    Article  MathSciNet  MATH  Google Scholar 

  • Browne MW (1984) Asymptotically distribution-free methods for the analysis of covariance structures. Br J Math Stat Psychol 37:62–83

    Article  MathSciNet  MATH  Google Scholar 

  • Browne MW, Shapiro A (1988) Robustness of normal theory methods in the analysis of linear latent variate models. Br J Math Stat Psychol 41:193–208

    Article  MathSciNet  MATH  Google Scholar 

  • Browne MW (2001) An overview of analytic rotation in exploratory factor analysis. Multivar Behav Res 36:111–150

    Article  Google Scholar 

  • Carroll JB (1953) An analytical solution for approximating simple structure in factor analysis. Psychometrika 18:23–38

    Article  MATH  Google Scholar 

  • Carvalho CM, Chang J, Lucas JE, Nevins JR, Wang Q, West M (2008) High-dimensional sparse factor modeling: applications in gene expression genomics. J Am Stat Assoc 103:1438–1456

    Article  MathSciNet  MATH  Google Scholar 

  • Choi J, Oehlert G, Zou H (2010) A penalized maximum likelihood approach to sparse factor analysis. Stat Interface 3:429–436

    Article  MathSciNet  MATH  Google Scholar 

  • Clarke MA (1970) A rapidly convergent method for maximum-likelihood factor analysis. Br J Math Stat Psychol 23:43–52

    Article  MATH  Google Scholar 

  • Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression (with discussion). Ann Stat 32:407–499

    Article  MathSciNet  MATH  Google Scholar 

  • Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96:1348–1360

    Article  MathSciNet  MATH  Google Scholar 

  • Friedman J, Hastie T, Höfling H, Tibshirani R (2007) Pathwise coordinate optimization. Ann Appl Stat 1:302–332

    Article  MathSciNet  MATH  Google Scholar 

  • Friedman J, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinate descent. J Stat Softw 33:1–22

    Article  Google Scholar 

  • Harman HH (1976) Modern factor analysis. 3rd ed. University of Chicago Press

  • Hastie T, Rosset S, Tibshirani R, Zhu J (2004) The entire regularization path for the support vector machine. J Mach Learn Res 5:1391–1415

    MathSciNet  MATH  Google Scholar 

  • Hirose K (2016) Simple structure estimation via prenet penalization. arXiv:1607.01145

  • Hirose K, Yamamoto M (2014) Estimation of an oblique structure via penalized likelihood factor analysis. Comput Stat Data Anal 79:120–132

    Article  MathSciNet  Google Scholar 

  • Hirose K, Yamamoto M (2015) Sparse estimation via nonconcave penalized likelihood in factor analysis model. Stat Comput 25:863–875

    Article  MathSciNet  MATH  Google Scholar 

  • Hirose K, Yamamoto M, Nagata H (2016) Fanc: Penalized Likelihood Factor Analysis via Nonconvex Penalty. R package version 2.2

  • Horst P (1965) Factor analysis of data matrices. Holt, Linehart & Winston, New York

    MATH  Google Scholar 

  • Jennrich RI (2001) A simple general procedure for orthogonal rotation. Psychometrika 66:289–306

    Article  MathSciNet  MATH  Google Scholar 

  • Jennrich RI (2002) A simple general procedure for oblique rotation. Psychometrika 67:7–20

    Article  MathSciNet  MATH  Google Scholar 

  • Jennrich RI (2004) Rotation to simple loadings using component loss functions: the orthogonal case. Psychometrika 69:257–273

    Article  MathSciNet  MATH  Google Scholar 

  • Jennrich RI (2006) Rotation to simple loadings using component loss functions: the oblique case. Psychometrika 71:173–191

    Article  MathSciNet  MATH  Google Scholar 

  • Jennrich RI, Robinson S (1969) A Newton-Raphson algorithm for maximum likelihood factor analysis. Psychometrika 34:111–123

    Article  MathSciNet  Google Scholar 

  • Jöreskog K (1967) Some contributions to maximum likelihood factor analysis. Psychometrika 32:443–482

    Article  MathSciNet  MATH  Google Scholar 

  • Kaiser HF (1958) The varimax criterion for analytic rotation in factor analysis. Psychometrika 23:187–200

    Article  MATH  Google Scholar 

  • Lorenzo-Seva U (2003) A factor simplicity index. Psychometrika 68:49–60

    Article  MathSciNet  MATH  Google Scholar 

  • Lucas J, Carvalho C, Wang Q, Bild A, Nevns JR, West M (2006) Sparse statistical modelling in gene expression genomics. In: Müller P, Do K, Vannucci M (eds) Bayesian inference for gene expression and proteomics. Cambridge University Press, Cambridge, pp 155–176

    Chapter  Google Scholar 

  • Ning L, Georgiou TT (2011) Sparse factor analysis via likelihood and \(\ell _{1}\) regularization. In: 50th IEEE Conference on Decision and Control and European Control Conference, pp 5188–5192

  • Revelle W (2016) psych: Procedures for Psychological, Psychometric, and Personality Research. R package version 1(6):6

  • Rubin D, Thayer D (1982) EM algorithms for ML factor analysis. Psychometrika 47:69–76

    Article  MathSciNet  MATH  Google Scholar 

  • Sherin RJ (1966) A matrix formulation of Kaiser’s varimax criterion. Psychometrika 31:535–538

    Article  MathSciNet  MATH  Google Scholar 

  • Thurstone LL (1947) Multiple factor analysis. University of Chicago Press

  • Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B 58:267–288

    MathSciNet  MATH  Google Scholar 

  • West M (2003) Bayesian factor regression models in the “large p, small n” paradigm. In: Bernardo JM, Bayarri MJ, Berger JO, Dawid AP, Heckerman D, Smith AFM, West M (eds.) Bayesian Statistics, 7. Oxford University Press, pp 723–732

  • Yamamoto M, Jennrich RI (2013) A cluster-based factor rotation. Br J Math Stat Psychol 66:488–502

    MathSciNet  Google Scholar 

  • Yates A (1987) Multivariate exploratory data analysis: a perspective on exploratory factor analysis. State University of New York Press, Albany

    Google Scholar 

  • Yuan M, Lin Y (2007) Model selection and estimation in the gaussian graphical model. Biometrika 94:19–35

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang C (2010) Nearly unbiased variable selection under minimax concave penalty. Ann Stat 38:894–942

    Article  MathSciNet  MATH  Google Scholar 

  • Zou H (2006) The adaptive lasso and its oracle properties. J Am Stat Assoc 101:1418–1429

    Article  MathSciNet  MATH  Google Scholar 

  • Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Ser B 67:301–320

    Article  MathSciNet  MATH  Google Scholar 

  • Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Graph Stat 15:265–286

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We thank the Editor and an anonymous reviewer for their constructive comments that helped to improve the quality of this article. This work was partially supported by the Japan Society for the Promotion of Science (JSPS) KAKENHI Grant Numbers 26730016 and 15K15949.

Author information

Authors and Affiliations

  1. Kyoto University, Kyoto, Japan

    Michio Yamamoto

  2. Kyushu University, Fukuoka, Japan

    Kei Hirose

  3. Osaka University, Osaka, Japan

    Haruhisa Nagata

Authors
  1. Michio Yamamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kei Hirose
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haruhisa Nagata
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michio Yamamoto.

Additional information

Communicated by Joe Suzuki.

An erratum to this article is available at http://dx.doi.org/10.1007/s41237-017-0017-9.

Appendix

Appendix

1.1 The complete R command lines for Sect. 4.1

figure a

1.2 The complete R command lines for Sect. 4.2

figure b

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yamamoto, M., Hirose, K. & Nagata, H. Graphical tool of sparse factor analysis. Behaviormetrika 44, 229–250 (2017). https://doi.org/10.1007/s41237-016-0007-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41237-016-0007-3

Keywords

Search

Navigation