On the penalized maximum likelihood estimation of high-dimensional approximate factor model

  • Shaoxin WangEmail author
  • Hu Yang
  • Chaoli Yao
Original Paper


In this paper, we mainly focus on the penalized maximum likelihood estimation of the high-dimensional approximate factor model. Since the current estimation procedure can not guarantee the positive definiteness of the error covariance matrix, by reformulating the estimation of error covariance matrix and based on the lagrangian duality, we propose an accelerated proximal gradient (APG) algorithm to give a positive definite estimate of the error covariance matrix. Combined the APG algorithm with EM method, a new estimation procedure is proposed to estimate the high-dimensional approximate factor model. The new method not only gives positive definite estimate of error covariance matrix but also improves the efficiency of estimation for the high-dimensional approximate factor model. Although the proposed algorithm can not guarantee a global unique solution, it enjoys a desirable non-increasing property. The efficiency of the new algorithm on estimation and forecasting is also investigated via simulation and real data analysis.


Error covariance matrix Positive definiteness EM precedure Accelerated proximal gradient algorithm 



The authors wish to thank the anonymous reviewers and the Editor for their helpful and very detailed comments, which have led a substantial improvement to the presentation of their paper. They also appreciate Yuan Liao for sharing the code used in Bai and Liao (2016).


  1. Bai J (2003) Inferential theory for factor models of large dimensions. Econometrica 71:135–171MathSciNetzbMATHGoogle Scholar
  2. Bai J, Li K (2016) Maximum likelihood estimation and inference for approximate factor models of high dimension. Rev Econ Stat 98(2):298–309Google Scholar
  3. Bai J, Liao Y (2016) Efficient estimation of approximate factor models via penalized maximum likelihood. J Econom 191:1–18MathSciNetzbMATHGoogle Scholar
  4. Bai J, Shi S (2011) Estimating high dimensional covariance matrices and its applications. Ann Econ Finance 12:199–215Google Scholar
  5. Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci 2:183–202MathSciNetzbMATHGoogle Scholar
  6. Bickel P, Levina E (2008) Covariance regularization by thresholding. Ann Stat 36:2577–2604MathSciNetzbMATHGoogle Scholar
  7. Bien J, Tibshirani R (2011) Sparse estimation of a covariance matrix. Biometrika 98:807–820MathSciNetzbMATHGoogle Scholar
  8. Boivin J, Ng S (2005) Understanding and comparing factor based macroeconomic forecasts. Int J Cent Bank 1(3):117–152Google Scholar
  9. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3:1–122zbMATHGoogle Scholar
  10. Bühlmann P, Van De Geer S (2011) Statistics for high-dimensional data: methods, theory and applications. Springer, New YorkzbMATHGoogle Scholar
  11. Cai T, Liu W (2011) Adaptive thresholding for sparse covariance matrix estimation. J Am Stat Assoc 106(494):672–684MathSciNetzbMATHGoogle Scholar
  12. Chamberlain G, Rothschild M (1983) Arbitrage, factor structure, and mean-variance analysis on large asset markets. Econometrica 51:1281–1304MathSciNetzbMATHGoogle Scholar
  13. Choi I (2012) Efficient estimation of factor models. J Econom Theory 28:274–308MathSciNetzbMATHGoogle Scholar
  14. Cui Y, Leng C, Sun D (2016) Sparse estimation of high-dimensional correlation matrices. Comput Stat Data Anal 93:390–403MathSciNetzbMATHGoogle Scholar
  15. Doz C, Giannone D, Reichlin L (2012) A quasi-maximum likelihood approach for large, approximate dynamic factor models. Rev Econ Stat 94(4):1014–1024zbMATHGoogle Scholar
  16. Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96:1348–1360MathSciNetzbMATHGoogle Scholar
  17. Fan J, Fan Y, Lv J (2008) High dimensional covariance matrix estimation using a factor model. J Econom 147(1):186–197MathSciNetzbMATHGoogle Scholar
  18. Fan J, Liao Y, Mincheva M (2011) High-dimensional covariance matrix estimation in approximate factor models. Ann Stat 39(6):3320–3356MathSciNetzbMATHGoogle Scholar
  19. Fan J, Liao Y, Mincheva M (2013) Large covariance estimation by thresholding principal orthogonal complements. J R Stat Soc Ser B Stat Methodol 75:603–680MathSciNetGoogle Scholar
  20. Fan J, Liao Y, Liu H (2016) An overview on the estimation of large covariance and precision matrices. Econom J 19:1–32MathSciNetGoogle Scholar
  21. Forni M, Hallin M, Lippi M, Reichlin L (2000) The generalized dynamic-factor model: identification and estimation. Rev Econ Stat 82:540–554zbMATHGoogle Scholar
  22. Geweke J (1978) The dynamic factor analysis of economic time series models. Social Systems Research Institute, University of Wisconsin-Madison, MadisonzbMATHGoogle Scholar
  23. Hiriart-Urruty JB, Lemaréchal C (1993) Convex analysis and minimization algorithms II: advanced theory and bundle methods. Grundlehren der mathematischen Wissenschaften, vol 306. Springer, BerlinzbMATHGoogle Scholar
  24. Hirose K, Miyuki I (2018) Sparse factor regression via penalized maximum likelihood estimation. Stat Pap 59:633–662MathSciNetzbMATHGoogle Scholar
  25. Hirose K, Yamamoto M (2015) Sparse estimation via nonconcave penalized likelihood in factor analysis model. Stat Comput 25:863–875MathSciNetzbMATHGoogle Scholar
  26. Lange K (2016) MM optimization algorithms. SIAM, PhiladelphiazbMATHGoogle Scholar
  27. Lawley D, Maxwell A (1971) Factor analysis as a statistical method, 2nd edn. Butterworths, LondonzbMATHGoogle Scholar
  28. Liu H, Wang L, Zhao T (2014) Sparse covariance matrix estimation with eigenvalue constraints. J Comput Graph Stat 23(2):439–459MathSciNetGoogle Scholar
  29. Luciani M (2014) Forecasting with approximate dynamic factor models: the role of non-pervasive shocks. Int J Forecast 30(1):20–29Google Scholar
  30. Ludvigson S, Ng S (2011) A factor analysis of bond risk premia. In: Ulah A, Giles D (eds) Handbook of empirical economics and finance. Chapman and Hall, London, pp 313–372Google Scholar
  31. Ma S, Xue L, Zou H (2013) Alternating direction methods for latent variable Gaussian graphical model selection. Neural Comput 25:2172–2198MathSciNetGoogle Scholar
  32. McLachlan G, Krishnan T (2007) The EM algorithm and extensions. Wiley, New YorkzbMATHGoogle Scholar
  33. Mulaik SA (2009) Foundations of factor analysis, 2nd edn. CRC Press, Boca RatonzbMATHGoogle Scholar
  34. Ross SA (1976) The arbitrage theory of capital asset pricing. J Econ Theory 13:341–360MathSciNetGoogle Scholar
  35. Rothman AJ (2012) Positive definite estimators of large covariance matrices. Biometrika 99(3):733–740MathSciNetzbMATHGoogle Scholar
  36. Rothman AJ, Bickel P, Levina E, Zhu J (2008) Sparse permutation invariant covariance estimation. Electron J Stat 2:494–515MathSciNetzbMATHGoogle Scholar
  37. Rothman AJ, Levina E, Zhu J (2009) Generalized thresholding of large covariance matrices. J Am Stat Assoc 104(485):177–186MathSciNetzbMATHGoogle Scholar
  38. Rubin DB, Thayer DT (1982) EM algorithms for ML factor analysis. Psychometrika 47:69–76MathSciNetzbMATHGoogle Scholar
  39. Stock JH, Watson MW (2002) Forecasting using principal components from a large number of predictors. J Am Stat Assoc 97:1167–1179MathSciNetzbMATHGoogle Scholar
  40. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Stat Methodol 58:267–288MathSciNetzbMATHGoogle Scholar
  41. Xue L, Ma S, Zou H (2012) Positive-definite \(l_1\)-penalized estimation of large covariance matrices. J Am Stat Assoc 107:1480–1491zbMATHGoogle Scholar
  42. Zhang CH (2010) Nearly unbiased variable selection under minimax concave penalty. Ann Stat 38:894–942MathSciNetzbMATHGoogle Scholar
  43. Zou H (2006) The adaptive lasso and its oracle properties. J Am Stat Assoc 101:1418–1429MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of StatisticsQufu Normal UniversityQufuPeople’s Republic of China
  2. 2.College of Mathematics and StatisticsChongqing UniversityChongqingPeople’s Republic of China

Personalised recommendations