Abstract
In this paper, we propose a shrinkage framework for jointly estimating multiple covariance matrices by shrinking the sample covariance matrices towards the pooled sample covariance matrix. This framework allows us to borrow information across different groups. We derive the optimal shrinkage parameters under the Stein and quadratic loss functions, and prove that our derived estimators are asymptotically optimal when the sample size or the number of groups tends to infinity. Simulation studies demonstrate that our proposed shrinkage method performs favorably compared to the existing methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barnard J, McCulloch R, Meng X (2000) Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Statistica Sinica 10:1281–1311
Boik RJ (2002) Spectral models for covariance matrices. Biometrika 89:159–182
Boik RJ (2003) Principal component models for correlation matrices. Biometrika 90:679–701
Cai T, Li H, Liu W, Xie J (2016) Joint estimation of multiple high-dimensional precision matrices. Statistica Sinica 26:445–464
Danaher P, Wang P, Witten DM (2014) The joint graphical lasso for inverse covariance estimation across multiple classes. J R Stat Soc Ser B 76:373–397
Daniels MJ (2006) Bayesian modeling of several covariance matrices and some results on propriety of the posterior for linear regression with correlated or heterogeneous errors. J Multivar Anal 97:1185–1207
Ferguson TS (1996) A course in large sample theory. Chapman & Hall, London
Friedman JH (1989) Regularized discriminant analysis. J Am Stat Assoc 84:165–175
Gaskins J, Daniels M (2016) Covariance partition priors: a Bayesian approach to simultaneous covariance estimation for longitudinal data. J Comput Gr Stat 25:167–186
Guionnet A (2009) Large random matrices: lectures on macroscopic asymptotics. Springer, Berlin
Guo J, Levina E, Michailidis G, Zhu J (2011) Joint estimation of multiple graphical models. Biometrika 98:1–15
Haff L (1980) Empirical Bayes estimation of the multivariate normal covariance matrix. Ann Stat 8:586–597
Haff L (1991) The variational form of certain Bayes estimators. Ann Stat 19:1163–1190
Hoff PD (2009) A hierarchical eigenmodel for pooled covariance estimation. J R Stat Soc Ser B 71:971–992
Ledoit O, Wolf M (2004) A well-conditioned estimator for large-dimensional covariance matrices. J Multivar Anal 88:365–411
Ledoit O, Wolf M (2012) Nonlinear shrinkage estimation of large-dimensional covariance matrices. Ann Stat 40:1024–1060
Le Y, Hastie T (2016) Sparse quadratic discriminant analysis and community Bayes, arXiv preprintarXiv:1407.4543
Letac G, Massam H (2004) All invariant moments of the Wishart distribution. Scand J Stat 31:295–318
Manly BF, Rayner J (1987) The comparison of sample covariance matrices using likelihood ratio tests. Biometrika 74:841–847
Pourahmadi M, Daniels MJ, Park T (2007) Simultaneous modelling of the Cholesky decomposition of several covariance matrices. J Multivar Anal 98:568–587
Price BS, Geyer CJ, Rothman AJ (2015) Ridge fusion in statistical learning. J Comput Gr Stat 24:439–454
Ramey JA, Stein CK, Young PD, Young DM (2016) High-dimensional regularized discriminant analysis, arXiv preprint arXiv:1602.01182
Schäfer J, Strimmer K (2005) A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat Appl Genet Mol Biol 4:1175–1189
Stein C (1956) Inadmissibility of the usual estimator of the mean of a multivariate normal distribution. Proc Third Berkeley Symp Math Stat Prob 1:197–206
Tao T (2012) Topics in random matrix theory. American Mathematical Society, Providence
Tong T, Wang Y (2007) Optimal shrinkage estimation of variances with applications to microarray data analysis. J Am Stat Assoc 102:113–122
Tong T, Jang H, Wang Y (2012) James–Stein type estimators of variances. J Multivar Anal 107:232–243
Yang R, Berger JO (1994) Estimation of a covariance matrix using the reference prior. Ann Stat 22:1195–1211
Acknowledgements
Zongliang Hu’s research was supported by the Natural Science Foundation grant (No. 2019083) of Shenzhen University, and the Foundation and Applied Basic Research Programs (No. 2019A1515110449) of Guangdong Province of China. Zhishui Hu’s research was partially supported by the National Natural Science Foundation of China grant (No. 11671373). Tiejun Tong’s research was supported by the General Research Fund (No. HKBU12303918), the National Natural Science Foundation of China (No. 11671338), and the Initiation Grant for Faculty Niche Research Areas (No. RC-IG-FNRA/17-18/13) of Hong Kong Baptist University. Yuedong Wang’s research was partially supported by the National Science Foundation grant (No. DMS-1507620). The authors also thank the editor, the associate editor, and two reviewers for their constructive comments that have led to a substantial improvement of this paper.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Hu, Z., Hu, Z., Dong, K. et al. A shrinkage approach to joint estimation of multiple covariance matrices. Metrika 84, 339–374 (2021). https://doi.org/10.1007/s00184-020-00781-3
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00184-020-00781-3