Honest confidence regions and optimality in high-dimensional precision matrix estimation

TEST volume 26, pages143–162(2017)Cite this article

Abstract

We propose methodology for estimation of sparse precision matrices and statistical inference for their low-dimensional parameters in a high-dimensional setting where the number of parameters p can be much larger than the sample size. We show that the novel estimator achieves minimax rates in supremum norm and the low-dimensional components of the estimator have a Gaussian limiting distribution. These results hold uniformly over the class of precision matrices with row sparsity of small order \(\sqrt{n}/\log p\) and spectrum uniformly bounded, under a sub-Gaussian tail assumption on the margins of the true underlying distribution. Consequently, our results lead to uniformly valid confidence regions for low-dimensional parameters of the precision matrix. Thresholding the estimator leads to variable selection without imposing irrepresentability conditions. The performance of the method is demonstrated in a simulation study and on real data.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. Belloni A, Chernozhukov V, Hansen C (2014) Inference on treatment effects after selection amongst high-dimensional controls. Rev Econ Stud 81(2):608–650

    MathSciNet  Article  Google Scholar 

  2. Belloni A, Chernozhukov V, Wang L (2011) Square-root Lasso: Pivotal recovery of sparse signals via conic programming. Biometrika 98(4):791–806

    MathSciNet  Article  MATH  Google Scholar 

  3. Bickel PJ, Klaassen CA, Ritov Y, Wellner JA (1993) Efficient and adaptive estimation for semiparametric models. Springer, New York

    Google Scholar 

  4. Bickel PJ, Levina E (2008) Covariance regularization by thresholding. Ann Statist 36(6):2577–2604

    MathSciNet  Article  MATH  Google Scholar 

  5. Bühlmann P, van de Geer S (2011) Statistics for high-dimensional data. Springer, New York

    Google Scholar 

  6. Cai T, Liu W, Luo X (2011) A constrained l1 minimization approach to sparse precision matrix estimation. J Am Statist Assoc 106:594–607

    Article  MATH  Google Scholar 

  7. Candes E, Tao T (2007) The dantzig selector: statistical estimation when p is much larger than n. Ann Statist 35(6):2313–2351

    MathSciNet  Article  MATH  Google Scholar 

  8. Chatterjee A, Lahiri SN (2011) Bootstrapping lasso estimators. J Am Statist Assoc 106(494):608–625

    MathSciNet  Article  MATH  Google Scholar 

  9. Chatterjee A, Lahiri SN (2013) Rates of convergence of the adaptive LASSO estimators to the oracle distribution and higher order refinements by the bootstrap. Ann Statist 41(3)

  10. Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Statist 32(2):407–451

    MathSciNet  Article  MATH  Google Scholar 

  11. Friedman J, Hastie T, Tibshirani R (2008) Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9:432–441

    Article  MATH  Google Scholar 

  12. Janková J, van de Geer S (2015) Confidence intervals for high-dimensional inverse covariance estimation. Electron J Statist 9:1205–1229

    MathSciNet  Article  MATH  Google Scholar 

  13. Javanmard A, Montanari A (2013) Model selection for high-dimensional regression under the generalized irrepresentability condition. In: Burges C, Bottou L, Welling M, Ghahramani Z, Weinberger K (eds) Advances in neural information processing systems 26:3012–3020

  14. Javanmard A, Montanari A (2014) Confidence intervals and hypothesis testing for high-dimensional regression. J Mach Learn Res 15(1):2869–2909

    MathSciNet  MATH  Google Scholar 

  15. Knight K, Fu W (2000) Asymptotics for lasso-type estimators. Ann Statist 28(5):1356–1378

    MathSciNet  Article  MATH  Google Scholar 

  16. Lauritzen SL (1996) Graphical models. Clarendon Press, Oxford

    Google Scholar 

  17. Li KC (1989) Honest confidence regions for nonparametric regression. Ann Statist 17(3):1001–1008

    MathSciNet  Article  MATH  Google Scholar 

  18. Mazumder R, Hastie T (2012) The Graphical Lasso: New Insights and Alternatives. Electron J Statist, pp 2125–2149

  19. Meinshausen N, Bühlmann P (2006) High-dimensional graphs and variable selection with the lasso. Ann Statist 34(3):1436–1462

    MathSciNet  Article  MATH  Google Scholar 

  20. Ng B, Varoquaux G, P J-B, Thirion B (2013) A novel sparse group gaussian graphical model for functional connectivity estimation. Information Processing in Medical Imaging

  21. Ravikumar P, Raskutti G, Wainwright MJ, Yu B (2008) High-dimensional covariance estimation by minimizing l1-penalized log-determinant divergence. Electron J Statist 5:935–980

    Article  MATH  Google Scholar 

  22. Ren Z, Sun T, Zhang C-H, Zhou HH (2015) Asymptotic normality and optimalities in estimation of large gaussian graphical models. Ann Statist 43(3):991–1026

    MathSciNet  Article  MATH  Google Scholar 

  23. Rothman AJ, Bickel PJ, Levina E, Zhu J (2008) Sparse permutation invariant covariance estimation. Electron J Statist 2:494–515

    MathSciNet  Article  MATH  Google Scholar 

  24. Sun T, Zhang C-H (2012) Sparse matrix inversion with scaled Lasso. J Mach Learn Res 14:3385–3418

    MathSciNet  MATH  Google Scholar 

  25. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Stat Methodol 58:267–288

    MathSciNet  MATH  Google Scholar 

  26. van de Geer S (2016) Worst possible sub-directions in high-dimensional models. J Multi Anal 146:248–260

    MathSciNet  Article  MATH  Google Scholar 

  27. van de Geer S, Bühlmann P, Ritov Y, Dezeure R (2013) On asymptotically optimal confidence regions and tests for high-dimensional models. Ann Statist 42(3):1166–1202

    MathSciNet  Article  MATH  Google Scholar 

  28. van der Vaart A (2000) Asymptotic statistics. Cambridge University Press, Cambridge

    Google Scholar 

  29. Yuan M (2010) High dimensional inverse covariance matrix estimation via linear programming. J Mach Learn Res 11:2261–2286

    MathSciNet  MATH  Google Scholar 

  30. Yuan M, Lin Y (2007) Model selection and estimation in the gaussian graphical model. Biometrika, page 117

  31. Zhang C-H, Zhang SS (2014) Confidence intervals for low-dimensional parameters in high-dimensional linear models. J R Stat Soc Ser B Stat Methodol 76:217–242

    MathSciNet  Article  Google Scholar 

Download references

Author information

Affiliations

  1. Seminar for Statistics, ETH Zürich, Rämistrasse 101, 8092, Zürich, Switzerland

    Jana Janková & Sara van de Geer

Authors
  1. Jana Janková
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sara van de Geer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jana Janková.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 362 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Janková, J., van de Geer, S. Honest confidence regions and optimality in high-dimensional precision matrix estimation. TEST 26, 143–162 (2017). https://doi.org/10.1007/s11749-016-0503-5

Download citation

Keywords

  • Precision matrix
  • Sparsity
  • Inference
  • Asymptotic normality
  • Confidence regions

Mathematics Subject Classification

  • 62J07
  • 62F12