Skip to main content
SpringerLink
Log in

Graphical Models

  • Chapter
  • First Online:
Fundamentals of High-Dimensional Statistics

Part of the book series: Springer Texts in Statistics ((STS))

Abstract

The beginning of this book illustrates that linear regression models can describe the relationships between the genes’ copy numbers and a biomarker. However, those models do not provide information about the relationships among the copy numbers themselves. To describe such relationships, we use a different type of models, called graphical models. We neglect the biomarker and summarize the measured copy numbers in vector-valued observations, where each vector corresponds to a specific subject and each coordinate of these vectors to a specific gene (in the linear regression model, these vectors are the rows of the design matrix). Graphical models then formulate the relationships among the copy numbers as conditional dependence networks among the coordinates of the vector-valued observations. If the observations follow a multivariate Gauss distribution, we speak of Gaussian graphical models. This is the most common class of graphical models and, therefore, the focus of this chapter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Bibliography

  • Aitchison, J. (1982). The statistical analysis of compositional data. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 44(2), 139–160.

    MathSciNet  MATH  Google Scholar 

  • Albert, A. (1972). Regression and the Moore–Penrose pseudoinverse. Elsevier.

    MATH  Google Scholar 

  • Almal, S., & Padh, H. (2012). Implications of gene copy-number variation in health and diseases. Journal of Human Genetics, 57(1), 6.

    Article  Google Scholar 

  • Anscombe, F. (1948). The transformation of Poisson, binomial and negative-binomial data. Biometrika, 35(3/4), 246–254.

    Article  MathSciNet  MATH  Google Scholar 

  • Antoniadis, A. (2010). Comments on: â„“1-penalization for mixture regression models. Test, 19, 257–258.

    Article  MathSciNet  MATH  Google Scholar 

  • Arlot, S., & Celisse, A. (2010). A survey of cross-validation procedures for model selection. Statistics Surveys, 4, 40–79.

    Article  MathSciNet  MATH  Google Scholar 

  • Bakin, S. (1999). Adaptive regression and model selection in data mining problems, PhD thesis, The Australian National University, Canberra.

    Google Scholar 

  • Banerjee, O., El Ghaoui, L., & d’Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9, 485–516.

    MathSciNet  MATH  Google Scholar 

  • Bellec, P., & Tsybakov, A. (2017). Bounds on the prediction error of penalized least squares estimators with convex penalty. Modern Problems of Stochastic Analysis and Statistics, 208, 315–333.

    Article  MathSciNet  MATH  Google Scholar 

  • Belloni, A., & Chernozhukov, V. (2013). Least squares after model selection in high-dimensional sparse models. Bernoulli, 19(2), 521–547.

    Article  MathSciNet  MATH  Google Scholar 

  • Belloni, A., Chernozhukov, V., & Wang, L. ( 2011). Square-root lasso: Pivotal recovery of sparse signals via conic programming. Biometrika, 98(4), 791–806.

    Article  MathSciNet  MATH  Google Scholar 

  • Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 36(2), 192–236.

    MathSciNet  MATH  Google Scholar 

  • Bickel, P., Klaassen, C., Ritov, Y., & Wellner, J. (1993). Efficient and adaptive estimation for semiparametric models. Johns Hopkins University Press.

    MATH  Google Scholar 

  • Bickel, P., Ritov, Y., & Tsybakov, A. (2009). Simultaneous analysis of lasso and Dantzig selector. The Annals of Statistics, 37(4), 1705–1732.

    Article  MathSciNet  MATH  Google Scholar 

  • Bien, J., Gaynanova, I., Lederer, J., & MĂĽller, C. (2018a). Non-convex global minimization and false discovery rate control for the TREX. Journal of Computational and Graphical Statistics, 27(1), 23–33.

    Article  MathSciNet  Google Scholar 

  • Bien, J., Gaynanova, I., Lederer, J., & MĂĽller, C. (2018b). Prediction error bounds for linear regression with the TREX. Test, 28(2), 451–474.

    Article  MathSciNet  MATH  Google Scholar 

  • Bien, J., & Wegkamp, M. (2013). Discussion of: Correlated variables in regression: Clustering and sparse estimation. Journal of Statistical Planning and Inference, 143(11), 1859–1862.

    Article  MathSciNet  MATH  Google Scholar 

  • Borgelt, C., & Kruse, R. (2002). Graphical models: Methods for data analysis and mining. Wiley.

    MATH  Google Scholar 

  • Boucheron, S., Lugosi, G., & Massart, P. (2013), Concentration inequalities: A nonasymptotic theory of independence. Oxford University Press.

    Book  MATH  Google Scholar 

  • Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge University Press.

    Book  MATH  Google Scholar 

  • Bu, Y., & Lederer, J. (2017). Integrating additional knowledge into estimation of graphical models. arXiv:1704.02739.

    Google Scholar 

  • Bunea, F., Lederer, J., & She, Y. (2014). The group square-root lasso: Theoretical properties and fast algorithms. IEEE Transactions on Information Theory, 60(2), 1313–1325.

    Article  MathSciNet  MATH  Google Scholar 

  • Cai, T., Liu, W., & Luo, X. (2011). A constrained â„“1 minimization approach to sparse precision matrix estimation. Journal of the American Statistical Association, 106(494), 594–607.

    Article  MathSciNet  MATH  Google Scholar 

  • Celisse, A. (2008), Model selection via cross-validation in density estimation, regression, and change-points detection, PhD thesis, UniversitĂ© Paris Sud-Paris XI.

    Google Scholar 

  • Chatterjee, S., & Jafarov, J. (2015). Prediction error of cross-validated lasso. arXiv:1502.06291.

    Google Scholar 

  • ChĂ©telat, D., Lederer, J., & Salmon, J. (2017). Optimal two-step prediction in regression. Electronic Journal of Statistics, 11(1), 2519–2546.

    Article  MathSciNet  MATH  Google Scholar 

  • Chichignoud, M., Lederer, J., & Wainwright, M. (2016). A practical scheme and fast algorithm to tune the lasso with optimality guarantees. Journal of Machine Learning Research, 17(1), 1–20.

    MathSciNet  MATH  Google Scholar 

  • Dalalyan, A., Hebiri, M., & Lederer, J. (2017). On the prediction performance of the lasso. Bernoulli, 23(1), 552–581.

    Article  MathSciNet  MATH  Google Scholar 

  • Dettling, M., & BĂĽhlmann, P. (2004). Finding predictive gene groups from microarray data. Journal of Multivariate Analysis, 90(1), 106–131.

    Article  MathSciNet  MATH  Google Scholar 

  • Diesner, J., & Carley, K. (2005). Exploration of communication networks from the Enron email corpus. In SIAM International Conference on Data Mining (pp. 3–14).

    Google Scholar 

  • Dobra, A., Hans, C., Jones, B., Nevins, J., Yao, G., & West, M. (2004). Sparse graphical models for exploring gene expression data. Journal of Multivariate Analysis, 90(1), 196–212.

    Article  MathSciNet  MATH  Google Scholar 

  • Dudley, R. (2002), Real analysis and probability (Vol. 74). Cambridge University Press.

    Google Scholar 

  • Durrett, R. (2010), Probability: Theory and examples (4th ed.). Cambridge University Press.

    Book  MATH  Google Scholar 

  • Edwards, D. (2012), Introduction to graphical modelling. Springer.

    MATH  Google Scholar 

  • Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004). Least angle regression. Annals of Statistics, 32(2), 407–499.

    Article  MathSciNet  MATH  Google Scholar 

  • Engl, H., Hanke, M., & Neubauer, A. (1996). Regularization of inverse problems (Vol. 375). Springer.

    Google Scholar 

  • Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.

    Article  MathSciNet  MATH  Google Scholar 

  • Frank, I., & Friedman, J. (1993). A statistical view of some chemometrics regression tools. Technometrics, 35(2), 109–135.

    Article  MATH  Google Scholar 

  • Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3), 432–441.

    Article  MATH  Google Scholar 

  • Fultz, N., Bonmassar, G., Setsompop, K., Stickgold, R., Rosen, B., Polimeni, J. , & Lewis, L. (2019). Coupled electrophysiological, hemodynamic, and cerebrospinal fluid oscillations in human sleep. Science, 366(6465), 628–631.

    Article  Google Scholar 

  • Gallavotti, G. (2013), Statistical mechanics: A short treatise. Springer.

    MATH  Google Scholar 

  • Geisser, S. (1975). The predictive sample reuse method with applications. Journal of the American Statistical Association, 70(350), 320–328.

    Article  MATH  Google Scholar 

  • Gold, D., Lederer, J., & Tau, J. (2020). Inference for high-dimensional nested regression. Journal of Econometrics, 217(1), 79–111.

    Article  MathSciNet  MATH  Google Scholar 

  • Golub, G., Heath, M., & Wahba, G. (1979). Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 21(2), 215–223.

    Article  MathSciNet  MATH  Google Scholar 

  • Greenshtein, E., & Ritov, Y. (2004). Persistence in high-dimensional linear predictor selection and the virtue of overparametrization. Bernoulli, 10(6), 971–988.

    Article  MathSciNet  MATH  Google Scholar 

  • Grimmett, G. (1973). A theorem about random fields. Bulletin of the London Mathematical Society, 5(1), 81–84.

    Article  MathSciNet  MATH  Google Scholar 

  • Hastie, T., Tibshirani, R., & Wainwright, M. (2015), Statistical learning with sparsity: The lasso and generalizations. Chapman and Hall.

    Book  MATH  Google Scholar 

  • Hebiri, M., & Lederer, J. (2013). How correlations influence lasso prediction. IEEE Transactions on Information Theory, 59(3), 1846–1854.

    Article  MathSciNet  MATH  Google Scholar 

  • Hiriart-Urruty, J.-B., & LemarĂ©chal, C. (2004). Convex analysis and minimization algorithms. Springer.

    MATH  Google Scholar 

  • Hoerl, A., & Kennard, R. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55–67.

    Article  MATH  Google Scholar 

  • Homrighausen, D., & McDonald, D. (2013a). The lasso, persistence, and cross-validation. In Proceedings of machine learning research (Vol. 28, pp. 1031–1039).

    Google Scholar 

  • Homrighausen, D., & McDonald, D. (2013b). Risk-consistency of cross-validation with lasso-type procedures. Statistica Sinica, 27(3), 1017–1036.

    MathSciNet  MATH  Google Scholar 

  • Homrighausen, D., & McDonald, D. (2014). Leave-one-out cross-validation is risk consistent for lasso. Machine Learning, 97(1–2), 65–78.

    Article  MathSciNet  MATH  Google Scholar 

  • Huang, S.-T., DĂĽren, Y., Hellton, K., & Lederer, J. (2019). Tuning parameter calibration for prediction in personalized medicine. arXiv:1909.10635.

    Google Scholar 

  • Javanmard, A., & Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. Journal of Machine Learning Research, 15(1), 2869–2909.

    MathSciNet  MATH  Google Scholar 

  • Judson, R., Salisbury, B., Schneider, J., Windemuth, A., & Stephens, J. (2002). How many SNPs does a genome-wide haplotype map require? Pharmacogenomics, 3(3), 379–391.

    Article  Google Scholar 

  • Karush, W. (1939), Minima of functions of several variables with inequalities as side constraints, aster’s thesis, University of Chicago.

    Google Scholar 

  • Kidd, J. et al. (2008). Mapping and sequencing of structural variation from eight human genomes. Nature, 453(7191), 56–64.

    Article  Google Scholar 

  • Kim, Y., Choi, H., & Oh, H.-S. (2008). Smoothly clipped absolute deviation on high dimensions. Journal of the American Statistical Association, 103(484), 1665–1673.

    Article  MathSciNet  MATH  Google Scholar 

  • Knight, K., & Fu, W. (2000). Asymptotics for lasso-type estimators. Annals of Statistics, 28(5), 1356–1378.

    MathSciNet  MATH  Google Scholar 

  • Kuhn, H., & Tucker, A. (1951). Nonlinear programming. In Proceedings of Second Berkeley Symposium (pp. 481–492). University of California Press.

    Google Scholar 

  • Kurtz, Z., MĂĽller, C., Miraldi, E., Littman, D., Blaser, M., & Bonneau, R. (2015). Sparse and compositionally robust inference of microbial ecological networks. PLoS Computational Biology, 11(5), e1004226.

    Article  Google Scholar 

  • Laszkiewicz, M., Fischer, A., & Lederer, J. (2020). Thresholded adaptive validation: Tuning the graphical lasso for graph recovery. arXiv:2005.00466.

    Google Scholar 

  • Lauritzen, S. (1996). Graphical models. Oxford University Press.

    MATH  Google Scholar 

  • Lederer, J. (2013). Trust, but verify: Benefits and pitfalls of least-squares refitting in high dimensions. arXiv:1306.0113.

    Google Scholar 

  • Lederer, J., & MĂĽller, C. (2015). Don’t fall for tuning parameters: Tuning-free variable selection in high dimensions with the TREX. In AAAI Conference on Artificial Intelligence.

    Google Scholar 

  • Lederer, J., Yu, L., & Gaynanova, I. (2019). Oracle inequalities for high-dimensional prediction. Bernoulli, 25(2), 1225–1255.

    Article  MathSciNet  MATH  Google Scholar 

  • Lepski, O., Mammen, E., & Spokoiny, V. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Annals of Statistics, 25(3), 929–947.

    Article  MathSciNet  MATH  Google Scholar 

  • Lepskii, O. (1991). On a problem of adaptive estimation in Gaussian white noise. Theory of Probability and its Applications, 35(3), 454–466.

    Article  MathSciNet  Google Scholar 

  • Li, W., & Lederer, J. (2019). Tuning parameter calibration for â„“1-regularized logistic regression. Journal of Statistical Planning and Inference, 202, 80–98.

    Article  MathSciNet  MATH  Google Scholar 

  • Mazumder, R., & Hastie, T. (2012). The graphical lasso: New insights and alternatives. Electronic Journal of Statistics, 6, 2125–2149.

    Article  MathSciNet  MATH  Google Scholar 

  • Meinshausen, N. (2007). Relaxed lasso. Computational Statistics and Data Analysis, 52(1), 374–393.

    Article  MathSciNet  MATH  Google Scholar 

  • Meinshausen, N. (2013). Sign-constrained least squares estimation for high-dimensional regression. Electronic Journal of Statistics, 7, 1607–1631.

    Article  MathSciNet  MATH  Google Scholar 

  • Meinshausen, N., & BĂĽhlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Annals of Statistics, 34(1), 1436–1462.

    MathSciNet  MATH  Google Scholar 

  • Merriam-Webster.com (2019). Oracle. Retrieved November 11, 2019 from â–ş https://www.merriam-webster.com

  • Mills, R., Luttig, C., Larkins, C., Beauchamp, A., Tsui, C., Pittard, W., & Devine, S. (2006). An initial map of insertion and deletion (INDEL) variation in the human genome. Genome Research, 16(9), 1182–1190.

    Article  Google Scholar 

  • Negahban, S., Yu, B., Wainwright, M., & Ravikumar, P. (2012). A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. Statistical Science, 27(4), 538–557.

    Article  MathSciNet  MATH  Google Scholar 

  • Obozinski, G., Jacob, L., & Vert, J.-P. (2011). Group lasso with overlaps: The latent group lasso approach. arXiv:1110.0413.

    Google Scholar 

  • Osborne, M., Presnell, B., & Turlach, B. (2000). On the lasso and its dual. Journal of Computational and Graphical Statistics, 9(2), 319–337.

    MathSciNet  Google Scholar 

  • Oztoprak, F., Nocedal, J., Rennie, S., & Olsen, P. (2012), Newton-like methods for sparse inverse covariance estimation. In Advances in neural information processing systems (pp. 755–763).

    Google Scholar 

  • Park, T., & Casella, G. (2008). The Bayesian lasso. Journal of the American Statistical Association, 103(482), 681–686.

    Article  MathSciNet  MATH  Google Scholar 

  • Penrose, R. (1955). A generalized inverse for matrices. Mathematical Proceedings of the Cambridge Philosophical Society, 51(3), 406–413.

    Article  MATH  Google Scholar 

  • Perrone, V., Jenatton, R., Seeger, M., & Archambeau, C. (2018). Scalable hyperparameter transfer learning. In Advances in neural information processing systems (pp. 6845–6855).

    Google Scholar 

  • Preston, C. (1973). Generalized Gibbs states and Markov random fields. Advances in Applied Probability, 5(2), 242–261.

    Article  MathSciNet  MATH  Google Scholar 

  • Schneider, U., & Ewald, K. (2017). On the distribution, model selection properties and uniqueness of the lasso estimator in low and high dimensions. arXiv:1708.09608.

    Google Scholar 

  • Sherman, S. (1973). Markov random fields and Gibbs random fields. Israel Journal of Mathematics, 14(1), 92–103.

    Article  MathSciNet  MATH  Google Scholar 

  • Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2013). A sparse-group lasso. Journal of Computational and Graphical Statistics, 22(2), 231–245.

    Article  MathSciNet  Google Scholar 

  • Spirtes, P., Glymour, C., Scheines, R., Heckerman, D., Meek, C., Cooper, G. , & Richardson, T. (2000). Causation, prediction, and search. MIT Press.

    Google Scholar 

  • Städler, N., BĂĽhlmann, P., & van de Geer, S. (2010). â„“1-penalization for mixture regression models. Test, 19, 209–285.

    Article  MathSciNet  MATH  Google Scholar 

  • Stock, J., & Trebbi, F. (2003). Retrospectives: Who invented instrumental variable regression? Journal of Economic Perspectives, 17(3), 177–194.

    Article  Google Scholar 

  • Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 36(2), 111–133.

    MathSciNet  MATH  Google Scholar 

  • Sun, T., & Zhang, C.-H. (2010). Comments on: â„“1-penalization for mixture regression models. Test, 19, 270–275

    Article  MathSciNet  MATH  Google Scholar 

  • Sun, T., & Zhang, C.-H. (2012). Scaled sparse linear regression. Biometrika, 99(4), 879–898.

    Article  MathSciNet  MATH  Google Scholar 

  • Taheri, M., Lim, N., & Lederer, J. (2020). Efficient feature selection with large and high-dimensional data. arXiv:1609.07195.

    Google Scholar 

  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 58(1), 267–288.

    MathSciNet  MATH  Google Scholar 

  • Tibshirani, R. (2013). The lasso problem and uniqueness. Electronic Journal of Statistics, 7, 1456–1490.

    Article  MathSciNet  MATH  Google Scholar 

  • Tikhonov, A. (1943). On the stability of inverse problems. Doklady Akademii Nauk SSSR, 39(5), 195–198.

    MathSciNet  Google Scholar 

  • van de Geer, S. (2007), The deterministic lasso. In JSM Proceedings.

    Google Scholar 

  • van de Geer, S., & BĂĽhlmann, P. (2009). On the conditions used to prove oracle results for the lasso. Electronic Journal of Statistics, 3, 1360–1392.

    MathSciNet  MATH  Google Scholar 

  • van de Geer, S., & BĂĽhlmann, P. (2011). Statistics for high-dimensional data: Methods, theory and applications. Springer.

    MATH  Google Scholar 

  • van de Geer, S., BĂĽhlmann, P., Ritov, Y., & Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. Annals of Statistics, 42(3), 1166–1202.

    MathSciNet  MATH  Google Scholar 

  • van de Geer, S., & Lederer, J. (2013). The lasso, correlated design, and improved oracle inequalities. In From probability to statistics and back: High-dimensional models and processes–a festschrift in honor of Jon A. Wellner’, IMS (pp. 303–316).

    Google Scholar 

  • van der Vaart, A. (2000). Asymptotic statistics (Vol. 3). Cambridge University Press.

    Google Scholar 

  • Wainwright, M. (2009). Sharp thresholds for high-dimensional and noisy sparsity recovery using â„“1-constrained quadratic programming (lasso). IEEE Transactions on Information Theory, 55(5), 2183–2202.

    Article  MathSciNet  MATH  Google Scholar 

  • Wainwright, M. (2014). Structured regularizers for high-dimensional problems: Statistical and computational issues. Annual Review of Statistics and Its Application, 1, 233–253.

    Article  Google Scholar 

  • Yuan, M., & Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 68(1), 49–67.

    Article  MathSciNet  MATH  Google Scholar 

  • Yuan, M., & Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika, 94(1), 19–35.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(2), 894–942.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, C.-H., & Zhang, T. (2012). A general theory of concave regularization for high-dimensional sparse estimation problems. Statistical Science, 27(4), 576–593.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhao, P., & Yu, B. (2006). On model selection consistency of lasso. Journal of Machine Learning Research, 7, 2541–2563.

    MathSciNet  MATH  Google Scholar 

  • Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 67(2), 301–320.

    Article  MathSciNet  MATH  Google Scholar 

  • Zuber, J.-B., & Itzykson, C. (1977). Quantum field theory and the two-dimensional Ising model. Physical Review D, 15(10), 2875.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Statistics, Machine Learning & Data Science, Ruhr-University Bochum, Bochum, Germany

    Johannes Lederer

Authors
  1. Johannes Lederer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Johannes Lederer .

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Lederer, J. (2022). Graphical Models. In: Fundamentals of High-Dimensional Statistics. Springer Texts in Statistics. Springer, Cham. https://doi.org/10.1007/978-3-030-73792-4_3

Download citation

Publish with us

Policies and ethics

Search

Navigation