Skip to main content
SpringerLink
Log in

Two Gaussian Regularization Methods for Time-Varying Networks

  • Published:
Journal of Agricultural, Biological and Environmental Statistics Aims and scope Submit manuscript

Abstract

We model time-varying network data as realizations from multivariate Gaussian distributions with precision matrices that change over time. To facilitate parameter estimation, we require not only that each precision matrix at any given time point be sparse, but also that precision matrices at neighboring time points be similar. We accomplish this with two different algorithms, by generalizing the elastic net and the fused LASSO, respectively. While similar approaches in the literature for modeling time-varying networks have predominantly extended the graphical LASSO of Friedman et al. (Biostatistics 9(3):432-441, 2008), we extend the regression approach of Meinshausen and Bühlmann (Ann Stat 34(3):1436–1462, 2006) and subsequently of Peng et al. (J Am Stat Assoc 104(486):735–746, 2009). This allows us to explicitly focus on and work with the partial correlation coefficients, which are more directly meaningful and interpretable parameters for the biological sciences. We develop efficient algorithms and convenient degree-of-freedom formulae for choosing tuning parameters. The proposed methods are demonstrated through simulation studies. By applying them to an hourly temperature dataset, we detect interesting time-varying connectivity among thirteen Canadian cities. Supplementary materials accompanying this paper appear online.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Code Availability

The relevant codes can be found  https://github.com/JieJian/TimeVaryingGGM/tree/main/code.

References

  • Amr A, Xing EP (2009) Recovering time-varying networks of dependencies in social and biological studies. Proc Natl Acad Sci 106(29):11878–11883

    Article  Google Scholar 

  • Bartlett TE, Kosmidis I, Silva R (2021) Two-way sparsity for time-varying networks with applications in genomics. Ann Appl Stat 15(2):856–879

    Article  MathSciNet  Google Scholar 

  • Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3(1):1–12201

    Article  Google Scholar 

  • Chen L-P, Yi GY (2021) Analysis of noisy survival data with graphical proportional hazards measurement error models. Biometrics 77(3):956–969

    Article  MathSciNet  Google Scholar 

  • Chen S, Witten DM, Shojaie A (2015) Selection and estimation for mixed graphical models. Biometrika 102(1):47–64

    Article  MathSciNet  Google Scholar 

  • Danaher P, Wang P, Witten DM (2014) The joint graphical lasso for inverse covariance estimation across multiple classes. J R Stat Soc: Ser B (Stat Method) 76(2):373–397

    Article  MathSciNet  Google Scholar 

  • Dohlman AB, Shen X (2019) Mapping the microbial interactome: Statistical and experimental approaches for microbiome network inference. Exp Biol Med 244(6):445–458

    Article  Google Scholar 

  • Drton M, Maathuis MH (2017) Structure learning in graphical modeling. Ann Rev Stat Appl 4:365–393

    Article  Google Scholar 

  • Emmert-Streib F, Glazko GV, De Matos Simoes R (2012) Statistical inference and reverse engineering of gene regulatory networks from observational expression data. Front Genet 3:8–8

    Article  Google Scholar 

  • Environment and Climate Change Canada. Environment and Climate Change Canada Historical Climate Data website. https://climate.weather.gc.ca/index_e.html, (2022). Accessed 16 Oct 2022

  • Ferreira LN, Ferreira NCR, Macau EEN, Donner RV (2021) The effect of time series distance functions on functional climate networks. Eur Phys J Spec Top 230(14):2973–2998

    Article  Google Scholar 

  • Foygel R, Drton M (2010) Extended bayesian information criteria for gaussian graphical models. Adv Neural Inf Process Syst 23

  • Friedman J, Hastie T, Tibshirani R (2008) Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3):432–441

    Article  Google Scholar 

  • Gibberd AJ, Nelson JDB (2017) Regularized estimation of piecewise constant gaussian graphical models: the group-fused graphical lasso. J Comput Graph Stat 26(3):623–634

    Article  MathSciNet  Google Scholar 

  • Hallac D, Park Y, Boyd S, Leskovec J (2017) Network inference via the time-varying graphical lasso. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp 205–213

  • Han I, Malioutov D, Avron H, Shin J (2017) Approximating spectral sums of large-scale matrices using stochastic Chebyshev approximations. SIAM J Sci Comput 39(4):A1558–A1585

    Article  MathSciNet  Google Scholar 

  • Hoefling H (2010) A path algorithm for the fused lasso signal approximator. J Comput Graph Stat 19(4):984–1006

    Article  MathSciNet  Google Scholar 

  • Kang X, Deng X, Tsui K-W, Pourahmadi M (2020) On variable ordination of modified Cholesky decomposition for estimating time-varying covariance matrices. Int Stat Rev 88(3):616–641

    Article  MathSciNet  Google Scholar 

  • Kittel T, Ciemer C, Lotfi N, Peron T, Rodrigues F, Kurths J, Donner RV (2021) Evolving climate network perspectives on global surface air temperature effects of enso and strong volcanic eruptions. Eur Phys J Spec Top 230(14):3075–3100

    Article  Google Scholar 

  • Kolar M, Xing EP (2011) On time varying undirected graphs. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, Fort Lauderdale, vol. 15, pp 407–415

  • Kolar M, Song L, Ahmed A, Xing EP (2010) Estimating time-varying networks. Ann Appl Stat 4(1):94–123

    Article  MathSciNet  Google Scholar 

  • Lauritzen SL (1996) Graphical models. The Clarendon Press, Oxford

    Book  Google Scholar 

  • Loh P-L (2017) Statistical consistency and asymptotic normality for high-dimensional robust \(M\)-estimators. Ann Stat 45(2):866–896. https://doi.org/10.1214/16-AOS1471

    Article  MathSciNet  Google Scholar 

  • Meinshausen N (2008) A note on the lasso for gaussian graphical model selection. Stat Probab Lett 78(7):880–884

    Article  MathSciNet  Google Scholar 

  • Meinshausen N, Bühlmann P (2006) High-dimensional graphs and variable selection with the lasso. Ann Stat 34(3):1436–1462

    Article  MathSciNet  Google Scholar 

  • Monti RP, Hellyer P, Sharp D, Leech R, Anagnostopoulos C, Montana G (2014) Estimating time-varying brain connectivity networks from functional MRI time series. Neuroimage 103:427–443

    Article  Google Scholar 

  • Peng J, Wang P, Zhou N, Zhu J (2009) Partial correlation estimation by joint sparse regression models. J Am Stat Assoc 104(486):735–746

    Article  MathSciNet  Google Scholar 

  • Sato T, Yamanishi Y, Horimoto K, Kanehisa M, Toh H (2006) Partial correlation coefficient between distance matrices as a new indicator of protein–protein interactions. Bioinformatics 22(20):2488–2492

    Article  Google Scholar 

  • Tibshirani R (1996) Regres Shrinkage Select via lasso. Journal of the Royal Statistical Society. Ser B. 58(1):267–288

    MathSciNet  Google Scholar 

  • Tibshirani R, Saunders M, Rosset S, Zhu J, Knight K (2005) Sparsity and smoothness via the fused lasso. J R Stat Soc. Ser B 67(1):91–108

    Article  MathSciNet  Google Scholar 

  • Tibshirani RJ, Taylor J (2011) The solution path of the generalized lasso. Ann Stat 39(3):1335–1371

    Article  MathSciNet  Google Scholar 

  • Tibshirani RJ, Taylor J (2012) Degrees of freedom in lasso problems. Ann Stat 40(2):1198–1232

    Article  MathSciNet  Google Scholar 

  • Voorman A, Shojaie A, Witten D (2014) Graph estimation with joint additive models. Biometrika 101(1):85–101

    Article  MathSciNet  Google Scholar 

  • Wang S, Xie C, Kang X (2023) A novel robust estimation for high-dimensional precision matrices. Stat Med 42(5):656–675

    Article  MathSciNet  Google Scholar 

  • Whittaker J (1990) Graphical models in applied multivariate statistics. Wiley, Chichester

    Google Scholar 

  • Xin B, Kawahara Y, Wang Y, Lingjing H, Gao W (2016) Efficient generalized fused lasso and its applications. ACM Trans Intell Syst Technol (TIST) 7(4):1–22

    Article  Google Scholar 

  • Xue L, Shu X, Annie Q (2020) Time-varying estimation and dynamic model selection with an application of network data. Stat Sin 30(1):251–284

    MathSciNet  Google Scholar 

  • Yang E, Ravikumar P, Allen GI, Liu Z (2015) Graphical models via univariate exponential family distributions. J Mach Learn Res 16(1):3813–3847

    MathSciNet  Google Scholar 

  • Yang J, Peng J (2020) Estimating time-varying graphical models. J Comput Graph Stat 29(1):191–202

    Article  MathSciNet  Google Scholar 

  • Ye GB, Xie X (2011) Split Bregman method for large scale fused Lasso. Comput Stat Data Anal 55(4):1552–1569

    Article  MathSciNet  Google Scholar 

  • Yuan M, Lin Y (2007) Model selection and estimation in the Gaussian graphical model. Biometrika 94(1):19–35

    Article  MathSciNet  Google Scholar 

  • Zhou S, Lafferty J, Wasserman L (2010) Time varying undirected graphs. Mach Learn 80(2–3):295–319

    Article  MathSciNet  Google Scholar 

  • Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J Roy Stat Soc B 67(2):301–320

    Article  MathSciNet  Google Scholar 

  • Zou H, Hastie T, Tibshirani R (2007) On the “degrees of freedom’’ of the lasso. Ann Stat 35(5):2173–2192

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Canada

    Jie Jian, Peijun Sang & Mu Zhu

Authors
  1. Jie Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peijun Sang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peijun Sang.

Ethics declarations

Conflict of interest

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 4039 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jian, J., Sang, P. & Zhu, M. Two Gaussian Regularization Methods for Time-Varying Networks. JABES (2024). https://doi.org/10.1007/s13253-023-00591-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13253-023-00591-w

Keywords

Search

Navigation