Skip to main content
SpringerLink
Log in

Learning Networks from Gaussian Graphical Models and Gaussian Free Fields

  • Published:
Journal of Statistical Physics Aims and scope Submit manuscript

Abstract

We investigate the problem of estimating the structure of a weighted network from repeated measurements of a Gaussian graphical model (GGM) on the network. In this vein, we consider GGMs whose covariance structures align with the geometry of the weighted network on which they are based. Such GGMs have been of longstanding interest in statistical physics, and are referred to as the Gaussian free field (GFF). In recent years, they have attracted considerable interest in the machine learning and theoretical computer science. In this work, we propose a novel estimator for the weighted network (equivalently, its Laplacian) from repeated measurements of a GFF on the network, based on the Fourier analytic properties of the Gaussian distribution. In this pursuit, our approach exploits complex-valued statistics constructed from observed data, that are of interest in their own right. We demonstrate the effectiveness of our estimator with concrete recovery guarantees and bounds on the required sample complexity. In particular, we show that the proposed statistic achieves the parametric rate of estimation for fixed network size. In the setting of networks growing with sample size, our results show that for Erdos–Renyi random graphs G(d, p) above the connectivity threshold, network recovery takes place with high probability as soon as the sample size n satisfies \(n \gg d^4 \log d \cdot p^{-2}\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

This manuscript has no associated data.

References

  1. Anandkumar, A., Tan, V., Willsky, A.: High-dimensional Gaussian graphical model selection: walk summability and local separation criterion. J. Mach. Learn. Res. 13, 07 (2011)

    MathSciNet  Google Scholar 

  2. Anandkumar, A., Tan, V.Y., Huang, F., Willsky, A.S.: High-dimensional structure estimation in Ising models: local separation criterion. Ann. Stat. 40, 1346–1375 (2012)

    Article  MathSciNet  Google Scholar 

  3. Banerjee, O., Ghaoui, L.: Model selection through sparse max likelihood estimation model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data. J. Mach. Learn. Res. 9, 08 (2007)

    Google Scholar 

  4. Banerjee, S., Ghosal, S.: Posterior convergence rates for estimating large precision matrices using graphical models. Electron. J. Stat. 8(2), 2111–2137 (2014)

    Article  MathSciNet  Google Scholar 

  5. Banerjee, S., Ghosal, S.: Bayesian structure learning in graphical models. J. Multivar. Anal. 136, 147–162 (2015)

    Article  MathSciNet  Google Scholar 

  6. Basso, K., Margolin, A.A., Stolovitzky, G., Klein, U., Dalla-Favera, R., Califano, A.: Reverse engineering of regulatory networks in human B cells. Nat. Genet. 37(4), 382–390 (2005). https://doi.org/10.1038/ng1532

  7. Belomestny, D., Trabs, M., Tsybakov, A.B.: Sparse covariance matrix estimation in high-dimensional deconvolution. Bernoulli 25(3), 8 (2019). https://doi.org/10.3150/18-BEJ1040A

  8. Berestycki, N.: Introduction to the Gaussian free field and Liouville quantum gravity. Lecture notes, 2018–2019 (2015)

  9. Berthet, Q., Rigollet, P., Srivastava, P.: Exact recovery in the ising blockmodel. Ann. Stat. 47(4), 1805–1834 (2019)

    Article  MathSciNet  Google Scholar 

  10. Bhattacharya, B. B., Mukherjee, S.: Inference in ising models. (2018)

  11. Bickel, P.J., Levina, E.: Regularized estimation of large covariance matrices. Ann. Stat. 36(1), 199–227 (2008)

    Article  MathSciNet  Google Scholar 

  12. Bickel, P.J., Levina, E.: Covariance regularization by thresholding. Ann. Stat. 36(6), 2577–2604 (2008)

    Article  MathSciNet  Google Scholar 

  13. Bresler, G.: Efficiently learning ising models on arbitrary graphs. In: Proceedings of the forty-seventh annual ACM symposium on Theory of computing, pp. 771–782 (2015)

  14. Cai, T.T., Zhang, C.-H., Zhou, H.H.: Optimal rates of convergence for covariance matrix estimation. Ann. Stat. 38(4), 2118–2144 (2010)

    Article  MathSciNet  Google Scholar 

  15. Cai, T.T., Li, H., Liu, W., Xie, J.: Covariate-adjusted precision matrix estimation with an application in genetical genomics. Biometrika 100(1), 139–156 (2012). (11)

    Article  MathSciNet  Google Scholar 

  16. Cai, T., Liu, W., Zhou, H.H.: Estimating sparse precision matrix: optimal rates of convergence and adaptive estimation. Ann. Stat. 44(2), 455–488 (2016)

    Article  MathSciNet  Google Scholar 

  17. Cai, T.T., Ren, Z., Zhou, H.H.: Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation. Electron. J. Stat. 10(1), 1–59 (2016)

    MathSciNet  Google Scholar 

  18. Cai, T., Liu, W., Luo, X.: A constrained \(\ell _1\) minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. 106(494), 594–607 (2011)

    Article  Google Scholar 

  19. Dahl, J., Vandenberghe, L., Roychowdhury, V.: Covariance selection for non-chordal graphs via chordal embedding. Optim. Methods Softw. 23(4), 501–520 (2008)

    Article  MathSciNet  Google Scholar 

  20. d’Aspremont, A., Banerjee, O., El Ghaoui, L.: First-order methods for sparse covariance selection. SIAM J. Matrix Anal. Appl. 30(1), 56–66 (2008)

    Article  MathSciNet  Google Scholar 

  21. Dempster, A.P.: Covariance selection. Biometrics 28(1), 157–175 (1972)

    Article  MathSciNet  Google Scholar 

  22. El Karoui, N.: Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Stat. 36(6), 2717–2756 (2008)

    MathSciNet  Google Scholar 

  23. Fan, J., Feng, Y., Yichao, W.: Network exploration via the adaptive LASSO and SCAD penalties. Ann. Appl. Stat. 3(2), 521–541 (2009)

    Article  MathSciNet  Google Scholar 

  24. Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2007). (12)

    Article  Google Scholar 

  25. Ghosh, S., Mukherjee, S. S.: Learning with latent group sparsity via heat flow dynamics on networks. arXiv:2201.08326 (2022)

  26. Huang, J.Z., Liu, N., Pourahmadi, M., Liu, L.: Covariance matrix selection and estimation via penalised normal likelihood. Biometrika 93(1), 85–98 (2006)

    Article  MathSciNet  Google Scholar 

  27. Huang, S., Li, J., Sun, L., Ye, J., Fleisher, A., Teresa, W., Chen, K., Reiman, E.: Learning brain connectivity of Alzheimer’s disease by sparse inverse covariance estimation. NeuroImage 50(3), 935–949 (2010). https://doi.org/10.1016/j.neuroimage.2009.12.120

    Article  Google Scholar 

  28. Kelner, J., Koehler, F., Meka, R., Moitra, A.: Learning some popular gaussian graphical models without condition number bounds. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 10986–10998. Curran Associates Inc, New York (2020)

    Google Scholar 

  29. Kelner, J., Koehler, F., Meka, R., Moitra, A.: Learning some popular gaussian graphical models without condition number bounds. Adv. Neural. Inf. Process. Syst. 33, 10986–10998 (2020)

    Google Scholar 

  30. Lam, C., Fan, J.: Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Stat. 37(6B), 4254–4278 (2009)

    Article  MathSciNet  Google Scholar 

  31. Lei, J., Rinaldo, A.: Consistency of spectral clustering in stochastic block models. Ann. Stat. 43(1), 215–237 (2015). https://doi.org/10.1214/14-AOS1274

    Article  MathSciNet  Google Scholar 

  32. Liu, H., Lafferty, J., Wasserman, L.: The nonparanormal: semiparametric estimation of high dimensional undirected graphs. J. Mach. Learn. Res. 10(80), 2295–2328 (2009)

    MathSciNet  Google Scholar 

  33. Loh, P.-L., Bühlmann, P.: High-dimensional learning of linear causal networks via inverse covariance estimation. J. Mach. Learn. Res. 15(1), 3065–3105 (2014)

    MathSciNet  Google Scholar 

  34. Ma, Y., Garnett, R., Schneider, J.: \(\sigma \)-optimality for active learning on gaussian random fields. Advances in Neural Information Processing Systems, 26 (2013)

  35. Malioutov, D.M., Johnson, J.K., Willsky, A.S.: Walk-sums and belief propagation in gaussian graphical models. J. Mach. Learn. Res. 7(73), 2031–2064 (2006)

    MathSciNet  Google Scholar 

  36. Meinshausen, N., Bühlmann, P.: High-dimensional graphs and variable selection with the Lasso. Ann. Stat. 34(3), 1436–1462 (2006)

    Article  MathSciNet  Google Scholar 

  37. Menéndez, P., Kourmpetis, Y.A., ter Braak, C.J., van Eeuwijk, F.A.: Gene regulatory networks from multifactorial perturbations using graphical lasso: application to the DREAM4 challenge. PLoS ONE 5(12), e14147 (2010). https://doi.org/10.1371/journal.pone.0014147

    Article  ADS  Google Scholar 

  38. Misra, S., Vuffray, M., Lokhov, A.Y.: Information theoretic optimal learning of gaussian graphical models. In: Jacob Abernethy and Shivani Agarwal, editors, Proceedings of Thirty Third Conference on Learning Theory, volume 125 of Proceedings of Machine Learning Research, pp. 2888–2909. PMLR, 09–12 (2020)

  39. Müller, A., Scarsini, M.: Archimedean copulae and positive dependence. J. Multivar. Anal. 93(2), 434–445 (2005). https://doi.org/10.1016/j.jmva.2004.04.003

    Article  MathSciNet  Google Scholar 

  40. Ravikumar, P., Wainwright, M.J., Lafferty, J.D.: High-dimensional ising model selection using \(\ell _1\)-regularized logistic regression (2010)

  41. Ravikumar, P., Wainwright, M.J., Raskutti, G., Bin, Yu.: High-dimensional covariance estimation by minimizing \(\ell _1\)-penalized log-determinant divergence. Electron. J. Stat. 5(none), 935–980 (2011)

    Article  MathSciNet  Google Scholar 

  42. Rish, I., Thyreau, B., Thirion, B., Plaze, M., Paillere-martinot, M., Martelli, C., Martinot, J., Poline, J., Cecchi, G.: Discriminative network models of schizophrenia. In: Bengio, Y., Schuurmans, D., Lafferty, J., Williams, C., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 22. Curran Associates Inc, New York (2009)

    Google Scholar 

  43. Rothman, A.J., Bickel, P.J., Levina, E., Zhu, J.: Sparse permutation invariant covariance estimation. Electron. J. Stat. 2, 494–515 (2008)

    Article  MathSciNet  Google Scholar 

  44. Schafer, J., Strimmer, K.: Learning large-scale graphical Gaussian models from genomic data. In: AIP Conference Proceedings, pp. 263–276. AIP (2005). https://doi.org/10.1063/1.1985393

  45. Sheffield, S.: Gaussian free fields for mathematicians. Probab. Theory Relat. Fields 139(3–4), 521–541 (2007)

    Article  MathSciNet  Google Scholar 

  46. Shi, W., Ghosal, S., Martin, R.: Bayesian estimation of sparse precision matrices in the presence of Gaussian measurement error. Electron. J. Stat. 15(2), 4545–4579 (2021)

    Article  MathSciNet  Google Scholar 

  47. Tropp, J.A.: Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theory 52(3), 1030–1051 (2006). https://doi.org/10.1109/TIT.2005.864420

    Article  MathSciNet  Google Scholar 

  48. Varoquaux, G., Baronnet, F., Kleinschmidt, A., Fillard, P., Thirion, B.: Detection of brain functional-connectivity difference in post-stroke patients using group-level covariance modeling. In: T. Jiang, N. Navab, J.P.W. Pluim, M.A. Viergever, (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2010, pp. 200–208, Springer, Berlin (2010)

  49. Varoquaux, G., Gramfort, A., Poline, J., Thirion, B.: Brain covariance selection: better individual functional connectivity models using population prior. In: Lafferty, J., Williams, C., Shawe-Taylor, J., Zemel, R., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 23. Curran Associates Inc, New York (2010)

    Google Scholar 

  50. Vershynin, R.: High-Dimensional Probability: An Introduction with Applications in Data Science, vol. 47. Cambridge University Press, Cambridge (2018)

    Google Scholar 

  51. Wainwright, M.J.: Sharp thresholds for high-dimensional and noisy sparsity recovery using \(\ell _{1}\)-constrained quadratic programming (lasso). IEEE Trans. Inf. Theory 55(5), 2183–2202 (2009). https://doi.org/10.1109/TIT.2009.2016018

    Article  MathSciNet  Google Scholar 

  52. Wille, A., Zimmermann, P., Vranová, E., Fürholz, A., Laule, O., Bleuler, S., Hennig, L., Prelić, A., von Rohr, P., Thiele, L., Zitzler, E., Gruissem, W., Bühlmann, P.: Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana. Genome Biol. 5(11), R92 (2004). https://doi.org/10.1186/gb-2004-5-11-r92

    Article  Google Scholar 

  53. Woodbury, M.A.: Inverting Modified Matrices. Princeton University, Department of Statistics, Princeton (1950)

    Google Scholar 

  54. Wu, W.B., Pourahmadi, M.: Non-parametric estimation of large covariance matrices of longitudinal data. Biometrika 90(4), 831–844 (2003)

    Article  MathSciNet  Google Scholar 

  55. Yuan, M.: High dimensional inverse covariance matrix estimation via linear programming. J. Mach. Learn. Res. 11(79), 2261–2286 (2010)

    MathSciNet  Google Scholar 

  56. Yuan, M., Lin, Y.: Model selection and estimation in the Gaussian graphical model. Biometrika 94, 19–35 (2007)

    Article  MathSciNet  Google Scholar 

  57. Zhao, P., Bin, Yu.: On model selection consistency of lasso. J. Mach. Learn. Res. 7(90), 2541–2563 (2006)

    MathSciNet  Google Scholar 

  58. Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th International conference on Machine learning (ICML-03), pp. 912–919 (2003)

  59. Zhu, X., Lafferty, J., Ghahramani, Z.: Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions. In: ICML 2003 workshop on the continuum from labeled to unlabeled data in machine learning and data mining, vol. 3 (2003)

  60. Zwiernik, P.: Semialgebraic Statistics and Latent Tree Models. Monographs on Statistics and Applied Probability, vol. 146. Chapman & Hall/CRC, Boca Raton (2016)

    Google Scholar 

Download references

Acknowledgements

S.G. was supported in part by the MOE Grants R-146-000-250-133, R-146-000-312-114 and MOE-T2EP20121-0013. S.S.M. was partially supported by an INSPIRE research Grant (DST/INSPIRE/04/2018/002193) from the Department of Science and Technology, Government of India and a Start-Up Grant from Indian Statistical Institute, Kolkata. H.S.T. was supported by the NUS Research Scholarship. We thank Satya Majumdar for helpful suggestions.

Author information

Author notes

  1. Subhro Ghosh, Soumendu Sundar Mukherjee and Hoang-Son Tran have contributted equally to this work.

Authors and Affiliations

  1. Department of Mathematics, National University of Singapore, Singapore, Singapore

    Subhro Ghosh, Hoang-Son Tran & Ujan Gangopadhyay

  2. Theoretical Statistics and Mathematics Unit, Indian Statistical Institute, Calcutta, India

    Soumendu Sundar Mukherjee

Authors
  1. Subhro Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soumendu Sundar Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hoang-Son Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ujan Gangopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hoang-Son Tran.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare.

Additional information

Communicated by Federico Ricci-Tersenghi.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghosh, S., Mukherjee, S.S., Tran, HS. et al. Learning Networks from Gaussian Graphical Models and Gaussian Free Fields. J Stat Phys 191, 45 (2024). https://doi.org/10.1007/s10955-024-03257-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10955-024-03257-0

Keywords

Mathematics Subject Classification

Search

Navigation