GMRF Estimation under Topological and Spectral Constraints

  • Victorin Martin
  • Cyril Furtlehner
  • Yufei Han
  • Jean-Marc Lasgouttes
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8725)


We investigate the problem of Gaussian Markov random field selection under a non-analytic constraint: the estimated models must be compatible with a fast inference algorithm, namely the Gaussian belief propagation algorithm. To address this question, we introduce the ⋆-IPS framework, based on iterative proportional scaling, which incrementally selects candidate links in a greedy manner. Besides its intrinsic sparsity-inducing ability, this algorithm is flexible enough to incorporate various spectral constraints, like e.g. walk summability, and topological constraints, like short loops avoidance. Experimental tests on various datasets, including traffic data from San Francisco Bay Area, indicate that this approach can deliver, with reasonable computational cost, a broad range of efficient inference models, which are not accessible through penalization with traditional sparsity-inducing norms.


Iterative proportional scaling Gaussian belief propagation walk-summability Gaussian Markov Random Field 


  1. 1.
    Banerjee, O., El Ghaoui, L., d’Aspremont, A.: Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. JMLR 9, 485–516 (2008)zbMATHGoogle Scholar
  2. 2.
    Bickson, D.: Gaussian Belief Propagation: Theory and Application. Ph.D. thesis, Hebrew University of Jerusalem (2008)Google Scholar
  3. 3.
    Chow, C., Liu, C.: Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory (1968)Google Scholar
  4. 4.
    Darroch, J., Ratcliff, D.: Generalized iterative scaling for log-linear models. Ann. Math. Statistics 43, 1470–1480 (1972)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Dong, B., Zhang, Y.: An efficient algorithm for l 0 minimization in wavelet frame based image restoration. Journal of Scientific Computing 54(2-3) (2012)Google Scholar
  6. 6.
    Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of American Statistical Association (2001)Google Scholar
  7. 7.
    Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)CrossRefzbMATHGoogle Scholar
  8. 8.
    Furtlehner, C., Han, Y., Lasgouttes, J.M., Martin, V., Marchal, F., Moutarde, F.: Spatial and temporal analysis of traffic states on large scale networks. In: ITSC, pp. 1215–1220 (2010)Google Scholar
  9. 9.
    Hsieh, C., Sustik, M.A., Dhillon, I.S., Ravikumar, K.: Sparse inverse covariance matrix estimation using quadratic approximation. In: NIPS (2011)Google Scholar
  10. 10.
    Jalali, A., Johnson, C.C., Ravikumar, P.D.: On learning discrete graphical models using greedy methods. In: NIPS, pp. 1935–1943 (2011)Google Scholar
  11. 11.
    Lee Dicker, B.H., Lin, X.: Variable selection and estimation with the seamless-L 0 penalty. Statistica Sinica 23(2), 929–962 (2012)Google Scholar
  12. 12.
    Malioutov, D., Johnson, J., Willsky, A.: Walk-sums and Belief Propagation in Gaussian graphical models. JMLR 7, 2031–2064 (2006)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Malouf, R.: A comparison of algorithms for maximum entropy parameter estimation. In: COLING, pp. 49–55 (2002)Google Scholar
  14. 14.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Network of Plausible Inference. Morgan Kaufmann (1988)Google Scholar
  15. 15.
    Scheinberg, K., Rish, I.: Learning sparse gaussian markov networks using a greedy coordinate ascent approach. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010, Part III. LNCS, vol. 6323, pp. 196–212. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Seneta, E.: Non-negative matrices and Markov chains. Springer (2006)Google Scholar
  17. 17.
    Speed, T., Kiiveri, H.: Gaussian Markov distributions over finite graphs. The Annals of Statistics 14(1), 138–150 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Weiss, Y., Freeman, W.T.: Correctness of belief propagation in Gaussian graphical models of arbitrary topology. Neural Computation (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Victorin Martin
    • 1
  • Cyril Furtlehner
    • 2
  • Yufei Han
    • 2
    • 3
  • Jean-Marc Lasgouttes
    • 3
  1. 1.Mines ParisTechCentre for RoboticsFrance
  2. 2.Inria Saclay–Île-de-France, TAO teamFrance
  3. 3.Inria Paris–Rocquencourt, RITS teamFrance

Personalised recommendations