GMRF Estimation under Topological and Spectral Constraints
We investigate the problem of Gaussian Markov random field selection under a non-analytic constraint: the estimated models must be compatible with a fast inference algorithm, namely the Gaussian belief propagation algorithm. To address this question, we introduce the ⋆-IPS framework, based on iterative proportional scaling, which incrementally selects candidate links in a greedy manner. Besides its intrinsic sparsity-inducing ability, this algorithm is flexible enough to incorporate various spectral constraints, like e.g. walk summability, and topological constraints, like short loops avoidance. Experimental tests on various datasets, including traffic data from San Francisco Bay Area, indicate that this approach can deliver, with reasonable computational cost, a broad range of efficient inference models, which are not accessible through penalization with traditional sparsity-inducing norms.
KeywordsIterative proportional scaling Gaussian belief propagation walk-summability Gaussian Markov Random Field
Unable to display preview. Download preview PDF.
- 2.Bickson, D.: Gaussian Belief Propagation: Theory and Application. Ph.D. thesis, Hebrew University of Jerusalem (2008)Google Scholar
- 3.Chow, C., Liu, C.: Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory (1968)Google Scholar
- 5.Dong, B., Zhang, Y.: An efficient algorithm for l 0 minimization in wavelet frame based image restoration. Journal of Scientific Computing 54(2-3) (2012)Google Scholar
- 6.Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of American Statistical Association (2001)Google Scholar
- 8.Furtlehner, C., Han, Y., Lasgouttes, J.M., Martin, V., Marchal, F., Moutarde, F.: Spatial and temporal analysis of traffic states on large scale networks. In: ITSC, pp. 1215–1220 (2010)Google Scholar
- 9.Hsieh, C., Sustik, M.A., Dhillon, I.S., Ravikumar, K.: Sparse inverse covariance matrix estimation using quadratic approximation. In: NIPS (2011)Google Scholar
- 10.Jalali, A., Johnson, C.C., Ravikumar, P.D.: On learning discrete graphical models using greedy methods. In: NIPS, pp. 1935–1943 (2011)Google Scholar
- 11.Lee Dicker, B.H., Lin, X.: Variable selection and estimation with the seamless-L 0 penalty. Statistica Sinica 23(2), 929–962 (2012)Google Scholar
- 13.Malouf, R.: A comparison of algorithms for maximum entropy parameter estimation. In: COLING, pp. 49–55 (2002)Google Scholar
- 14.Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Network of Plausible Inference. Morgan Kaufmann (1988)Google Scholar
- 16.Seneta, E.: Non-negative matrices and Markov chains. Springer (2006)Google Scholar
- 18.Weiss, Y., Freeman, W.T.: Correctness of belief propagation in Gaussian graphical models of arbitrary topology. Neural Computation (2001)Google Scholar