Skip to main content
SpringerLink
Log in

Latent binary MRF for online reconstruction of large scale systems

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

We present a novel method for online inference of real-valued quantities on a large network from very sparse measurements. The target application is a large scale system, like e.g. a traffic network, where a small varying subset of the variables is observed, and predictions about the other variables have to be continuously updated. A key feature of our approach is the modeling of dependencies between the original variables through a latent binary Markov random field. This greatly simplifies both the model selection and its subsequent use. We introduce the mirror belief propagation algorithm, that performs fast inference in such a setting. The offline model estimation relies only on pairwise historical data and its complexity is linear w.r.t. the dataset size. Our method makes no assumptions about the joint and marginal distributions of the variables but is primarily designed with multimodal joint distributions in mind. Numerical experiments demonstrate both the applicability and scalability of the method in practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Baxter, R.: Exactly solved models in statistical mechanics. Dover Publications (2008)

  2. Beyer, K., Goldstein, J., Ramakrishnan, R., Shaft, U.: When is “nearest neighbor” meaningful? In Database Theory-ICDT’99, pp 217–235. Springer (1999)

  3. Bickson, D.: Gaussian Belief Propagation: Theory and Application. PhD thesis, Hebrew University of Jerusalem (2008)

  4. Bilmes, J.: On soft evidence in Bayesian networks. Technical report, University of Washington (2004)

  5. Boyen, X.: Inference and Learning in Complex Stochastic Processes. PhD thesis, Stanford University, Computer Science Department, 229 (2002)

  6. Chan, H., Darwiche, A.: On the revision of probabilistic beliefs using uncertain evidence. Artif. Intell. 163(1), 67–90 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  7. Cocco, S., Monasson, R.: Adaptive cluster expansion for the inverse Ising problem: convergence, algorithm and tests. J. Stat. Phys. 147(2), 252–314 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13(1), 21–27 (1967)

    Article  MATH  Google Scholar 

  9. Darroch, J., Ratcliff, D.: Generalized iterative scaling for log-linear models. The Annals Math. Stat. 43(5), 1470–1480 (1972)

    Article  MathSciNet  MATH  Google Scholar 

  10. Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc. Ser. B (Methodological), 1–38 (1977)

  11. Doucet, A., de Freitas, N., Gordon, N.: An introduction to sequential monte carlo methods. Springer-Verlag, New York (2001)

    Book  MATH  Google Scholar 

  12. Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)

    Article  MATH  Google Scholar 

  13. Furtlehner, C.: Approximate inverse Ising models close to a Bethe reference point. J. Stat. Mech.: Theory Exp. 2013(09), P09020 (2013)

    Article  MathSciNet  Google Scholar 

  14. Furtlehner, C., Han, Y., Lasgouttes, J.-M., Martin, V., Marchal, F., Moutarde, F.: Spatial and temporal analysis of traffic states on large scale networks. In: Procceding of the 13th International IEEE Conference on Intelligent Transportation Systems, 1215–1220 (2010)

  15. Furtlehner, C., Lasgouttes, J.-M., Auger, A.: Learning multiple belief propagation fixed points for real time inference. Phys. A: Stat. Mech. Appl. 389(1), 149–163 (2010)

    Article  MathSciNet  Google Scholar 

  16. Han, T.X., Ning, H., Huang, T.S.: Efficient nonparametric belief propagation with application to articulated body tracking. In: Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 1, 214–221 (2006)

  17. Herrera, J., Work, D., Herring, R., Ban, X., Jacobson, Q., Bayen, A.: Evaluation of traffic data obtained via GPS-enabled mobile phones: The mobile century field experiment. Transp. Res. Part C: Emerg. Technol. 18(4), 568–583 (2010)

    Article  Google Scholar 

  18. Ihler, A., Fischer, J.I., Willsky, A.: Loopy belief propagation: convergence and effects of message errors. J. Mach. Learn. Res. 6, 905–936 (2005)

    MathSciNet  MATH  Google Scholar 

  19. Jalali, A., Johnson, C., Ravikumar, P.: On learning discrete graphical models using greedy methods. arXiv:1107.3258 (2011)

  20. Jaynes, E.T.: Prior probabilities. IEEE Trans. Syst. Sci. Cybern. 4(3), 227–241 (1968)

    Article  MATH  Google Scholar 

  21. Jaynes, E.T.: Probability Theory: The Logic of Science (Vol 1). Cambridge University Press, 2003. ISBN 0521592712

  22. Kschischang, F.R., Frey, B.J., Loeliger, H.: Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory 47(2), 498–519 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  23. Mackay, D.J., Yedidia, J.S., Freeman, W.T., Weiss, Y., et al.: A conversation about the Bethe free energy and sum-product. Available at, http://www.merl.com/publications/TR2001-018/ (2001)

  24. Martin, V.: Modélisation Probabiliste et inférence par l’algorithme belief propagation. PhD thesis, Mines-ParisTech (2013)

  25. Martin, V., Furtlehner, C., Han, Y., Lasgouttes, J.-M.: GMRF Estimation under Spectral and Topological constraints. In: Machine Learning and Knowledge Discovery in Databases, volume 8725 of Lecture Notes in Computer Science, pages 370–385. Springer Berlin Heidelberg (2014)

  26. Mézard, M., Mora, T.: Constraint satisfaction problems and neural networks: A statistical physics perspective. J. Physiology-Paris 103(1-2), 107–113 (2009)

    Article  Google Scholar 

  27. Mézard, M., Parisi, G., Virasoro, M.: Spin glass theory and beyond. World scientific, Singapore (1987)

  28. Min, W., Wynter, L.: Real-time road traffic prediction with spatio-temporal correlations. Transp. Res. Part C 19, 606–616 (2011)

    Article  Google Scholar 

  29. Mooij, J.M., Kappen, H.J.: Sufficient conditions for convergence of the sum-product algorithm. IEEE Trans. Inf. Theory 53(12), 4422–4437 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  30. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Network of Plausible Inference. Morgan Kaufmann (1988)

  31. PUMAS project: http://team.inria.fr/pumas/ (in French)

  32. Ravikumar, P., Wainwright, M.J., Lafferty, J.D.: High-dimensional Ising model selection using L 1 regularized logistic regression. Ann. Stat. 38(3), 1287–1319 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  33. Smith, B.L., Williams, B.M., Keith Oswald, R.: Comparison of parametric and nonparametric models for traffic flow forecasting. Trans. Res, Part C: Emerg. Technol. 10(4), 303–321 (2002)

    Article  Google Scholar 

  34. Sudderth, E., Ihler, A., Isard, M., Freeman, W., Willsky, A.: Nonparametric Belief Propagation. Commun. ACM 53(10), 95–103 (Oct. 2010)

    Article  Google Scholar 

  35. Tatikonda, S., Jordan, M.: Loopy Belief Propagation and Gibbs measures. In: Proceedings of the 18th Conference in Uncertainty in Artificial Intelligence, 493–50 (2002)

  36. Teh, Y.W., Welling, M.: Passing and bouncing messages for generalized inference. Technical report, UCL (2001)

  37. Wainwright, M.J.: Estimating the “wrong” graphical model: benefits in the computation-limited setting. J. Mach. Learn. Res. 7, 1829–1859 (2006)

    MathSciNet  MATH  Google Scholar 

  38. Welling, M., Teh, Y.W.: Approximate inference in Boltzmann machines. Artif. Intell. 143(1), 19–50 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  39. Yasuda, M., Tanaka, K.: Approximate learning algorithm in Boltzmann machines. Neural Comput. 21(11), 3130–3178 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  40. Yedidia, J.S., Freeman, W.T., Weiss, Y.: Constructing free-energy approximations and generalized Belief Propagation algorithms. IEEE Trans. Inf. Theory 51(7), 2282–2312 (2005)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Robotics, Mines ParisTech, PSL Research University, 60 Boulevard Saint-Michel, 75006, Paris, France

    Victorin Martin

  2. Inria Paris–Rocquencourt, Domaine de Voluceau, BP. 105, 78153, Le Chesnay CEDEX, France

    Jean-Marc Lasgouttes

  3. Inria Saclay–Île-de-France LRI Bât 660 Université Paris-Sud, 91405, Orsay CEDEX, France

    Cyril Furtlehner

Authors
  1. Victorin Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jean-Marc Lasgouttes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cyril Furtlehner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Victorin Martin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martin, V., Lasgouttes, JM. & Furtlehner, C. Latent binary MRF for online reconstruction of large scale systems. Ann Math Artif Intell 77, 123–154 (2016). https://doi.org/10.1007/s10472-015-9470-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-015-9470-x

Keywords

Mathematics Subject Classification (2010)

Search

Navigation