Efficient Belief Propagation with Learned Higher-Order Markov Random Fields

  • Xiangyang Lan
  • Stefan Roth
  • Daniel Huttenlocher
  • Michael J. Black
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3952)


Belief propagation (BP) has become widely used for low-level vision problems and various inference techniques have been proposed for loopy graphs. These methods typically rely on ad hoc spatial priors such as the Potts model. In this paper we investigate the use of learned models of image structure, and demonstrate the improvements obtained over previous ad hoc models for the image denoising problem. In particular, we show how both pairwise and higher-order Markov random fields with learned clique potentials capture rich image structures that better represent the properties of natural images. These models are learned using the recently proposed Fields-of-Experts framework. For such models, however, traditional BP is computationally expensive. Consequently we propose some approximation methods that make BP with learned potentials practical. In the case of pairwise models we propose a novel approximation of robust potentials using a finite family of quadratics. In the case of higher order MRFs, with 2× 2 cliques, we use an adaptive state space to handle the increased complexity. Extensive experiments demonstrate the power of learned models, the benefits of higher-order MRFs and the practicality of BP for these problems with the use of simple principled approximations.


Belief Propagation Variable Node Image Denoising Factor Node Factor Graph 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE PAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  2. 2.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE PAMI 26(9), 1124–1137 (2004)CrossRefMATHGoogle Scholar
  3. 3.
    Kolmogorov, V., Zabih, R.: Multi-camera scene reconstruction via graph cuts. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 82–96. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. In: CVPR, vol. 1, pp. 261–268 (2004)Google Scholar
  5. 5.
    Tappen, M.F., Russell, B.C., Freeman, W.T.: Efficient graphical models for processing images. In: CVPR, vol. 2, pp. 673–680 (2004)Google Scholar
  6. 6.
    Yedidia, J.S., Freeman, W.T., Weiss, Y.: Constructing free energy approximations and generalized belief propagation algorithms. Technical Report TR-2004-040, Mitsubishi Electric Research Laboratories, Cambridge, Massachusetts (2004)Google Scholar
  7. 7.
    Zhu, S.C., Wu, Y., Mumford, D.: Filters, random fields and maximum entropy (FRAME): Towards a unified theory for texture modeling. IJCV 27(2), 107–126 (1998)CrossRefGoogle Scholar
  8. 8.
    Paget, R., Longstaff, I.D.: Texture synthesis via a noncausal nonparametric multiscale Markov random field. IEEE T. Image Proc. 7(6), 925–931 (1998)CrossRefGoogle Scholar
  9. 9.
    Roth, S., Black, M.J.: Fields of experts: A framework for learning image priors. In: CVPR, vol. 2, pp. 860–867 (2005)Google Scholar
  10. 10.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE PAMI 6, 721–741 (1984)CrossRefMATHGoogle Scholar
  11. 11.
    Gimel’farb, G.L.: Texture modeling by multiple pairwise pixel interactions. IEEE PAMI 18(11), 1110–1114 (1996)CrossRefGoogle Scholar
  12. 12.
    Zalesny, A., van Gool, L.: A compact model for viewpoint dependent texture synthesis. In: Pollefeys, M., Van Gool, L., Zisserman, A., Fitzgibbon, A.W. (eds.) SMILE 2000. LNCS, vol. 2018, pp. 124–143. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  13. 13.
    Huang, J.: Statistics of Natural Images and Models. PhD thesis, Brown University (2000)Google Scholar
  14. 14.
    Hinton, G.E.: Products of experts. In: ICANN, vol. 1, pp. 1–6 (1999)Google Scholar
  15. 15.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423 (2001)Google Scholar
  16. 16.
    Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neur. Comp. 14(8), 1771–1800 (2002)CrossRefMATHGoogle Scholar
  17. 17.
    Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D.H., Cohen, M.: Interactive digital photomontage. ACM SIGGRAPH 23(3), 294–302 (2004)CrossRefGoogle Scholar
  18. 18.
    Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters. In: ICCV, vol. 2, pp. 900–907 (2003)Google Scholar
  19. 19.
    Kschischang, F.R., Frey, B.J., Loelinger, H.A.: Fractor graphs and the sum-product algorithm. IEEE T. Info. Th. 47(2), 498–519 (2001)CrossRefGoogle Scholar
  20. 20.
    Sun, J., Zhen, N.N., Shum, H.Y.: Stereo matching using belief propagation. IEEE PAMI 25(7), 787–800 (2003)CrossRefGoogle Scholar
  21. 21.
    Black, M.J., Rangarajan, A.: On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. IJCV 19(1), 57–92 (1996)CrossRefGoogle Scholar
  22. 22.
    Portilla, J., Strela, V., Wainwright, M.J., Simoncelli, E.P.: Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE T. Image Proc. 12(11), 1338–1351 (2003)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error visibility to structural similarity. IEEE T. Image Proc. 13(4), 600–612 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Xiangyang Lan
    • 1
  • Stefan Roth
    • 2
  • Daniel Huttenlocher
    • 1
  • Michael J. Black
    • 2
  1. 1.Computer Science DepartmentCornell UniversityIthacaUSA
  2. 2.Department of Computer ScienceBrown UniversityProvidenceUSA

Personalised recommendations