International Journal of Computer Vision

, Volume 110, Issue 1, pp 2–13 | Cite as

PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation

  • Frederic Besse
  • Carsten Rother
  • Andrew Fitzgibbon
  • Jan Kautz
Article

Abstract

PatchMatch (PM) is a simple, yet very powerful and successful method for optimizing continuous labelling problems. The algorithm has two main ingredients: the update of the solution space by sampling and the use of the spatial neighbourhood to propagate samples. We show how these ingredients are related to steps in a specific form of belief propagation (BP) in the continuous space, called max-product particle BP (MP-PBP). However, MP-PBP has thus far been too slow to allow complex state spaces. In the case where all nodes share a common state space and the smoothness prior favours equal values, we show that unifying the two approaches yields a new algorithm, PMBP, which is more accurate than PM and orders of magnitude faster than MP-PBP. To illustrate the benefits of our PMBP method we have built a new stereo matching algorithm with unary terms which are borrowed from the recent PM Stereo work and novel realistic pairwise terms that provide smoothness. We have experimentally verified that our method is an improvement over state-of-the-art techniques at sub-pixel accuracy level.

Keywords

Correspondence fields Belief propagation PatchMatch 

References

  1. Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009). PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (Proceedings of SIGGRAPH), 28(3), 24.Google Scholar
  2. Barnes, C., Shechtman, E., Goldman, D. B., & Finkelstein, A. (2010). The generalized PatchMatch correspondence algorithm. In Proceedings of ECCV.Google Scholar
  3. Bleyer, M., Rhemann, C., & Rother, C. (2011). PatchMatch Stereo—Stereo matching with slanted support windows. In Proceedings of BMVC.Google Scholar
  4. Boltz, S., & Nielsen, F. (2010). Randomized motion estimation. In Proceedings of ICIP (pp. 781–784).Google Scholar
  5. HaCohen, Y., Shechtman, E., Goldman, D. B., & Lischinski, D. (2011). Non-rigid dense correspondence with applications for image enhancement. ACM Transactions on Graphics (Proceedings of SIGGRAPH), 30(4), 70:1–70:9.Google Scholar
  6. He, K., Rhemann, C., Rother, C., Tang, X., & Sun, J. (2011). A global sampling method for alpha matting. In Proceedings of CVPR (pp. 2049–2056).Google Scholar
  7. Ihler, A., & McAllester, D. (2009). Particle belief propagation. In Proceedings of AISTATS (Vol. 5, pp. 256–263).Google Scholar
  8. Isard, M., MacCormick, J., & Achan, K. (2008). Continuously-adaptive discretization for message-passing algorithms. In Proceedings of NIPS (pp. 737–744).Google Scholar
  9. Kolmogorov, V. (2006). Convergent tree-reweighted message passing for energy minimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10), 1568–1583.CrossRefGoogle Scholar
  10. Korman, S., & Avidan, S. (2011). Coherency sensitive hashing. In Proceedings of ICCV (pp. 1607–1614).Google Scholar
  11. Kothapa, R., Pachecho, J., & Sudderth, E. B. (2011). Max-product particle belief propagation. Master’s Thesis, Brown University. Google Scholar
  12. Mansfield, A., Prasad, M., Rother, C., Sharp, T., Kohli, P., & Van Gool, L. (2011). Transforming image completion. In Proceedings of BMVC.Google Scholar
  13. Noorshams, N., & Wainwright, M. J. (2011). Stochastic belief propagation: Low-complexity message-passing with guarantees. In Communication, Control, and Computing (Allerton), 2011 49th Annual Allerton Conference. IEEE (pp. 269–276).Google Scholar
  14. Nowozin, S., & Lampert, C. (2011). Structured learning and prediction in computer vision (Vol. 6). Boston: Now publishers Inc.Google Scholar
  15. Pal, C., Sutton, C., & McCallum, A. (2006). Sparse forward–backward using minimum divergence beams for fast training of conditional random fields. In Proceedings of ICASSP (Vol. 5).Google Scholar
  16. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Francisco: Morgan Kaufmann Publishers Inc.Google Scholar
  17. Peng, J., Hazan, T., McAllester, D. A., & Urtasun, R. (2011). Convex max-product algorithms for continuous MRFs with applications to protein folding. In Proceedings of ICML.Google Scholar
  18. Sudderth, E. B., Ihler, A. T., Isard, M., Freeman, W. T., & Willsky, A. S. (2010). Nonparametric belief propagation. Communications of the ACM, 53(10), 95–103.CrossRefGoogle Scholar
  19. Yamaguchi, K., Hazan, T., McAllester, D., & Urtasun, R. (2012). Continuous Markov random fields for robust stereo estimation. In Computer Vision—ECCV 2012 (pp. 45–58). Berlin, Heidelberg: Springer.Google Scholar
  20. Yedidia, J., Freeman, W., & Weiss, Y. (2005). Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Transactions on Information Theory, 51(7), 2282–2312.MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Frederic Besse
    • 1
  • Carsten Rother
    • 1
  • Andrew Fitzgibbon
    • 1
  • Jan Kautz
    • 1
  1. 1.University College LondonLondonUK

Personalised recommendations