Abstract
PatchMatch is a fast algorithm for computing dense approximate nearest neighbor correspondences between patches of two image regions [1]. This paper generalizes PatchMatch in three ways: (1) to find k nearest neighbors, as opposed to just one, (2) to search across scales and rotations, in addition to just translations, and (3) to match using arbitrary descriptors and distances, not just sum-of-squared-differences on patch colors. In addition, we offer new search and parallelization strategies that further accelerate the method, and we show performance improvements over standard kd-tree techniques across a variety of inputs. In contrast to many previous matching algorithms, which for efficiency reasons have restricted matching to sparse interest points, or spatially proximate matches, our algorithm can efficiently find global, dense matches, even while matching across all scales and rotations. This is especially useful for computer vision applications, where our algorithm can be used as an efficient general-purpose component. We explore a variety of vision applications: denoising, finding forgeries by detecting cloned regions, symmetry detection, and object detection.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (Proc. SIGGRAPH) 28, 24 (2009)
Buades, A., Coll, B., Morel, J.: A non-local algorithm for image denoising. In: Proc. CVPR II, p. 60 (2005)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proc. ICCV, vol. 5 (2007)
Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1615–1630 (2005)
Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 91–110 (2004)
Brox, T., Malik, J.: Large displacement optical flow. In: CVPR (2009)
Simon, I., Seitz, S.: A probabilistic model for object recognition, segmentation, and non-rigid correspondence. In: Proc. CVPR, pp. 1–7 (2007)
Hays, J., Leordeanu, M., Efros, A., Liu, Y.: Discovering texture regularity as a higher-order correspondence problem. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, p. 522. Springer, Heidelberg (2006)
Boiman, O., Irani, M.: Detecting irregularities in images and in video. International Journal of Computer Vision 74, 17–31 (2007)
Bagon, S., Boiman, O., Irani, M.: What is a good image segment? A unified approach to segment extraction. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, p. 44. Springer, Heidelberg (2008)
Popescu, A., Farid, H.: Exposing digital forgeries by detecting duplicated image regions. Department of Computer Science, Dartmouth College (2004)
Freeman, W., Jones, T., Pasztor, E.: Example-based super-resolution. IEEE Computer Graphics and Applications, 56–65 (2002)
Efros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: Proc. ICCV, vol. 2, pp. 1033–1038 (1999)
Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. on Image Processing 13 (2004)
Simakov, D., Caspi, Y., Shechtman, E., Irani, M.: Summarizing visual data using bidirectional similarity. In: Proc. CVPR (2008)
Hertzmann, A., Jacobs, C., Oliver, N., Curless, B., Salesin, D.: Image analogies. ACM Transactions on Graphics (Proc. SIGGRAPH), 327–340 (2001)
Mount, D.M., Arya, S.: ANN: A library for approx. nearest neighbor search (1997)
Boiman, O., Shechtman, E., Irani, M.: In defense of nearest-neighbor based image classification. In: Proc. CVPR., vol. 2, p. 6 (2008)
Niyogi, S., Freeman, W.: Example-based head tracking. In: Proc. of Conf. on Automatic Face and Gesture Recognition (FG 1996), p. 374 (1996)
Kumar, N., Zhang, L., Nayar, S.K.: What is a good nearest neighbors algorithm for finding similar patches in images? In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 364–378. Springer, Heidelberg (2008)
Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: Proc. CVPR, vol. 5 (2006)
Muja, M., Lowe, D.: Fast approximate nearest neighbors with automatic algorithm configuration. In: VISAPP (2009)
Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.: Locality-sensitive hashing scheme based on p-stable distributions. In: Symp. on Comp. Geom., pp. 253–262 (2004)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Processing 16 (2007)
Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: Proc. of ICCV (2009)
Brox, T., Kleinschmidt, O., Cremers, D.: Efficient nonlocal means for denoising of textural patterns. IEEE Transactions on Image Processing 17, 1083–1092 (2008)
Bayram, S., Sencar, H., Memon, N.: A Survey of Copy-Move Forgery Detection Techniques. In: IEEE Western New York Image Processing Workshop (2008)
Park, M., Leey, S., Cheny, P., Kashyap, S., Butty, A., Liuy, Y.: Performance evaluation of state-of-the-art discrete symmetry detection. In: CVPR (2008)
Fischler, M., Bolles, R.: Random sample consensus: A paradigm for model fitting with apps. to image analysis and automated cartography. ACM Comm. 24 (1981)
Jain, A., Zhong, Y., Dubuisson-Jolly, M.: Deformable template models: A review. Signal Processing 71, 109–129 (1998)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proc. CVPR (2001)
Guo, G., Dyer, C.: Patch-based image correlation with rapid filtering. In: The 2nd Beyond Patches Workshop, in Conj. with IEEE CVPR 2007 (2007)
Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.T.: SIFT flow: Dense correspondence across different scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, p. 28. Springer, Heidelberg (2008)
Glasner, D., Bagon, S., Irani, M.: Super-Resolution from a Single Image. In: Proc. of ICCV (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
1 Electronic Supplementary Material
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Barnes, C., Shechtman, E., Goldman, D.B., Finkelstein, A. (2010). The Generalized PatchMatch Correspondence Algorithm. In: Daniilidis, K., Maragos, P., Paragios, N. (eds) Computer Vision – ECCV 2010. ECCV 2010. Lecture Notes in Computer Science, vol 6313. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15558-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-15558-1_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15557-4
Online ISBN: 978-3-642-15558-1
eBook Packages: Computer ScienceComputer Science (R0)