Abstract
Many tasks in computer vision require to match image parts. While higher-level methods consider image features such as edges or robust descriptors, low-level approaches (so-called image-based) compare groups of pixels (patches) and provide dense matching. Patch similarity is a key ingredient to many techniques for image registration, stereo-vision, change detection or denoising. Recent progress in natural image modeling also makes intensive use of patch comparison.
A fundamental difficulty when comparing two patches from “real” data is to decide whether the differences should be ascribed to noise or intrinsic dissimilarity. Gaussian noise assumption leads to the classical definition of patch similarity based on the squared differences of intensities. For the case where noise departs from the Gaussian distribution, several similarity criteria have been proposed in the literature of image processing, detection theory and machine learning.
By expressing patch (dis)similarity as a detection test under a given noise model, we introduce these criteria with a new one and discuss their properties. We then assess their performance for different tasks: patch discrimination, image denoising, stereo-matching and motion-tracking under gamma and Poisson noises. The proposed criterion based on the generalized likelihood ratio is shown to be both easy to derive and powerful in these diverse applications.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alter, F., Matsushita, Y., & Tang, X. (2006). An intensity similarity measure in low-light conditions. Lecture Notes in Computer Science, 3954, 267.
Baxter, J. (1995). Learning internal representations. In Proceedings of the eighth annual conference on computational learning theory (pp. 311–320). New York: ACM.
Baxter, J., & Bartlett, P. (1998). The canonical distortion measure in feature space and 1-nn classification. In Advances in neural information processing systems 10: proceedings of the 1997 conference (Vol. 10, p. 245). Cambridge: MIT Press.
Boulanger, J., Kervrann, C., Bouthemy, P., Elbau, P., Sibarita, J., & Salamero, J. (2010). Patch-based nonlocal functional for denoising fluorescence microscopy image sequences. IEEE Transactions on Medical Imaging, 29(2), 442–454.
Boykov, Y., Veksler, O., & Zabih, R. (1998). Markov random fields with efficient approximations. In Computer vision and pattern recognition, 1998. Proceedings. 1998 IEEE computer society conference on (pp. 648–655). New York: IEEE.
Boykov, Y., Veksler, O., & Zabih, R. (2001). Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1222–1239.
Brown, L., Cai, T., Zhang, R., Zhao, L., & Zhou, H. (2010). The root–unroot algorithm for density estimation as implemented via wavelet block thresholding. Probability Theory and Related Fields, 146(3), 401–433.
Buades, A., Coll, B., & Morel, J. (2005a). a non-local algorithm for image denoising. In Proc. IEEE computer society conf. CVPR (Vol. 2, pp. 60–65).
Buades, A., Coll, B., & Morel, J. (2005b). A review of image denoising algorithms, with a new one. Multiscale Modeling and Simulation, 4(2), 490.
Buades, A., Coll, B., & Morel, J. M. (2009). Non-local means denoising. Image Processing on Line http://www.ipol.im/pub/algo/bcm_non_local_means_denoising/.
Chen, J., Chen, Y., An, W., Cui, Y., & Yang, J. (2011). Nonlocal filtering for polarimetric sar data: a pretest approach. IEEE Transactions on Geoscience and Remote Sensing, 49(5), 1744–1754.
Cho, T., Avidan, S., & Freeman, W. (2009). The patch transform. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1489–1501.
Comaniciu, D., Ramesh, V., & Meer, O. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 564–577.
Criminisi, A., Pérez, P., & Toyama, K. (2004). Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing, 13(9), 1200–1212.
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080.
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2008). A nonlocal and shape-adaptive transform-domain collaborative filtering. In Int. workshop on local and non-local approximation in image processing, LNLA.
Deledalle, C., Denis, L., & Tupin, F. (2009a). Débruitage non-local itératif fondé sur un critère de similarité probabiliste. In Proceedings of GRETSI Dijon, France, September 2009.
Deledalle, C., Denis, L., & Tupin, F. (2009b). Iterative weighted maximum likelihood denoising with probabilistic patch-based weights. IEEE Transactions on Image Processing, 18(12), 2661–2672. doi:10.1109/TIP.2009.2029593.
Deledalle, C., Tupin, F., & Denis, L. (2010). Poisson NL means: unsupervised non local means for Poisson noise. In Image processing (ICIP), 2010 17th IEEE international conference on (pp. 801–804). New York: IEEE.
Deledalle, C., Denis, L., & Tupin, F. (2011a). NL-InSAR: non-local interferogram estimation. IEEE Transaction on Geoscience and Remote Sensing, 49, 4.
Deledalle, C. A., Duval, V., & Salmon, J. (2011b). Non-local methods with shape-adaptive patches (NLM-SAP). Journal of Mathematical Imaging and Vision, 1–18.
Deledalle, C. A., Tupin, F., & Denis, L. (2011c). Patch similarity under non-Gaussian noise. In Image processing (ICIP), 18th IEEE international conference on. New York: IEEE.
Efros, A., & Freeman, W. (2001). Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on computer graphics and interactive techniques (pp. 341–346). New York: ACM.
Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15(12), 3736–3745.
Freeman, W., Jones, T., & Pasztor, E. (2002). Example-based super-resolution. IEEE Computer Graphics and Applications, 56–65.
Gilboa, G., & Osher, S. (2008). Nonlocal linear image regularization and supervised segmentation. Multiscale Modeling and Simulation, 6(2), 595–630.
Goodman, J. (1976). Some fundamental properties of speckle. Journal of the Optical Society of America, 66(11), 1145–1150.
Hartley, R., & Zisserman, A. (2000). Multiple view geometry, Vol. 642. Cambridge: Cambridge University Press.
Horn, B., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17(1–3), 185–203.
Hudson, H. M. (1978). A natural identity for exponential families with applications in multiparameter estimation. Annals of Statistics, 6(3), 473–484.
Hyvärinen, A., Hurri, J., & Hoyer, P. (2009). Natural image statistics: a probabilistic approach to early computational vision. New York: Springer.
Ishikawa, H. (2003). Exact optimization for Markov random fields with convex priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1333–1336.
Jain, A. (1989). Fundamentals of digital image processing. Upper Saddle River: Prentice-Hall.
Katkovnik, V., Foi, A., Egiazarian, K., & Astola, J. (2010). From local kernel to nonlocal multiple-model image denoising. International Journal of Computer Vision, 86(1), 1–32.
Kay, S. (1998). Fundamentals of statistical signal processing. Volume 2: Detection theory. New York: Prentice Hall.
Kendall, M., & Stuart, A. (1979). The advanced theory of statistics. Vol. 2: Inference and relationship. London: Charles Griffin and Co, Ltd.
Kervrann, C., & Boulanger, J. (2008). Local adaptivity to variable smoothness for exemplar-based image regularization and representation. International Journal of Computer Vision, 79(1), 45–69.
Kervrann, C., Boulanger, J., & Coupé, P. (2007). Bayesian non-local means filter, image redundancy and adaptive dictionaries for noise removal. In Proceedings of the 1st international conference on scale space and variational methods in computer vision (pp. 520–532). Berlin: Springer.
Kim, H., & Hero, A. III (2001). Comparison of GLR and invariant detectors under structured clutter covariance. IEEE Transactions on Image Processing, 10(10), 1509–1520.
Kwatra, V., Schödl, A., Essa, I., Turk, G., & Bobick, A. (2003). Graphcut textures: image and video synthesis using graph cuts. ACM Transactions on Graphics, 22(3), 277–286.
Lehmann, E. (1959). Optimum invariant tests. Annals of Mathematical Statistics, 30(4), 881–884.
Liang, L., Liu, C., Xu, Y., Guo, B., & Shum, H. (2001). Real-time texture synthesis by patch-based sampling. ACM Transactions on Graphics, 20(3), 127–150.
Lowe, D. (1992). Robust model-based motion tracking through the integration of search and estimation. International Journal of Computer Vision, 8(2), 113–122.
Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009). Non-local sparse models for image restoration. In ICCV.
Mäkitalo, M., & Foi, A. (2011). Optimal inversion of the Anscombe transformation in low-count Poisson image denoising. IEEE Transactions on Image Processing, 20(1), 99–109.
Mäkitalo, M., Foi, A., Fevralev, D., & Lukin, V. (2010). Denoising of single-look SAR images based on variance stabilization and nonlocal filters. In Proc. int. conf. math. meth. electromagn. th., MMET 2010, Kiev, Ukraine.
Matsushita, Y., & Lin, S. (2007). A probabilistic intensity similarity measure based on noise distributions. In IEEE conference on computer vision and pattern recognition. CVPR’07 (pp. 1–8).
Minka, T. (1998). Bayesian inference, entropy, and the multinomial distribution (Tech. rep.), CMU.
Minka, T. (2000). Distance measures as prior probabilities (Tech. rep.), CMU.
Parrilli, S., Poderico, M., Angelino, C., Scarpa, G., & Verdoliva, L. (2010). A nonlocal approach for SAR image denoising. In Geoscience and remote sensing symposium (IGARSS) (pp. 726–729). New York: IEEE.
Peyré, G., Bougleux, S., & Cohen, L. (2008). Non-local regularization of inverse problems. In Computer vision—ECCV 2008 (pp. 57–68). Berlin: Springer.
Salmon, J. (2010). On two parameters for denoising with non-local means. IEEE Signal Processing Letters, 17, 269–272.
Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), 7–42.
Seeger, M. (2002). Covariance kernels from Bayesian generative models. In Advances in neural information processing systems 14: proceedings of the 2001 conference (p. 905). Cambridge: MIT Press.
Teuber, T., & Lang, A. (2011). A new similarity measure for nonlocal filtering in the presence of multiplicative noise (Preprint). University of Kaiserslautern.
Van De Ville, D., & Kocher, M. (2009). SURE-based non-local means. IEEE Signal Processing Letters, 16(11), 973–976.
Varma, M., & Zisserman, A. (2003). Texture classification: are filter banks necessary. In Computer vision and pattern recognition, 2003. Proceedings. 2003 IEEE computer society conference on (Vol. 2, pp. II–691). New York: IEEE.
Yianilos, P. (1995). Metric learning via normal mixtures (Tech. rep.). NEC Research Institute, Princeton, NJ.
Zhang, X., Burger, M., Bresson, X., & Osher, S. (2010). Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM Journal on Imaging Sciences, 3(3), 253–276. doi:10.1137/090746379, http://link.aip.org/link/?SII/3/253/1.
Zitova, B., & Flusser, J. (2003). Image registration methods: a survey. Image and Vision Computing, 21(11), 977–1000.
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
11263_2012_519_MOESM1_ESM.pdf
Patch similarity under Gaussian, gamma, Poisson and Cauchy noise: Derivation of closed-form expression of similarity criteria, and Proof sketches of some properties (PDF 274 kB)
Rights and permissions
About this article
Cite this article
Deledalle, CA., Denis, L. & Tupin, F. How to Compare Noisy Patches? Patch Similarity Beyond Gaussian Noise. Int J Comput Vis 99, 86–102 (2012). https://doi.org/10.1007/s11263-012-0519-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-012-0519-6