Advertisement

International Journal of Computer Vision

, Volume 99, Issue 1, pp 86–102 | Cite as

How to Compare Noisy Patches? Patch Similarity Beyond Gaussian Noise

  • Charles-Alban DeledalleEmail author
  • Loïc Denis
  • Florence Tupin
Article

Abstract

Many tasks in computer vision require to match image parts. While higher-level methods consider image features such as edges or robust descriptors, low-level approaches (so-called image-based) compare groups of pixels (patches) and provide dense matching. Patch similarity is a key ingredient to many techniques for image registration, stereo-vision, change detection or denoising. Recent progress in natural image modeling also makes intensive use of patch comparison.

A fundamental difficulty when comparing two patches from “real” data is to decide whether the differences should be ascribed to noise or intrinsic dissimilarity. Gaussian noise assumption leads to the classical definition of patch similarity based on the squared differences of intensities. For the case where noise departs from the Gaussian distribution, several similarity criteria have been proposed in the literature of image processing, detection theory and machine learning.

By expressing patch (dis)similarity as a detection test under a given noise model, we introduce these criteria with a new one and discuss their properties. We then assess their performance for different tasks: patch discrimination, image denoising, stereo-matching and motion-tracking under gamma and Poisson noises. The proposed criterion based on the generalized likelihood ratio is shown to be both easy to derive and powerful in these diverse applications.

Keywords

Patch similarity Likelihood ratio Detection Matching 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

11263_2012_519_MOESM1_ESM.pdf (274 kb)
Patch similarity under Gaussian, gamma, Poisson and Cauchy noise: Derivation of closed-form expression of similarity criteria, and Proof sketches of some properties (PDF 274 kB)

References

  1. Alter, F., Matsushita, Y., & Tang, X. (2006). An intensity similarity measure in low-light conditions. Lecture Notes in Computer Science, 3954, 267. CrossRefGoogle Scholar
  2. Baxter, J. (1995). Learning internal representations. In Proceedings of the eighth annual conference on computational learning theory (pp. 311–320). New York: ACM. CrossRefGoogle Scholar
  3. Baxter, J., & Bartlett, P. (1998). The canonical distortion measure in feature space and 1-nn classification. In Advances in neural information processing systems 10: proceedings of the 1997 conference (Vol. 10, p. 245). Cambridge: MIT Press. Google Scholar
  4. Boulanger, J., Kervrann, C., Bouthemy, P., Elbau, P., Sibarita, J., & Salamero, J. (2010). Patch-based nonlocal functional for denoising fluorescence microscopy image sequences. IEEE Transactions on Medical Imaging, 29(2), 442–454. CrossRefGoogle Scholar
  5. Boykov, Y., Veksler, O., & Zabih, R. (1998). Markov random fields with efficient approximations. In Computer vision and pattern recognition, 1998. Proceedings. 1998 IEEE computer society conference on (pp. 648–655). New York: IEEE. Google Scholar
  6. Boykov, Y., Veksler, O., & Zabih, R. (2001). Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1222–1239. Google Scholar
  7. Brown, L., Cai, T., Zhang, R., Zhao, L., & Zhou, H. (2010). The root–unroot algorithm for density estimation as implemented via wavelet block thresholding. Probability Theory and Related Fields, 146(3), 401–433. MathSciNetzbMATHCrossRefGoogle Scholar
  8. Buades, A., Coll, B., & Morel, J. (2005a). a non-local algorithm for image denoising. In Proc. IEEE computer society conf. CVPR (Vol. 2, pp. 60–65). Google Scholar
  9. Buades, A., Coll, B., & Morel, J. (2005b). A review of image denoising algorithms, with a new one. Multiscale Modeling and Simulation, 4(2), 490. MathSciNetzbMATHCrossRefGoogle Scholar
  10. Buades, A., Coll, B., & Morel, J. M. (2009). Non-local means denoising. Image Processing on Line http://www.ipol.im/pub/algo/bcm_non_local_means_denoising/.
  11. Chen, J., Chen, Y., An, W., Cui, Y., & Yang, J. (2011). Nonlocal filtering for polarimetric sar data: a pretest approach. IEEE Transactions on Geoscience and Remote Sensing, 49(5), 1744–1754. CrossRefGoogle Scholar
  12. Cho, T., Avidan, S., & Freeman, W. (2009). The patch transform. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1489–1501. Google Scholar
  13. Comaniciu, D., Ramesh, V., & Meer, O. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 564–577. Google Scholar
  14. Criminisi, A., Pérez, P., & Toyama, K. (2004). Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing, 13(9), 1200–1212. CrossRefGoogle Scholar
  15. Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080. MathSciNetCrossRefGoogle Scholar
  16. Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2008). A nonlocal and shape-adaptive transform-domain collaborative filtering. In Int. workshop on local and non-local approximation in image processing, LNLA. Google Scholar
  17. Deledalle, C., Denis, L., & Tupin, F. (2009a). Débruitage non-local itératif fondé sur un critère de similarité probabiliste. In Proceedings of GRETSI Dijon, France, September 2009. Google Scholar
  18. Deledalle, C., Denis, L., & Tupin, F. (2009b). Iterative weighted maximum likelihood denoising with probabilistic patch-based weights. IEEE Transactions on Image Processing, 18(12), 2661–2672. doi: 10.1109/TIP.2009.2029593. MathSciNetCrossRefGoogle Scholar
  19. Deledalle, C., Tupin, F., & Denis, L. (2010). Poisson NL means: unsupervised non local means for Poisson noise. In Image processing (ICIP), 2010 17th IEEE international conference on (pp. 801–804). New York: IEEE. CrossRefGoogle Scholar
  20. Deledalle, C., Denis, L., & Tupin, F. (2011a). NL-InSAR: non-local interferogram estimation. IEEE Transaction on Geoscience and Remote Sensing, 49, 4. Google Scholar
  21. Deledalle, C. A., Duval, V., & Salmon, J. (2011b). Non-local methods with shape-adaptive patches (NLM-SAP). Journal of Mathematical Imaging and Vision, 1–18. Google Scholar
  22. Deledalle, C. A., Tupin, F., & Denis, L. (2011c). Patch similarity under non-Gaussian noise. In Image processing (ICIP), 18th IEEE international conference on. New York: IEEE. Google Scholar
  23. Efros, A., & Freeman, W. (2001). Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on computer graphics and interactive techniques (pp. 341–346). New York: ACM. Google Scholar
  24. Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15(12), 3736–3745. MathSciNetCrossRefGoogle Scholar
  25. Freeman, W., Jones, T., & Pasztor, E. (2002). Example-based super-resolution. IEEE Computer Graphics and Applications, 56–65. Google Scholar
  26. Gilboa, G., & Osher, S. (2008). Nonlocal linear image regularization and supervised segmentation. Multiscale Modeling and Simulation, 6(2), 595–630. MathSciNetCrossRefGoogle Scholar
  27. Goodman, J. (1976). Some fundamental properties of speckle. Journal of the Optical Society of America, 66(11), 1145–1150. CrossRefGoogle Scholar
  28. Hartley, R., & Zisserman, A. (2000). Multiple view geometry, Vol. 642. Cambridge: Cambridge University Press. zbMATHGoogle Scholar
  29. Horn, B., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17(1–3), 185–203. CrossRefGoogle Scholar
  30. Hudson, H. M. (1978). A natural identity for exponential families with applications in multiparameter estimation. Annals of Statistics, 6(3), 473–484. MathSciNetzbMATHCrossRefGoogle Scholar
  31. Hyvärinen, A., Hurri, J., & Hoyer, P. (2009). Natural image statistics: a probabilistic approach to early computational vision. New York: Springer. zbMATHGoogle Scholar
  32. Ishikawa, H. (2003). Exact optimization for Markov random fields with convex priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1333–1336. Google Scholar
  33. Jain, A. (1989). Fundamentals of digital image processing. Upper Saddle River: Prentice-Hall. zbMATHGoogle Scholar
  34. Katkovnik, V., Foi, A., Egiazarian, K., & Astola, J. (2010). From local kernel to nonlocal multiple-model image denoising. International Journal of Computer Vision, 86(1), 1–32. MathSciNetCrossRefGoogle Scholar
  35. Kay, S. (1998). Fundamentals of statistical signal processing. Volume 2: Detection theory. New York: Prentice Hall. Google Scholar
  36. Kendall, M., & Stuart, A. (1979). The advanced theory of statistics. Vol. 2: Inference and relationship. London: Charles Griffin and Co, Ltd. Google Scholar
  37. Kervrann, C., & Boulanger, J. (2008). Local adaptivity to variable smoothness for exemplar-based image regularization and representation. International Journal of Computer Vision, 79(1), 45–69. CrossRefGoogle Scholar
  38. Kervrann, C., Boulanger, J., & Coupé, P. (2007). Bayesian non-local means filter, image redundancy and adaptive dictionaries for noise removal. In Proceedings of the 1st international conference on scale space and variational methods in computer vision (pp. 520–532). Berlin: Springer. CrossRefGoogle Scholar
  39. Kim, H., & Hero, A. III (2001). Comparison of GLR and invariant detectors under structured clutter covariance. IEEE Transactions on Image Processing, 10(10), 1509–1520. zbMATHCrossRefGoogle Scholar
  40. Kwatra, V., Schödl, A., Essa, I., Turk, G., & Bobick, A. (2003). Graphcut textures: image and video synthesis using graph cuts. ACM Transactions on Graphics, 22(3), 277–286. CrossRefGoogle Scholar
  41. Lehmann, E. (1959). Optimum invariant tests. Annals of Mathematical Statistics, 30(4), 881–884. MathSciNetzbMATHCrossRefGoogle Scholar
  42. Liang, L., Liu, C., Xu, Y., Guo, B., & Shum, H. (2001). Real-time texture synthesis by patch-based sampling. ACM Transactions on Graphics, 20(3), 127–150. CrossRefGoogle Scholar
  43. Lowe, D. (1992). Robust model-based motion tracking through the integration of search and estimation. International Journal of Computer Vision, 8(2), 113–122. CrossRefGoogle Scholar
  44. Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009). Non-local sparse models for image restoration. In ICCV. Google Scholar
  45. Mäkitalo, M., & Foi, A. (2011). Optimal inversion of the Anscombe transformation in low-count Poisson image denoising. IEEE Transactions on Image Processing, 20(1), 99–109. MathSciNetCrossRefGoogle Scholar
  46. Mäkitalo, M., Foi, A., Fevralev, D., & Lukin, V. (2010). Denoising of single-look SAR images based on variance stabilization and nonlocal filters. In Proc. int. conf. math. meth. electromagn. th., MMET 2010, Kiev, Ukraine. Google Scholar
  47. Matsushita, Y., & Lin, S. (2007). A probabilistic intensity similarity measure based on noise distributions. In IEEE conference on computer vision and pattern recognition. CVPR’07 (pp. 1–8). CrossRefGoogle Scholar
  48. Minka, T. (1998). Bayesian inference, entropy, and the multinomial distribution (Tech. rep.), CMU. Google Scholar
  49. Minka, T. (2000). Distance measures as prior probabilities (Tech. rep.), CMU. Google Scholar
  50. Parrilli, S., Poderico, M., Angelino, C., Scarpa, G., & Verdoliva, L. (2010). A nonlocal approach for SAR image denoising. In Geoscience and remote sensing symposium (IGARSS) (pp. 726–729). New York: IEEE. Google Scholar
  51. Peyré, G., Bougleux, S., & Cohen, L. (2008). Non-local regularization of inverse problems. In Computer vision—ECCV 2008 (pp. 57–68). Berlin: Springer. CrossRefGoogle Scholar
  52. Salmon, J. (2010). On two parameters for denoising with non-local means. IEEE Signal Processing Letters, 17, 269–272. CrossRefGoogle Scholar
  53. Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), 7–42. zbMATHCrossRefGoogle Scholar
  54. Seeger, M. (2002). Covariance kernels from Bayesian generative models. In Advances in neural information processing systems 14: proceedings of the 2001 conference (p. 905). Cambridge: MIT Press. Google Scholar
  55. Teuber, T., & Lang, A. (2011). A new similarity measure for nonlocal filtering in the presence of multiplicative noise (Preprint). University of Kaiserslautern. Google Scholar
  56. Van De Ville, D., & Kocher, M. (2009). SURE-based non-local means. IEEE Signal Processing Letters, 16(11), 973–976. CrossRefGoogle Scholar
  57. Varma, M., & Zisserman, A. (2003). Texture classification: are filter banks necessary. In Computer vision and pattern recognition, 2003. Proceedings. 2003 IEEE computer society conference on (Vol. 2, pp. II–691). New York: IEEE. Google Scholar
  58. Yianilos, P. (1995). Metric learning via normal mixtures (Tech. rep.). NEC Research Institute, Princeton, NJ. Google Scholar
  59. Zhang, X., Burger, M., Bresson, X., & Osher, S. (2010). Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM Journal on Imaging Sciences, 3(3), 253–276. doi: 10.1137/090746379, http://link.aip.org/link/?SII/3/253/1. MathSciNetzbMATHCrossRefGoogle Scholar
  60. Zitova, B., & Flusser, J. (2003). Image registration methods: a survey. Image and Vision Computing, 21(11), 977–1000. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Charles-Alban Deledalle
    • 1
    Email author
  • Loïc Denis
    • 2
    • 3
    • 4
  • Florence Tupin
    • 1
  1. 1.Institut Telecom, Telecom ParisTechCNRS LTCIParis cedex 13France
  2. 2.Université de LyonSaint-EtienneFrance
  3. 3.UMR5516, Laboratoire Hubert CurienCNRSSaint-EtienneFrance
  4. 4.Université de Saint-Etienne, Jean MonnetSaint-EtienneFrance

Personalised recommendations