Advertisement

International Journal of Computer Vision

, Volume 79, Issue 1, pp 45–69 | Cite as

Local Adaptivity to Variable Smoothness for Exemplar-Based Image Regularization and Representation

  • Charles KervrannEmail author
  • Jérôme Boulanger
Article

Abstract

A novel adaptive and exemplar-based approach is proposed for image restoration (denoising) and representation. The method is based on a pointwise selection of similar image patches of fixed size in the variable neighborhood of each pixel. The main idea is to associate with each pixel the weighted sum of data points within an adaptive neighborhood. We use small image patches (e.g. 7×7 or 9×9 patches) to compute these weights since they are able to capture local geometric patterns and texels seen in images. In this paper, we mainly focus on the problem of adaptive neighborhood selection in a manner that balances the accuracy of approximation and the stochastic error, at each spatial position. The proposed pointwise estimator is then iterative and automatically adapts to the degree of underlying smoothness with minimal a priori assumptions on the function to be recovered. The method is applied to artificially corrupted real images and the performance is very close, and in some cases even surpasses, to that of the already published denoising methods. The proposed algorithm is demonstrated on real images corrupted by non-Gaussian noise and is used for applications in bio-imaging.

Keywords

Exemplar-based methods Estimation Bias-variance trade-off Restoration Denoising Nonlinear filtering Detection Energy minimization Bio-imaging 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aujol, J. F., Aubert, G., Blanc-Féraud, L., & Chambolle, A. (2005). Image decomposition into a bounded variation component and an oscillating component. Journal of Mathematical Imaging and Vision, 22(1), 71–88. CrossRefMathSciNetGoogle Scholar
  2. Aurich, V., & Weule, J. (1995). Nonlinear Gaussian filters performing edge preserving diffusion. In Proceedings of the 17th DAGM symposium (pp. 538–545), Bielefeld, Germany. Google Scholar
  3. Awate, S. P., & Whitaker, R. T. (2005). Higher-order image statistics for unsupervised, information-theoretic, adaptive image filtering. In Proceedings of the computer vision and pattern recognition (CVPR’05) (Vol. 2, pp. 44–51), San Diego, CA. Google Scholar
  4. Azzabou, N., Paragios, N., Cao, F., & Guichard, F. (2007). Variable bandwidth image denoising using image-based noise models. In Proceedings of the IEEE international conference on computer vision and pattern recognition (CVPR’07) (pp. 1–7). Minneapolis, MN. Google Scholar
  5. Barash, D. (2002). A fundamental relationship between bilateral filtering, adaptive smoothing and the nonlinear diffusion equation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(6), 844–847. CrossRefGoogle Scholar
  6. Barash, D., & Comaniciu, D. (2004). A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean-shift. Image and Video Computing, 22(1), 73–81. CrossRefGoogle Scholar
  7. Blake, A., & Zisserman, A. (1987). Visual reconstruction. Cambridge: MIT Press. Google Scholar
  8. Black, M. J., & Sapiro, G. (1999). Edges as outliers: anisotropic smoothing using local image statistics. In Lecture notes in computer science : Vol. 1682. Proceedings of the scale-space theories in computer vision (Scale-Space’99) (pp. 259–270), Kerkyra, Greece. Berlin: Springer. CrossRefGoogle Scholar
  9. Black, M. J., Sapiro, G., Marimont, D. H., & Heeger, D. (1998). Robust anisotropic diffusion. IEEE Transactions on Image Processing, 7(3), 421–432. CrossRefGoogle Scholar
  10. Boulanger, J., Kervrann, K., & Bouthemy, P. (2007). Space–time adaptation for patch based image sequence restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(6), 1096–1102. CrossRefGoogle Scholar
  11. Boykov, Y., Veksler, O., & Zabih, R. (1998). A variable window approach to early vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12), 1283–1294. CrossRefGoogle Scholar
  12. Brox, T., & Cremers, D. (2007). Iterated non-local means for texture restoration. In In Proceedings of the conference on scale-space and variational methods (SSVM’ 07), Ischia, Italy. Google Scholar
  13. Brox, T., & Weickert, J. (2004). A TV flow based local scale measure for texture discrimination. In Proceedings of the European conference on computer vision (ECCV’04) (Vol. 2, pp. 578–590), Prague, Czech Republic. Google Scholar
  14. Buades, A., Coll, B., & Morel, J.-M. (2005a). A review of image denoising algorithms, with a new one. Multiscale Modeling and Simulation, 4(2), 490–530. CrossRefMathSciNetzbMATHGoogle Scholar
  15. Buades, A., Coll, B., & Morel, J.-M. (2005b). A non local algorithm for image denoising. In Proceedings of the computer vision and pattern recognition (CVPR’05) (Vol. 2, pp. 60–65), San Diego, CA. Google Scholar
  16. Catte, F., Lions, P.-L., Morel, J.-M., & Coll, T. (1992). Image selective smoothing and edge-detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 29(1), 182–193. CrossRefMathSciNetzbMATHGoogle Scholar
  17. Chan, T. F., Osher, S., & Shen, J. (2001). The digital TV filter and nonlinear denoising. IEEE Transactions on Image Processing, 10(2), 231–241. CrossRefzbMATHGoogle Scholar
  18. Cheng, I. (1995). Mean-shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8), 790–799. CrossRefGoogle Scholar
  19. Chu, C. K., Glad, K., Godtliebsen, F., & Marron, J. S. (1998). Edge-preserving smoothers for image processing. Journal of the American Statistical Association, 93(442), 526–555. CrossRefMathSciNetzbMATHGoogle Scholar
  20. Comaniciu, D., & Meer, P. (2002). Mean-shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619. CrossRefGoogle Scholar
  21. Comaniciu, D., Ramesh, V., & Meer, P. (2001). The variable bandwidth mean-shift and data-driven scale selection. In Proceedings of the international conference on computer vision (ICCV’01) (Vol. 1, pp. 438–445), Vancouver, Canada. Google Scholar
  22. Criminisi, A., Pérez, P., & Toyama, K. (2004). Region filling and object removal by exemplar-based inpainting. IEEE Transactions on Image Processing, 13(9), 1200–1212. CrossRefGoogle Scholar
  23. Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8). Google Scholar
  24. De Bonet, J. S. (1997).Noise reduction through detection of signal redundancy. In Rethinking artificial intelligence. MIT AI Lab. Google Scholar
  25. Donoho, D. L., & Johnston, I. M. (1994). Ideal spatial adaptation via wavelet shrinkage. Biometrika, 81, 425–455. CrossRefMathSciNetzbMATHGoogle Scholar
  26. Donoho, D. L., & Johnston, I. M. (1995). Denoising by soft-thresholding. IEEE Transactions on Information Theory, 41, 613–627. CrossRefzbMATHGoogle Scholar
  27. Efros, A., & Leung, T. (1999) Texture synthesis by non-parametric sampling. In Proceedings of the international conference on computer vision (ICCV’99) (pp. 1033–1038), Kerkyra, Greece. Google Scholar
  28. Elad, M. (2002). On the bilateral filter and ways to improve it. IEEE Transactions on Image Processing, 11(10), 1141–1151. CrossRefMathSciNetGoogle Scholar
  29. Elad, M., Aharon, M. (2006) Image denoising via learned dictionaries and sparse representation. In Proceedings of the conference on computer vision and pattern recognition (CVPR’06) (Vol. 1, pp. 895–900), New York. Google Scholar
  30. Fischl, B., & Schwartz, E. L. (1999). Adaptive nonlocal filtering: a fast alternative to anisotropic diffusion for image enhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(1), 42–48. CrossRefGoogle Scholar
  31. Fitzgibbon, A., Wexler, Y., & Zisserman, A. (2003). Image-based rendering using image-based priors. In Proceedings of the international conference on computer vision (ICCV’03), Nice, France. Google Scholar
  32. Freeman, W. T., Pasztor, E. C., & Carmichael, O. T. (2000). Learning low-level vision. International Journal of Computer Vision, 40(1), 25–47. CrossRefzbMATHGoogle Scholar
  33. Gasser, T., Sroka, L., & Jennen Steinmetz, C. (1986). Residual variance and residual pattern in nonlinear regression. Biometrika, 73, 625–633. CrossRefMathSciNetzbMATHGoogle Scholar
  34. Geman, D., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6), 721–741. zbMATHCrossRefGoogle Scholar
  35. Geman, D., Geman, S., Graffigne, C., & Dong, P. (1990). Boundary detection by constrained optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7), 609–628. CrossRefGoogle Scholar
  36. Ghazel, M., Freeman, G. H., & Vrscay, E. R. (2003). Fractal image denoising. IEEE Transactions on Image Processing, 12(12), 1560–1578. CrossRefGoogle Scholar
  37. Gijbels, I., Lambert, A., & Qiu, P. (2006). Edge-preserving image denoising and estimation of discontinuous surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(7), 1075–1087. CrossRefGoogle Scholar
  38. Gilboa, G., & Osher, S. (2007). Nonlocal linear image regularization and supervised segmentation. SIAM Journal on Multiscale Modelling and Simulation, 6, 595–630. CrossRefMathSciNetzbMATHGoogle Scholar
  39. Gilboa, G., Zeevi, Y. Y., & Sochen, N. (2003). Texture preserving variational denoising using an adaptive fidelity term. In Proceedings VLSM’03, Nice, France. Google Scholar
  40. Godtliebsen, F., Spjotvoll, E., & Marron, J. S. (1997). A nonlinear Gaussian filter applied to images with discontinuities. Journal of Nonparametric Statistics, 8, 21–43. CrossRefMathSciNetzbMATHGoogle Scholar
  41. Goldenshluger, A., & Nemirovsky, A. (1997). On spatial adaptive estimation of nonparametric regression. Mathematical Methods of Statistics, 6(2), 135–170. MathSciNetzbMATHGoogle Scholar
  42. Gomez, G., Marroquin, J. L., & Sucar, L. E. (2000). Probabilistic estimation of local scale. In Proceedings of the international conference on pattern recognition (ICPR’00) (Vol. 3, pp. 798–801), Barcelona, Spain. Google Scholar
  43. Hardle, W., & Linton, O. (1994). Applied nonparametric methods. In R.F. Engle & D.L. McFadden (Eds.), Handbook of econometrics (Vol. IV, pp. 2295–2381). North Holland: Amsterdam. Google Scholar
  44. Jojic, N., Frey, B., & Kannan, A. (2003). Epitomic analysis of appearance and shape. In Proceedings of the international conference on computer vision (ICCV’03) (Vol. 1, pp. 34–41), Nice, France. Google Scholar
  45. Juditsky, A. (1997). Wavelet estimators: adapting to unknown smoothness. Mathematical Methods of Statistics, 1, 1–20. MathSciNetGoogle Scholar
  46. Katkovnik, V., Egiazarian, K., & Astola, J. (2002). Adaptive window size image denoising based on intersection of confidence intervals (ICI) rule. Journal of Mathematical Imaging and Vision, 16(3), 223–235. CrossRefMathSciNetzbMATHGoogle Scholar
  47. Kervrann, C. (2004). An adaptive window approach for image smoothing and structures preserving. In Proceedings of the European conference on computer vision (ECCV’04) (Vol. 3, pp. 132–144), Prague, Czech Republic. Google Scholar
  48. Kervrann, C., & Boulanger, J. (2005). Local adaptivity to variable smoothness for exemplar-based image denoising and representation. INRIA Research Report, RR-5624, July 2005. Google Scholar
  49. Kervrann, C., & Boulanger, J. (2006). Unsupervised patch-based image regularization and representation. In Proceedings of the European conference on computer vision (ECCV’06) (Vol. 4, pp. 555–567), Graz, Austria. Google Scholar
  50. Kervrann, C., & Heitz, F. (1995). A Markov random field model-based approach to unsupervised texture segmentation using local and global spatial statistics. IEEE Transactions on Image Processing, 4(6), 856–862. CrossRefGoogle Scholar
  51. Kervrann, C., Boulanger, J., & Coupé, P. (2007). Bayesian non-local means filter, image redundancy and adaptive dictionaries for noise removal. In Proceedings of the conference on scale-space and variational methods (SSVM’ 07), Ischia, Italy. Google Scholar
  52. Kinderman, S., Osher, S., & Jones, P. W. (2005). Deblurring and denoising of images by nonlocal functionals. Multiscale Modeling and Simulation, 4, 1091–1115. CrossRefMathSciNetGoogle Scholar
  53. Lee, J. S. (1983). Digital image smoothing and the sigma filter. Computer Vision, Graphics, and Image Processing, 24, 255–269. CrossRefGoogle Scholar
  54. Le Pennec, E., & Mallat, S. (2005). Sparse geometric image representation with bandelets. IEEE Transactions on Image Processing, 14(4), 423–438. CrossRefMathSciNetGoogle Scholar
  55. Lepskii, O. (1990). On a problem of adaptive estimation on white Gaussian noise. Theory of Probability and Its Applications, 35, 454–466. CrossRefMathSciNetGoogle Scholar
  56. Lepskii, O. (1991). Asymptotically minimax adaptive estimation, 1: uppers bounds. Theory of Probability and Its Applications, 36(4), 654–659. CrossRefMathSciNetGoogle Scholar
  57. Lepski, O. V., Mammen, E., & Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors. Annals of Statistics, 25(3), 929–947. CrossRefMathSciNetzbMATHGoogle Scholar
  58. Lindeberg, T. (1998). Edge detection and ridge detection with automatic scale selection. International Journal of Computer Vision, 30(2), 117–154. CrossRefGoogle Scholar
  59. Maurizot, M., Bouthemy, P., Delyon, B., Juditski, A., & Odobez, J.-M. (1995). Determination of singular points in 2D deformable flow fields. In IEEE proceedings of the international conference on image processing (ICIP’95) (Vol. 3, pp. 488–491), Washington, DC. Google Scholar
  60. Meyer, Y. (2002). University lecture series: vol. 22. Oscillating patterns in image processing and nonlinear evolution equations. Providence: AMS. Google Scholar
  61. Mairal, J., Sapiro, G., & Elad, M. (2007). Multiscale sparse image representation with learned dictionaries. In Proceedings of the international conference on image processing (ICIP’07), San Antonio, TX, USA. Google Scholar
  62. Mrazek, P. (2003). Selection of optimal stopping time for nonlinear diffusion filtering. International Journal of Computer Vision, 52(2/3), 189–203. CrossRefGoogle Scholar
  63. Mrazek, P., Weickert, J., & Bruhn, A. On robust estimation and smoothing with spatial and tonal kernels. Preprint No. 51, University of Bremen, Germany. Google Scholar
  64. Mumford, D., & Shah, J. (1989). Optimal approximations by piecewise smooth functions and variational problems. Communications on Pure and Applied Mathematics, 42(5), 577–685. CrossRefMathSciNetzbMATHGoogle Scholar
  65. Nitzberg, M., & Shiota, T. (1992). Nonlinear image filtering with edge and corner enhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(8), 826–833. CrossRefGoogle Scholar
  66. Osher, S., Solé, A., & Vese, L. (2003). Image decomposition and restoration using total variation minimization and the H −1 norm. Multiscale Modeling and Simulation, 1(3), 349–370. CrossRefMathSciNetzbMATHGoogle Scholar
  67. Perona, P., & Malik, J. (1990). Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7), 629–639. CrossRefGoogle Scholar
  68. Pizurica, A., & Philips, W. (2006). Estimating probability of presence of a signal of interest in multiresolution single and multiband image denoising. IEEE Transactions on Image Processing, 15(3), 654–665. CrossRefGoogle Scholar
  69. Polzehl, J., & Spokoiny, V. (2000). Adaptive weights smoothing with application to image restoration. Journal of the Royal Statistical Society, Series B, 62(2), 335–354. CrossRefMathSciNetGoogle Scholar
  70. Portilla, J., Strela, V., Wainwright, M., & Simoncelli, E. (2003). Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Transactions on Image Processing, 12(11), 1338–1351. CrossRefMathSciNetGoogle Scholar
  71. Roth, S., & Black, M. J. (2005). Fields of experts: a framework for learning image priors with applications. In Proceedings of the conference on computer vision and pattern recognition (CVPR’05) (Vol. 2, pp. 860–867), San Diego, CA. Google Scholar
  72. Rudin, L., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60, 259–268. CrossRefzbMATHGoogle Scholar
  73. Saint-Marc, P., Chen, J. S., & Médioni, G. (1991). Adaptive smoothing: a general tool for early vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6), 514–529. CrossRefGoogle Scholar
  74. Scott, D. W. (1992). Multivariate density estimation. New York: Wiley. zbMATHGoogle Scholar
  75. Singh, M., & Ahuja, N. Regression based bandwidth selection for segmentation using Parzen windows. In Proceedings of the international conference on computer vision (ICCV’03) (Vol. 1, pp. 2–9), Nice, France. Google Scholar
  76. Smith, S. M., & Brady, M. (1997). SUSAN—a new approach to low-level image processing. International Journal of Computer Vision, 23(1), 45–78. CrossRefGoogle Scholar
  77. Spokoiny, V. G. (1998). Estimation of a function with discontinuities via local polynomial fit with an adaptive window choice. Annals of Statistics, 26(4), 141–170. MathSciNetGoogle Scholar
  78. Stankovic, L. (2004). Performance analysis of the adaptive algorithm for bias-to-variance trade-off. IEEE Transactions on Signal Processing, 52(5), 1228–1234. CrossRefMathSciNetGoogle Scholar
  79. Sochen, N., Kimmel, R., & Bruckstein, A. M. (2001). Diffusions and confusions in signal and image processing. Journal of Mathematical Imaging and Vision, 14(3), 237–244. CrossRefMathSciNetGoogle Scholar
  80. Spira, A., Kimmel, R., & Sochen, N. (2003). Efficient Beltrami flow using a short-time kernel. In Proceedings of the international conference on scale-space theories in computer vision (Scale-Space’03) (pp. 551–522), Isle of Skye, Scotland. Google Scholar
  81. Starck, J. L., Candes, E., & Donoho, D. L. (2002). The curvelet transform for image denoising. IEEE Transactions on Image Processing, 11(6), 670–684. CrossRefMathSciNetGoogle Scholar
  82. Stewart, C. V., Tsai, C.-L., & Roysam, B. (2003). The dual-bootstrap iterative closest point algorithm with application to retinal image registration. IEEE Transactions on Medical Imaging, 22(11), 1379–1394. CrossRefGoogle Scholar
  83. Tomasi, C., & Manduchi, R. (1998). Bilateral filtering for gray and color images. In Proceedings of the international confernce on computer vision (ICCV’98) (pp. 839–846), Bombay, India. Google Scholar
  84. Tschumperlé, D. (2006). Curvature-preserving regularization of multi-valued images using PDE’s. In Proceedings of the European confernce on computer vision (ECCV’06) (Vol. 2, pp. 295–307), Graz, Austria. Google Scholar
  85. van de Weijer, J., & van den Boomgaard, R. (2001). Local mode filtering. In Proceedings of the confernce on computer vision and pattern recognition (CVPR’01) (Vol. II, pp. 428–433), Kauai, HI. Google Scholar
  86. van den Boomgaard, R., & van de Weijer, J. (2002). On the equivalence of local-mode finding, robust estimation and mean-shift analysis as used in early vision tasks. In Proceedings of the international confernce on pattern recognition (ICPR’02) (Vol. III, pp. 927–930), Quebec City, Canada. Google Scholar
  87. Wang, Z., & Zhang, D. (1998). Restoration of impulse noise corrupted images using long-range correlation. IEEE Signal Processing Letters, 5(0), 4–6. CrossRefGoogle Scholar
  88. Weickert, J. (1998). Anisotropic diffusion in image processing. Stuttgart: Teubner. zbMATHGoogle Scholar
  89. Weickert, J. (1999). Coherence-enhancing diffusion filtering. International Journal of Computer Vision, 31(2–3), 111–127. CrossRefGoogle Scholar
  90. Yang, G. Z., Burger, P., Firmin, D. N., & Underwood, S. R. (1996). Structure adaptive anisotropic image filtering. Image and Vision Computing, 14, 135–145. CrossRefGoogle Scholar
  91. Yaroslavsky, L. P., & Eden, M. (1996). Fundamentals of digital optics. Boston: Birkhäuser. zbMATHGoogle Scholar
  92. Zhang, D., & Wang, Z. (2002). Image information restoration based on long-range correlation. IEEE Transactions on Circuits and Systems for Video Technology, 12, 331–341. CrossRefGoogle Scholar
  93. Zhu, S. C., Wu, Y., & Mumford, D. (1998). Filters, random fields and maximum entropy (FRAME): towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2), 107–126. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.IRISA/INRIA RennesCampus Universitaire de BeaulieuRennes cedexFrance
  2. 2.INRAUR341 Mathématiques et Informatique AppliquéesJouy-en-JosasFrance

Personalised recommendations