Abstract
Image denoising has been a popularly studied problem for several decades in image processing and low-level computer vision communities. Many effective denoising approaches, such as BM3D, utilize spatial redundancy of patches (relatively small, cropped windows) either within a single natural image, or within a large collection of natural images. In this chapter, we summarize our previous finding that “Internal-Denoising” (based on internal noisy patches) can outperform “External Denoising” (based on external clean patches), especially in the presence of high noise levels. We explain this phenomenon in terms of “Patch Signal-to-Noise Ratio” (PatchSNR) , an inherent characteristic of a noisy patch that determines its preference of either internal or external denoising. We further experiment with the recent state-of-the-art convolutional residual neural network for Gaussian denoising. We show that it closes the gap on the previously reported external denoising bounds. We further compare its performance to internal local multi-scale Oracle (that has the same receptive field as the network). We show that for patches with low PatchSNR, the network does not manage to reconstruct the best “clean” patch that resides in the network’s receptive field. This suggests that the future challenge of denoising community is to train an image-specific CNN that will exploit local recurrence of patches, without relying on external examples, as was recently successfully done for super-resolution task. Combining such a model with external-based models may push PSNR bounds further up and improve denoising by \(\sim \)1–2 dB, especially for higher noise levels.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Using Matlab “imresize” with a bicubic kernel.
References
Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Siganl Process 54(11):4311–4322
Aharon M, Elad M, Bruckstein A (2005) Design of dictionaries for sparse representation. In: Workshop on signal processing with adaptative sparse structured representations (SPARS). Rennes, France
Alvarez L, Lions PL, Morel JM (2005) Image selective smoothing and edge detection by nonlinear diffusion. J Numer Analysis 29:845–866
Bae W, Yoo J, Ye JC (2017) Beyond deep residual learning for image restoration: persistent homology-guided manifold simplification. In: Conference on computer vision and pattern recognition (CVPR)
Berkely segmentation dataset. https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/
Buades A, Coll B, Morel J (2005) A non-local algorithm for image denoising. In: IEEE international conference on computer vision and pattern recognition (CVPR)
Buadess A, Coll B, Morel J (2005) A review of image denoising algorithms, with a new one. Multiscale Model Simul 4(2):490–530
Burger HC, Schuler CJ, Harmeling S (2013) Learning how to combine internal and external denoising methods. In: German conference on pattern recognition (GCPR13), pp 121–130
Chatterjee P, Milanfar P (2010) Is denoising dead? IEEE Trans Image Process 19(4):896–911
Chen F, Zhang L, Yu H (2015) External patch prior guided internal clustering for image denoising. In: IEEE international conference on computer vision (ICCV)
Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3D transform-domain collaborative filtering. IEEE Trans Image Process 16(8)
Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 54(12):3736–3745
Elad M, Aharon M (2006) Image denoising via learned dictionaries and sparse representations. In: IEEE conference on computer vision and pattern recognition (CVPR)
Harmeling S, Schuler CJ, Burger, HC (2002) Image denoising: can plain neural networks compete with BM3D? In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2392–2399
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Conference on computer vision and pattern recognition (CVPR)
Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5)
Levin A, Nadler B (2011) Natural image denoising: optimality and inherent bounds. In: IEEE conference on computer vision and pattern recognition (CVPR)
Levin A, Nadler B, Durand F, Freeman WT (2012) Patch complexity, finite pixel correlations and optimal denoising. In: European conference on computer vision (ECCV)
Luo E, Chan SH, Nguyen TQ (2016) Adaptive image denoising by mixture adaptation. IEEE Trans Image Process 25(10)
Luo W, Li Y, Urtasun R, Zemel R (2016) Understanding the effective receptive field in deep convolutional neural networks. In: Advances in neural information processing systems
Mairal J, Elad M, Sapiro G (2008) Sparse representation for color image restoration. IEEE Trans Image Process 17(1):53–69
Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2009) Non-local sparse models for image restoration (ICCV)
Mosseri I, Zontak M, Irani M (2013) Combining the power of internal and external denoising. In: IEEE international conference on computational photography (ICCP)
Parzen E (1962) On estimation of a probability density function and mode. Ann Math Stat 33:1065–1076
Perona P, Malik J (1990) Scale space and edge detection using anisotropic diffusion. IEEE Trans Patt Anal Mach Intell 12:629–639
Portilla J, Strela V, Wainwright M, Simoncelli E (2003) Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans Image Process 12(11)
Shocher A, Cohen N, Irani M (2018) “Zero-Shot” super-resolution using deep internal learning. In: Conference on computer vision and pattern recognition (CVPR)
Simoncelli E, Freeman W (1995) The steerable pyramid: a exible architecture for multi-scale derivative computation. In: International conference on image processing (ICIP), vol 3, pp 444–447
Xie J, Chen E (2012) Image denoising and inpainting with deep neural networks. In: Conference on neural information processing systems (NIPS)
Yue H, Sun X, Yang J, Wu F (2015) Image denoising by exploring external and internal correlations. IEEE Trans Image Process 24(6)
Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155
Zontak M, Irani M (2011) The internal statistics of a single natural image. In: IEEE conference on computer vision and pattern recognition (CVPR)
Zontak M, Mosseri I, Irani M (2013) Sperating signal from noise using patch recurrence across scales. In: IEEE Conference on computer vision and pattern recognition (CVPR)
Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: IEEE international conference on computer vision (ICCV)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Zontak, M., Irani, M. (2018). Internal Versus External Denoising—Benefits and Bounds. In: Bertalmío, M. (eds) Denoising of Photographic Images and Video. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-96029-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-96029-6_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-96028-9
Online ISBN: 978-3-319-96029-6
eBook Packages: Computer ScienceComputer Science (R0)