Abstract
Image denoising faces significant challenges, arising from the sources of noise. Specifically, Gaussian, impulse, salt, pepper, and speckle noise are complicated sources of noise in imaging. Convolutional neural network (CNN) has increasingly received attention in image denoising task. Several CNN methods for denoising images have been studied. These methods used different datasets for evaluation. In this paper, we offer an elaborate study on different CNN techniques used in image denoising. Different CNN methods for image denoising were categorized and analyzed. Popular datasets used for evaluating CNN image denoising methods were investigated. Several CNN image denoising papers were selected for review and analysis. Motivations and principles of CNN methods were outlined. Some state-of-the-arts CNN image denoising methods were depicted in graphical forms, while other methods were elaborately explained. We proposed a review of image denoising with CNN. Previous and recent papers on image denoising with CNN were selected. Potential challenges and directions for future research were equally fully explicated.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
In the last decade, the utilization of images has grown tremendously. Images are corrupted with noise in the process of acquisition, compression, and transmission. Environmental, transmission, and other channels are mediums through which images are corrupted by noise. In image processing, image noise is the variation in signal (in random form) that affects the brightness or color of image observation and information extraction. Noise adversely affects image processing tasks (such as video processing, image analysis, and segmentation) resulting in wrong diagnosis [1]. Hence, image denoising is a fundamental aspect which strengthens the understanding of image processing task.
Due to the increasing generation of digital images captured in poor conditions, image denoising methods have become an imperative tool for computer-aided analysis. Nowadays, the process of restoring information from noisy images to obtain a clean image is a problem of urgent importance. Image denoising procedures remove noise and restore a clean image. A major problem in image denoising is how to distinguish between noise, edge, and texture (since they all have high-frequency components). Interestingly, the most discussed noise in literature is the: additive white Gaussian noise (AWGN) [2], impulse noise [3], quantization noise [4], Poisson noise [5], and speckle noise [6]. AWGN occurs in analog circuitry, while impulse, speckle, Poisson, and quantization noise occur due to faulty manufacturing, bit error, and inadequate photon count [7]. Image denoising methods are used in the field of medical imaging, remote sensing, military surveillance, biometrics and forensics, industrial and agricultural automation, and in the recognition of individuals. In medical and biomedical imaging, denoising algorithms are fundamental pre-processing steps used to remove medical noise such as speckle, Rician, Quantum, and others [8, 9]. In remotes sensing, denoising algorithms are used to remove salt and pepper, and additive white Gaussian noise [10, 11]. Synthetic aperture radar (SAR) images provide space and airborne operation in military surveillance [12]. Image denoising algorithms have helped to reduce speckle in SAR images [13]. Moreover, forensic images do not have a specific kind of noise, they could be corrupted by any kind of noise. This noise can reduce the quality of evidence in the image thus, image denoising methods have helped suppress noise in forensic images [14]. Image denoising methods were used to filter paddy leaf and detect rice plant disease. Undoubtedly, image denoising is a hot area of research, encompassing all spheres of academic endeavor.
The linear, non-linear and non-adaptive filters were the first filters used for image applications [15]. Noise reduction filters are categorized into six (linear, non-linear, adaptive, wavelet-based, partial differential equation (PDE), and total variation filters). Linear filters appropriate output pixels with input neighboring pixels (using a matrix multiplication procedure) to reduce noise. Non-linear filters preserve edge information and still suppress noise. In most filtering applications, the non-linear filter is used in place of the linear filter. Linear filter does not preserve edge information; hence, it is considered a poor filtering method. A simple example of a non-linear filter is the median filter (MF) [16]. Adaptive filters employ statistical components for real-time applications (least mean square [17] and recursive mean square [18] are examples). Wavelets-based filters transform images to the wavelet domain and are used to reduce additive noise [19, 20]. A detailed review of different denoising filters is available in reference [21, 22].
Most of the above-mentioned filters have produced reasonably good results, however, they have some drawbacks. These drawbacks include poor test phase optimization, manual parameter settings, and specific denoising models. Fortunately, the flexibility of convolutional neural networks (CNN) has shown the ability to solve these drawbacks [23]. CNN algorithms have shown a strong ability to solve many problems [24]. For example, CNN has achieved excellent results in image recognition [25], robotics [26], self-driving [27], facial expression [28], natural language processing [29], handwritten digital recognition [30] and so many other areas. Chiang and Sullivan [31] were the first to use CNN (deep learning) for image denoising tasks. A neural network (weighting factor) was used to remove complex noise, then a feedforward network [32] produced a balance between efficiency and performance of the denoised image. In the early developments of CNN, the vanishing gradient, activation function (sigmoid [33] and Tanh [34]), and unsupported hardware platform made CNN difficult. However, the development of AlexNet [35] in 2012 has changed the difficulty in CNN usage. More CNN architecture (such as; VGG [36] and GoogleNet [37]) have been applied to computer vision tasks. References [38, 39] were the first CNN architecture used in image denoising tasks. Zhang et al. [40] used the denoising CNN (DnCNN) for image denoising, super-resolution, and JPEG image blocking. The network consists of convolutions, back-normalization [41], rectified linear unit (ReLU) [42] and residual learning [43].
The use of CNN is not limited to general image denoising alone, CNN produced excellent results for blind denoising [44], real noisy images [45], and many others. Although several researchers have developed CNN methods for image denoising, only a few have proposed a review to summarize methods. Reference [46] summarized CNN methods for image denoising with categories based on noise type. Although this review is elaborate; it does not consider several methods for specific images. Again, the research did not consider recent methods (the year 2020 methods); hence, several research works published in late 2020 were unintentionally omitted. Our review provides an overview of CNN image denoising methods for different kinds of noise (including specific image noise). We discuss state-of-the-arts methods with emphases on image type and noise specification. The outline of CNN image denoising methods is depicted in Fig. 1. It is hoped that explanations in this study will provide an understanding of CNN architectures used in image denoising. Our contribution is summarized as follows:
-
1.
Analysis of different CNN image denoising models, database, and image type.
-
2.
The highlight of commonly used objective evaluation methods in CNN image denoising
-
3.
Potential challenges and road maps in CNN image denoising.
The rest of the paper is organized as follows. In Sect. 2, we review different CNN image denoising methods. In Sect. 3, we review databases for CNN image denoising algorithms. Section 4 gives an analysis of CNN image denoising; finally, the paper is concluded in Sect. 5.
Literature review
In this section, several existing methods for CNN image denoising will be discussed. We divide CNN image denoising approaches into two: (1) CNN denoising for general images, and (2) CNN denoising for specific images. The first approach uses CNN architectures to denoising general images, while the second approach uses CNN to denoise specific images. The first approach is widely used in CNN denoising applications when compared to the second. General images refer to images that represent a general purpose rather than the details (See [47] for samples of general images). Specific images are images intentionally created with a special or particular kind. For example, medical images, infrared images, remote sensing images, and others are kinds of specific images. The reason for dividing CNN denoising by image category is to bring readers up to speed with the latest CNN architecture with regards to image types. A block diagram depicting different approaches is shown in Fig. 1.
CNN denoising for general images
Reference [48] proposed the attention-guided CNN (ADNet) for image denoising. ADNet consists of 17 layers with 4 blocks (sparse block (SB), feature enhancement block (FEB), attention block (AB), and reconstruction block (RB)). The use of sparsity has shown to be effective for image application [49]; hence, the SB was used to improve efficiency, performance, and to reduce the depth of the denoising framework. The SB has 12 layers with two types (dilated Conv + BN + ReLU, and Conv + BN + ReLU). The FEB has 4 layers with 3 types (Conv + BN + ReLU, Conv, and Tanh), while the AB has a single convolution layer. The AB was used to guide the SB and FEB which are useful for unknown noise. Finally, the RB performs reconstruction to produce a clean image. For training, the mean square error [50] was used to create model training (see Fig. 2).
Some deep learning algorithm produces excellent results with synthetic noise; however, most of this network do not achieve good results in image corrupted by realistic noise. The research by Guo et al. [51] proposed the noise estimation removal network (NERNet). NERNet reduced noise on images with realistic noise. The architecture was divided into two modules; the noise estimation module and the noise removal module. The noise estimation module appropriates the noise-level map with the symmetric dilated block [52, 53] and the pyramid feature fusion [54]. Meanwhile, the removal module used the estimated noise-level map to remove noise. The global and local information for preserving details and texture were aggregated into the removal module. The output of the noise estimate module was passed into the removal module to produce clean images.
It is no gainsaying that CNN learns noise patterns and image patches effectively. However, this learning produces a network with a large amount of training data and image patches. Because of the aforementioned, reference [55] proposed the patch complexity local divide and deep conquer network (PCLDCNet). The network was divided into local subtask (according to clean image patch and conquer block) and was trained on its local space. Each noisy patch weighting mixture was combined with the local subtask. Finally, image patches were grouped by complexity [56], while the training of the k network was achieved by the modified stacked denoising autoencoders [57]. Network degradation is another problem in a deep learning network (the deeper the layer the higher the error rate). Although the introduction of ResNet [58] resolved this issue, there is still room for improvement. Shi et al. [59] proposed the hierarchical residual learning that does not require the identity mapping for image denoising. The network has 3 sub-networks: feature extraction, inference, and fusion. Feature extraction sub-network extracts patches representing higher dimension feature maps. The interference sub-network [60] contains cascaded convolutions that produce a large receptive field. The cascaded procedure was performed to learn noise maps from multiscale information and to produce tolerating errors in noise estimation. Finally, the fusion sub-network fuses the entire noise map to produce estimation.
Gai and Bao [61], used the improved CNN (MP-DCNN) for image denoising. MP-DCNN is an end-to-end adaptive residual CNN constructed for modeling noisy images. Noise from the input image was extracted by the leaky ReLU, and the image features were reconstructed. An initial denoised image was inputted into the SegNet to obtain edge information. The MSE and the perceptive loss function [62] were used to obtain the final denoised image (see Fig. 3).
Another research by reference [63] proposed a new dictionary learning model for a mixture of Gaussian (MOG) distribution. The method was used for the expectation–maximization framework [64]. A minimization problem that uses the sparse coding and dictionary updating with quantitative and visual comparison was adopted. Specifically, this method was used to learn hierarchical mapping functions, and to prevent vanishing problems, Zhang et al. [65] proposed the separation aggregation network (SANet). SANet used three blocks (convolutional separation block, deep mapping block, and band aggregation block) to remove noise. The convolution separation block decomposed the input noise into sub-blocks [66, 67]. Then, each band was mapped into a clean and latent form using the convolution and ReLU layers. Finally, the band aggregation block concatenates all maps and convolutes features to produce the output. The SANet model was inspired by the non-local patch (NPL) model [67]. NPL model consists of patch grouping, transformation, and patch aggregation. Residual images obtained by learning the difference between noisy and clean image pairs produce loss of information. This information is important in producing an effective noise-free output image. Reference [68] proposed the detail retaining CNN (DRCNN) to navigate between noisy and clean pairs without losing information. The model (DRCNN) focused on the integrity of high-frequency image content and produces better generalization ability. A minimization problem was analyzed, designed, and solved from the detail loss function. DRCNN has two modules: the generalization module (GM), and the detail retaining module (DRM). GM involves convolution layers with a stride of 1, while DRM involves several convolution layers. Unlike several architectures, DRCNN does not have BN.
Computation cost is an emerging problem in CNN applications, a very large network always occupies a large memory space and requires high computational capacity. These networks are unsuitable for applications on smart and portable devices. Because of the above problem, Yin et al. [69] proposed a side window CNN (SW-CNN) for image filtering. SW-CNN has two parts: side kernel convolution (SKC), fusion, and regression (FR). SKC aligns slide or corner of operation window with the target pixel to preserve edges. SKC was combined with CNN to provide effective representation power. A residual learning strategy [70] was adopted to map layers. FR involves two convolutional phases consisting of three operations: pattern expression, non-linear mapping, and weight calculations. The pattern expression calculates the gradient from the feature map tensor to produce a pattern tensor. Non-linear mapping convolutes the pattern tensor with different kernels to produce a tensor with (Hxwxd) dimension. Finally, the weight calculations generated the weighting coefficient of each pixel.
Single noise reduction with CNN is a difficult task. A more difficult task is to remove mixed noise from an image using CNN. Most mixed noise removal algorithms involve pre-processing outlier. Reference [71] proposed the denoising-based generative adversarial network (DeGAN) for removing mixed noise from images. The generative adversarial network (GAN) [72] has been widely used in deep learning applications. The DeGAN involved the generator, discriminator, and feature extractor network. The generator network used the U-Net [73] architecture, while the discriminator network consists of 10 end-to-end design layers. The main purpose of the discriminator network was to check whether the image estimated by U-Net (extractor network) was noise free. Finally, the feature extraction network used the VGG19 [74] to extract features and to assist the model training by calculating the loss function (see Fig. 4).
Xu et al. [75] proposed the Bayesian deep matrix factorization (BDMF) for multiple image denoising. BDMF used the deep neural network (DNN) for low-rank components and optimization via stochastic gradient variation Bayes [76,77,78]. The network is a combination of the deep matrix factorization (DMF) network and the Bayesian method. Synthetic and hyperspectral images were used to evaluate the methods. Reference [79] proposed the classifier/regression CNN for image denoising. The regression network was used for restoring the noisy pixel identified by the classifier network, while the classifier network detects impulse noise. The classifier network involves convolution, BN, ReLU, a softmax, and a skip connection. Meanwhile, based on the label predicted by the classifier network, the regression network used four layers and a skip connection to predict clean images (see Fig. 5).
Reference [80] proposed the complex-valued CNN (CDNet) for image denoising. First, the input image was passed to 24 sequential connected convolutional units (SCCU). SCCU involve the complex-valued (CV) convolutional layer, complex-valued (CV) ReLU, and complex-valued (CV) BN. A 64 convolutional kernel was used in the network. The residual block was implemented for the middle 18 units. The convolution/deconvolution layer with a stride of 2 was used to improve computational efficiency. Finally, the merging layer transformed the complex-valued features into a real-value image. Overall, CDNet has five blocks: complex-valued (CV) Conv, complex-valued (CV) ReLU, complex-valued (CV) BN, complex-valued (CV) residual block (RB), and merging layer (see Fig. 6).
Zhang et al. [81] proposed the detection and reconstruction of CNN for color images (3-channel). The method has three networks; classifier network, denoiser network, and reconstruction network. The classifier network predicts color channels to determine the probability of impulse noise in the image. Decision-maker procedures (that compute the label vector of each pixel) were employed to ascertain noisy or noise-free color pixels. Sparse clean image replaced corrupted channels (0 for noise free). Finally, the denoised image was reconstructed by the image reconstruction architecture. In a nutshell, the classifier network (consist of convolution, and ReLU layers) predicts the probability of channels, then the denoiser network (consist of convolution, BN, and ReLU layers) corresponds to the noise-free color pixel, while the reconstruction network (has only convolutions) reconstructs the images. Although the networks have the same structures, the depth and the number of nodes are different. Adaptive moment estimation (Adam) [82] was used to optimize the networks.
Reference [83] proposed the CNN variation model (CNN-VM) for denoising of images. The CNN used in this research was termed EdgeNet and it consists of multiple scale residual blocks (MSRB). EdgeNet extracts feature from the noisy image through an edge regularization method. The total variation regularization was used to obtain superior performance in the shape edge. The Bregman splitting method was used to obtain solutions to the model. MSRB employed a kernel of two for each bypass to detect local features. A skip connection was used for inputting data and to generate output features. Another skip connection was used after each MSRB block with a bottleneck layer fusing detected features. Four MSRB blocks were adopted in the EdgeNet training procedure. A comparison of different methods in this section is available in Tables 1 and 2.
CNN denoising for specific images
Islam et al. [84] proposed a feedforward CNN method to remove mixed noise (Gaussian–impulse). The method adopts a computational efficiency transfer learning approach for noise removal. The model consists of a pre-processing, and four convolution filtering stages. A rank order filtering operation was applied to each stage and the convolution layer preceded the ReLU and max-pooling layers. The output of the first stage was fed into the ReLU and the output of the ReLU was pooled (max pooling). The second and third stages used convolution and ReLU layers, and the last stage adopts the convolution layer. A back-propagation algorithm (with differentiable and traceable loss function) was used to train the model. Finally, the model used a data argumentation [85, 86] for effective learning. Another research by Tian et al. [87] proposed a deep learning method based on U-Net [73] and Noise2Noise [88] method. First, the noise was validated on computer-generated holography (CGH) images. Then, the classical Gerchberg–Saxton (GS) algorithm [89] was used to generate different holograms (two-phase). Next, the noise reduction mechanism (UNET and Noise2Noise) was obtained. Finally, the MSE was used as the loss function and the learning rate was set at 0.001. Like the previous method, the MSE was adopted as the loss function; apparently, MSE can act as a good loss function in image denoising.
Reference [90] proposed the spectral–spatial denoising residual network (SSDRN). SSDRN used the spectral difference [91] mapping method based on CNN with residual learning for denoising. The network was an end-to-end algorithm that preserves spectral profile and removes noise. A key band was selected based on a principal transform matrix with DnCNN [40]. Overall, SSDRN involves three parts: spectral difference learning, key band selection, and denoising (by DnCNN) model. Unlike most CNN denoising models, SSDRN used the batch normalization [92] layer in each block of the algorithm. Reference [93] proposed the patch group deep learning for image denoising. A training set with a patch group was created and then the deep learning method [94, 95] was used to reduce the noise. Reference [96] developed an end-to-end deep neural network (DDANet) for computational ghost image reconstruction. DDANet used a bucket signal with multiple tunable noise-level maps. A clear image was outputted after training with the simulated bucket signals and the ground-truth image. DDANet has 21 layers that include: fully connected layers, dense blocks, and convolution layers. The inputs, transformation, noise adding [97] encoding [98], and object recovery layers were used in the DDANet architecture. A skip connection [99, 100] for passing high-frequency feature information was utilized. The attention gate (AG) [101] and dilated convolution were used to filter the features. Finally, the dropout layer [102] was used to avoid overfitting, while the BN accelerated loss function.
Zhang et al. [103] proposed the deep spatio-spectral Bayesian posterior network (DSSBPNet) for hyperspectral images. A blend of Bayesian variation posterior and deep neural network produced the DSSBPNet. Specifically, the method was divided into two parts: deep spatio-spectral (DSS) network and Bayesian posterior. The DSS network split the input image into three parts producing a spatio-spectral gradient [104] for each part. Different convolutions were used in the DSS network. Meanwhile, the likelihood of original data, noise estimate, noise distribution, and sparse noise gradient constitute the Bayesian posterior method. Finally, a forward–backward propagation method was used to connect the DSS with the Bayesian posterior. Reference [105] proposed the two-stage cascaded residual CNN to remove mixed noise from infrared images. The model used the mixed convolutional layer combining dilated convolutions, sub-pixel convolutions, and standard convolutions to extract and improve accuracy. A residual learning method was used to estimate the calibration parameter from the input image. Five feature extraction blocks (FEBs) used the coarse–fine convolution unit (CF-Conv) and the spatial-channel noise attention unit (SCNAU) to stack noise features. The last convolution layer for each network consists of a single filter with kernel size (see Fig. 7). Giannatou et al. [106] proposed the residual learning CNN (SEMD) for noise removal in scanning electron microscopic images. SEMD is a residual learning method inspired by the DnCNN and trained to estimate the noise at each pixel of a noisy image. The input block in SEMD consists of a convolutional layer followed by a ReLU and BN. The output block consists of a convolution with one filter for reconstruction. Jiang et al. [107] proposed the generative adversarial network based on the deep network for denoising of underwater images (UDnNet). UDnNet consists of two sub-networks: a generator network, and a discriminator network. The generator network generates realistic samples using the training procedure, asymmetric codec structure, and a skip connection. The output of the generative network was processed by the convolution-instance Norm-Leaky ReLU. A deconvolution-instance Norm-Leaky ReLU decodes the features.
Reference [108] combined the bilateral filter, the hybrid optimization, and the CNN to remove noise. The bilateral filter [100, 109] was used to remove noise, while the hybrid optimization used the swarm insight strategy [110] to preserve edges. Finally, a CNN classifier (with convolution layers, pooling layer with feature extraction, and fully connected layer) was used to classify the image. For the evaluation procedure, the peak signal to noise ratio, vector root mean square error, structural similarity index, and root mean square error was adopted [8, 111]. A major challenge when using CNN for speckle reduction is labeling. Ultrasound images are not labeled; hence, it becomes very difficult for deep learning to identify speckle. Feng et al. [112] proposed a hybrid CNN method for speckle reduction. The method involves a three knowledge system. Since speckle noise was similar to Gaussian distribution in the logarithm transform domain, the distribution parameters were also estimated in the logarithm transformation domain with maximum likelihood estimation. Second, a transfer denoising network was trained with a clean natural image dataset. Finally, the VGGNet was used to extract structural boundaries from the trained images. Overall, the transferable denoising network was trained based on Gaussian prior knowledge of Ultrasound clean images. Then, fine-tuning of the pre-trained network with prior knowledge of structural boundaries was performed. Ultrasound images (breast, liver, and spinal) and artificially generated phantom (AGP) images were used to evaluate the method.
Reference [113] used the pre-trained residual learning network (RLN) for despeckling of ultrasound images. The model consists of noise and pre-trained RLN models. A noise model was created from the training dataset, and then random patches were generated from the speckle noise images. The RLN was then used to train the random patches, and a despeckled image was created. The pre-trained RLN has 59 layers (consist of Conv, ReLU, and BN) for training and testing. The method was tested with artificial and natural images corrupted with speckle noise (see Fig. 8).
Kim and Lee [114] proposed the conditional generative adversarial network (CGAN) for noise reduction in low-dose chest images. CGAN involves the generative model [115], discriminator model [116], and the prediction model. The generator model has 14 layers and focused on synthesized realistic images from random vector sample noise distribution. Meanwhile, the discriminator model has 4 layers and trains on ground-truth images. The tensor library was used to accomplish the CGAN architecture. Li et al. [117] proposed a progressive network learning strategy (PNLS) that fits the Racian distribution with large convolutional filters. The network consists of two residual blocks (used for fitting pixel domain, and matching pixel domains). The first residual block used the Conv, and ReLU layers without the BN layer. The second residual block used the Conv, ReLU, and BN layers. Each block has 5 layers with three convolution layers acting as the intermediary between blocks (see Fig. 9).
Reference [118] proposed a novel CNN method for denoising MRI scans (CNN-DMRI). The network used convolutions to separate image features from noise. CNN-DMRI is an encoder–decoder structure for preserving important features and ignoring the unimportant ones. The neural network learns prior features from the image domain and produced clean images. A down-sampling and up-sampling factor of 2 was adopted. CNN-DMRI is a four-layer network; the first two layers have 64 filters followed by CONV layers. The down-sampling layer has 128 filters followed by 4 residual blocks and a 64 up-sampling filter. Finally, a concatenation of the noisy image and the network was performed to produce a clean MRI. Comparison of different methods in this section is available in Tables 3 and 4.
CNN image denoising performance measures
Performance evaluations are key indices in image denoising. Over the years, researchers have used different objective evaluation methods for CNN image denoising. Below are different evaluation methods adopted by researchers in CNN denoising.
The mean square error (MSE): Is the average of the square of the difference between the original image and the denoised image. Lower MSE values signify better image quality.
Peak signal to noise ratio (PSNR): is determined through the MSE. It is an engineering term that measures the ratio between maximum original signal and MSE. Higher PSNR values signify better image quality.
Structural similarity index measure (SSIM): measure perceptual difference (such as luminance, contrast, and structure) of two similar images. Higher SSIM values signify better image quality.
where \(\mu _{I}\) and \(\mu _{L}\) are the average gray values, \(\sigma _{I}\) and \(\sigma _{L}\) are the variance of patches, \(\sigma _{{IL}}\) is the covariance of I and L, and Q1 and Q2 denote two small positive constants (typically 0.01).
Root mean square error (RMSE): measure the difference between estimated predictions and actual observed values. The MSE is the scale square of RMSE. The RMSE between two image metrics (P, Q) is:
Feature Similarity (FSIM and FSIMc): is designed for gray-scale images and luminance components of color images. It computes local similarity maps and pools these maps into a single similarity score.
To learn more about the FSIM and FSIMc see reference [119].
The signal to noise ratio (SNR): measures noise level relative to the original image as follows.
The Spectral Angle mapper (SAM), and the Erreur Relative Globale Adimensionnelle de Synthèse (ERGAS) [120] are used with other evaluation methods in remote sensing images. Overall, the PSNR and the SSIM are the most widely used evaluation methods for CNN denoising. These two methods are popular because they are easy and are considered tested and valid [121].
Datasets
This section provides a list of datasets used for CNN image denoising algorithms. They include: ImageNet large scale visual recognition challenge object detection (ILSVRC-DET) [122], Places2 [123], BerkeleySegmentation Dataset (BSD) [124], waterloo explorationdatabase [125], EMNIST [126], COCO dataset [127], MIT-Adobe five k [128], ImageNet [129], BSD68 [130], Set 14 [131], Renoir [132], NC12 [133], NAM [134], SIDD [135], SUN397 [136], Set5 [137], CAVE [138],Harvard database [139], MRI brain dataset [140], LIVE1, Chelsea,DIV2K dataset [141], HIS dataset Xiongan, first Affiliated Hospital of Sun Yat-Sen University, Shenzen third people’s Hospital, Artificially generated phantom (AGP) [142], Ultrasound dataset [143, 144], SPIE American association of physicist in medicine lung CT-challenge database [145], SIAT-CAS MRI dataset, Brainweb [146, 147], IXI dataset [148], Multiple sclerosis [149], Prostrate MRI [150], ThammasatUniversity Hospital dataset [151]. A few samples of data in the dataset used by researchers for CNN denoising are shown in Fig. 10. Similarly, Fig. 11 is the graph of datasets used for evaluating CNN denoising methods. However, the Berkeley segmentation dataset has the highest usage because it is particularly suited for image denoising research. Three major points that matter when selecting datasets are relevance, usability, and quality [47, 152]. Hence, we believe that datasets with higher usage for CNN denoising tasks have these aforementioned points. It should be noted that some datasets were not recorded in the graph (Fig. 10). The reason for this was because they have a fewer appearance in CNN denoising researchers.
Analysis
A total of 152 references were included in this paper, specifically, 31 research papers related to CNN image denoising. In this review, a conscious effort was made to include all research articles relating to CNN image denoising; however, some studies might have been skipped. A graph depicting the number and the publication year for CNN image denoising methods is available in Fig. 12. From the figure, it is clear that researchers have just recently adopted CNN for image denoising. This is open research for further experimentation and exploration. Finally, the graph of image type adopted by the CNN image denoising methods is available in Fig. 13
Conclusions and future directions
Recently, CNN architectures are becoming quite useful in image denoising. We have proposed a survey of different techniques relating to CNN image denoising. A clear understanding of different concepts and methods was elucidated to give readers a grasp of recent trends. Several techniques for CNN denoising have been enumerated. A total of 144 references were included in this paper. From the study, we observed that the GAN was the most used method for CNN image denoising. Several methods used the generator and the discriminator for extraction and clean image generation. Interestingly, some researchers combined the GAN method with the DCNN methods. The feedforward CNN and U-Net were also used. The residual network was used severally by researchers. A reason for the high usage of the residual network could be its effectiveness and efficiency. Researchers used the residual network to limit the numbers of convolutions in their network. A creative measure adopted by the researcher was to try to mixed noise (impulse Gaussian noise). To reduce mixed noise in images, several careful deep convolutions were required. The Rician and speckle noise is common in medical images. Pre-trained networks have worked excellently in medical image noise reduction. The database from Berkeley was the most used in CNN image denoising. In addition, the attention mechanism and residual networks are commonly used CNN techniques in image denoising tasks. The reason for such wide acceptance is because of their popularity and effectiveness in image denoising.
Some problems confronted by CNN image denoising methods include not enough memory for CNN applications, and difficulty in solving unsupervised denoising tasks. Conclusively, only very few CNN methods were used for medical images. It will be encouraging if more CNN methods could be applied to denoise medical images. In addition, the authors try to collect codes and software; however, it was not available. The provision of more memory allocations for the CNN task will be very helpful. This could be a research area for future discussion.
The findings of the review can be summarized below:
-
From the available literature, it is clear that CNN can considerably remove all kinds of noise from images and advance capability in image denoising. Several studies reported higher performance of CNN architecture for image denoising. CNN architectures support end-to-end procedures and are implemented promptly.
-
CNN architecture can be customized for noise removal tasks creating patterns that remove the bottleneck of vanishing gradients.
-
CNN methods are designed using technical knowledge and principles in concert with understanding the noise type and noise models.
-
Most studies used pre-trained CNN models; however, noise properties are in a continuous nature and need a model built from scratch. Building such a model creates room for readjustment and fine-tuning. However, building a model from the beginning require lots of computation space and time. With the introduction of cloud-based methods (e.g., COLAB), it is hoped that the problem of space and time would be resolved.
-
The use of spatial patterns in CNN architecture could create a shift from conventional methods to deep learning methods. Contrary to perceptions that CNN is a black box, features visualization methods provide a trusted platform for noise removal, however, the greatest challenge remains the computational time and space.
References
Diwakar M, Kumar M (2018) A review on CT image noise and its denoising. Biomed Signal Process Control 42:73–88. https://doi.org/10.1016/j.bspc.2018.01.010
Buades A, Coll B, Morel JM (2005) A review of image denoising algorithms, with a new one. Multiscale Model Simul 4(2):490–530
Awad A (2019) Denoising images corrupted with impulse, Gaussian, or a mixture of impulse and Gaussian noise, Engineering Science and Technology, an. Int J 22(3):746–753
Bingo W-KL, Charlotte YFH, Qingyun D, Reiss JD (2014) Reduction of quantization noise via periodic code for oversampled input signals and the corresponding optimal code design. Digit Signal Process 24:209–222
Rajagopal A, Hamilton RB, Scalzo F (2016) Noise reduction in intracranial pressure signal using causal shape manifolds. Biomed Signal Process Control 28:19–26
Ilesanmi AE, Idowu OP, Chaumrattanakul U, Makhanov SS (2021) Multiscale hybrid algorithm for pre-processing of ultrasound images. Biomed Signal Process Control 66:102396
Goyal B, Dogra A, Agrawal S, Sohi BS, Sharma A (2020) Image denoising review: from classical to state-of-the-art approaches. Inform Fusion 55:220–244
Gai S, Zhang B, Yang C, Lei Yu (2018) Speckle noise reduction in medical ultrasound image using monogenic wavelet and Laplace mixture distribution. Digital Signal Process 72:192–207
Baselice F, Ferraioli G, Pascazio V, Sorriso A (2019) Denoising of MR images using Kolmogorov–Smirnov distance in a non local framework. Magn Reson Imaging 57:176–193
Vijay M, Devi LS (2012) Speckle noise reduction in satellite images using spatially adaptive wavelet thresholding. Int J Comput Sci Inf Technol 3:3432–3435
Bhosale NP, Manza RR (2013) Analysis of effect of noise removal filters on noisy remote sensing images. Int J Sci Eng Res 4:1151
Berens P. Introduction to synthetic aperture radar (SAR). NATO OTAN, pp 1–14
Sivaranjani R, Mohamed-Mansoor-Roomi S, Senthilarasi M (2019) Speckle noise removal in SAR images using multi-objective PSO (MOPSO) algorithm. Appl Soft Comput 76:671–681
Aljarf A, Amin S (2015) Filtering and reconstruction system for gray forensic images. World Acad Sci Eng Technol Int J Inform Commun Eng 9(1)
Huang T (1971) Stability of two-dimensional recursive filters (mathematical model for stability problem in two-dimensional recursive filtering)
Jaspin-Jeba-Sheela C, Suganthi G (2020) An efficient denoising of impulse noise from MRI using adaptive switching modified decision based unsymmetric trimmed median filter. Biomed Signal Process Control 55:101657
Zhao H, Zheng Z (2016) Bias-compensated affine-projection-like algorithms with noisy input. Electron Lett 52(9):712–714
Dinga F, Wanga Y, Ding J (2015) Recursive least squares parameter identification algorithms for systems with colored noise using the filtering technique and the auxilary model. Digit Signal Process 37:100–108
Stolojescu-Crisan C (2015) A hyperanalytic wavelet based denoising technique for ultrasound images. In: International Conference on Bioinformatics and Biomedical Engineering, pp 193–200
Zhang X, Feng X (2014) Multiple-step local wiener filter with proper stopping in wavelet domain. J Vis Commun Image Represent 25(2):254–262
MohdSagheer SV, George SN (2020) A review on medical image denoising algorithms. Biomed Signal Process Control 61:102036
Fan L, Zhang F, Fan H et al (2019) Brief review of image denoising techniques. Vis Comput Ind Biomed Art 2:7. https://doi.org/10.1186/s42492-019-0016-7
Lucas A, Iliadis M, Molina R, Katsaggelos AK (2018) Using deep neural networks for inverse problems in imaging: beyond analytical methods. IEEESignal Process Mag 35(1):20–36
Khan A, Sohail A, Zahoora U et al (2020) A survey of the recent architectures of deep convolutional neural networks. Artif Intell Rev 53:5455–5516
Zhu P, Isaacs J, Fu B et al. (2017) Deep learning feature extraction for target recognition and classification in underwater sonar images. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC), IEEE, 2017
Paolini R, Rodriguez A, Srinivasa SS et al (2014) A data-driven statistical framework for post-grasp manipulation. Int J Robot Res 33(4):600–615
Ramos S, Gehrig S, Pinggera P et al (2017) Detecting unexpected obstacles for self driving cars: Fusing deep learning and geometric modeling. In: 2017 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2017
Wu H, Liu Y, Liu Y, Liu S (2019) Efficient facial expression recognition via convolution neural network and infrared imaging technology. Infrared Phys Technol 102:103031
Firmansyah I, Yamaguchi Y (2020) FPGA-based implementation of a chirp signal generator using an OpenCL design. Microprocess Microsyst 77:103199
LeCun Y, Bottou L, Bengio Y, Haffner P et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Chiang Y, Sullivan BJ (1989) Multi-frame image restoration using a neural network. In: Proceedings of the 32nd midwest symposium on circuits and systems, IEEE, pp 744–747
Hu J, Wang X, Shao F, Jiang Q (2020) TSPR: deep network-based blind image quality assessment using two-side pseudo reference images. Digit Signal Process 106:102849
Marreiros AC, Daunizeau J, Kiebel SJ, Friston KJ (2008) Population dynamics: variance and the sigmoid activation function. Neuroimage 42(1):147–157
Jarrett K, Kavukcuoglu K, Ranzato M, LeCun Y (2009) What is the best multi-stage architecture for object recognition. In: 2009 IEEE 12th international conference on computer vision, pp 2146–2153
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural information Processing Systems, pp 1097–1105
Ha I, Kim HJ, Park S, Kim H (2018) Image retrieval using BIM and features from pretrained VGG network for indoor localization. Build Environ 140:23–31
Tang P, Wang H, Kwong S (2017) G-MS2F: GoogLeNet based multi-stage feature fusion of deep CNN for scene recognition. Neurocomputing 225:188–197
Liang J, Liu R (2015) Stacked denoising autoencoder and dropout together to prevent overfitting in deep neural network, In: 2015 8th international congress on image and signal processing (CISP), pp 697–701
Xu J, Zhang L, Zuo W, Zhang D, Feng X (2015) Patch group based nonlocal self-similarity prior learning for image denoising. In: Proceedings of the IEEE international conference on computer vision, pp 244–252
Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a Gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155
Li Y, Wang N, Shi J, Hou X, Liu J (2018) Adaptive batch normalization for practical domain adaptation. Pattern Recogn 80:109–117
Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814
Zhang M, Yang L, Yu D, An J (2021) Synthetic aperture radar image despeckling with a residual learning of convolutional neural network. Optik 228:165876
Zhang F, Liu D, Wang X, Chen W, Wang W (2018) Random noise attenuation method for seismic data based on deep residual networks. In: International geophysical conference, Beijing, China, 24–27, 2018, Society of Exploration Geophysicists and Chinese Petroleum Society, pp 1774–1777
Guo Z, Sun Y, Jian M, Zhang X (2018) Deep residual network with sparse feedback for image restoration. Appl Sci 8(12):2417
ChunweiTian LunkeFei, WenxianZheng YX, WangmengZuo C-W (2020) Deep learning on image denoising: an overview. Neural Netw 131:251–275
Koesten L, Simperl E, Blount T, Kacprzak E, Tennison J (2020) Everything you always wanted to know about a dataset: studies in data summarization. Int J Human-Comp Stud 135:102367
Schwenker F, Kestler HA, Palm G (2001) Three learning phases for radial-basis-function networks. Neural Netw 14(4):439–458
Tian C, Zhang Q, Sun G, Song Z, Li S (2018) FFT consolidated sparse and collaborative representation for image classification. Arab J Sci Eng 43(2):741–758
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition, ArXiv preprint arXiv: 1409.1556
Guo B, Song K, Dong H, Yan Y, Tu Z, Zhu L (2020) NERNet: noise estimation and removal network for image denoising. J Vis Commun Image R 71:102851
Wei Y et al (2018) Revisiting dilated convolution: a simple approach for weakly-and semi-supervised semantic segmentation. IEEE Conf Comp Vis Pattern Recogn (Cvpr) 2018:7268–7277
Li X et al (2019) Selective kernel networks. IEEE Conf Comp Vis Pattern Recogn (Cvpr) 2019:510–519
He KM et al (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Computer Vision - Eccv 2014, Pt Iii,. 8691: pp 346–361
Hong I, Hwang Y, Kim D (2019) Efficient deep learning of image denoising using patch complexity local divide and deep conquer. Pattern Recogn 96:106945
Chatterjee P, Milanfar P (2011) Practical bounds on image denoising: from estimation to information. IEEE Trans Image Process 20(5):1221–1233
Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P-A (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778
Shi W, Jiang F, Zhang S, Wang R, Zhao D, Zhou H (2019) Hierarchical residual learning for image denoising. Signal Process Image Commun 76:243–251
Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1646–1654
Gai S, Bao Z (2019) New image denoising algorithm via improved deep convolutional neural network with perceptive loss. Expert Syst Appl 138:112815
Wu CZ, Chen X, Ji D, Zhan S (2018) Image denoising via residual network based on perceptual 1oss. J Image Graphics 23(10):1483–1491
Zhang J, Luo H, Hui B, Chang Z (2019) Unknown noise removal via sparse representation model. ISA Trans 94:135–143
Liu J, Tai X, Huang H, Huan Z (2013) A weighted dictionary learning models for denoising images corrupted by mixed noise. IEEE Trans Image Process 22(3):1108–1120
Zhang L, Li Y, Wang P, Wei W, Xu S, Zhang Y (2019) A separation–aggregation network for image denoising. Appl Soft Comp J 83:105603
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2921–2929
Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2862–2869
Li X, Xiao J, Zhou Y, Ye Y, Lv N, Wang X, Wang S, Gao S (2020) Detail retaining convolutional neural network for image denoising. J Vis Commun Image R 71:102774
Yin H, Gong Y, Qiu G (2020) Fast and efficient implementation of image filtering using a side window convolutional neural network. Signal Process 176:107717
Zhang K, Zou W, Chen Y, Meng D, Zhang L (2017) Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans Image Process 26(7):3142–3155
Lyu Q, Guo M, Pei Z (2020) DeGAN: mixed noise removal via generative adversarial networks. Appl Soft Comp J 95:106478
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley B, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceedings of the 28th Annual Conference on Neural Information Proceeding System, pp 2672–2680
Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 234–241
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. ArXiv preprint arXiv: 1409.1556.
Xu S, Zhang C, Zhang J (2020) Bayesian deep matrix factorization network for multiple images denoising. Neural Netw 123:420–428
Blei DM, Kucukelbir A, McAuliffe JD (2017) Variational inference: a review for statisticians. J Am Stat Assoc 112(518):859–877
Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D (2015) Weight uncertainty in neural networks. In: ICML, pp 1613–1622
Kingma DP, Welling M (2014) Auto-encoding variational Bayes. In: International conference on learning representations, ICLR Banff, AB, Canada, pp 14–16
Jin L, Zhang W, Ma G, Song E (2019) Learning deep CNNs for impulse noise removal in images. J Vis Commun Image R 62:193–205
Quan Y, Chen Y, Shao Y, Teng H, Xu Y, Ji H (2021) Image denoising using complex-valued deep CNN. Pattern Recogn 111:107639
Zhang W, Jin L, Song E, Xu X (2019) Removal of impulse noise in color images based on convolutional neural network. Appl Soft Comp J 82:105558
Kingma D, Ba J (2015) Adam: a method for stochastic optimization. In: International Conference on Learning Representation
Fang Y, Zeng T (2020) Learning deep edge prior for image denoising. Comp Vis Image Understand 200:103044
Islam MT, Rahman SMM, Ahmad MO, Swamy MNS (2018) Mixed Gaussian-impulse noise reduction from images using convolutional neural network. Signal Process Image Commun 68:26–41
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proc. Int. Conf. Neural Information Processing Systems, Lake Tahoe, NV, pp 1097–1105
Oyelade ON, Ezugwu AE (2021) A deep learning model using data augmentation for detection of architectural distortion in whole and patches of images. Biomed Signal Process Control 65:102366
Yin D, Gu Z, Zhang Y, Gu F, Nie S, Feng S, Ma J, Yuan C (2020) Speckle noise reduction in coherent imaging based on deep learning without clean data. Opt Lasers Eng 133:106151
Lehtinen J, Munkberg J, Hasselgren J, Laine S, Karras T, Aittala M et al (2018) Noise2Noise: learning image restoration without clean data. arXiv: 1803.04189
Gerchberg RW, Saxton WO (1972) A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik (Stuttg) 35(2):237–250
Xi W, Li Y, Jia X (2018) Deep convolutional networks with residual learning for accurate spectral-spatial denoising. Neurocomputing 312:372–381
Li Y, Hu J, Zhao X, Xie W, Li J (2017) Hyperspectral image super-resolution using deep convolutional neural network. Neurocomputing 266:29–42
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the International Conference on Machine Learning (ICML), pp 448–456
Park M, Lee S, Choi S, Lee S, Han S, Lee H, Kang S-H, Lee Y (2020) Deep learning-based noise reduction algorithm using patch group technique in cadmium zinc telluride fusion imaging system: A Monte Carlo simulation study. Opt Int J Light Electron Opt 207:164472
Feng J, Song L, Huo X, Yang X, Zhang W (2015) An optimized pixel-wise weighting approach for patch-based image denoising. IEEE Signal Process Lett 22:115–119
Xu J, Ren D, Zhang L, Zhang D (2017) Patch group based Bayesian learning for blind image denoising. In: Computer Vision-ACCV 2016 Workshops. ACCV 2016, Lecture Notes in Computer Science 10116, pp 79–95
Wu et al (2020) Deep-learning denoising computational ghost imaging. Opt Lasers Eng 134(2020):106183.
Zhang K, Zuo W, Zhang L (2018) FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE T Image Process 27(9):4608–4622
Li S, Deng M, Lee J, Sinha A, Barbastathis G (2018) Imaging through glass diffusers using densely connected convolutional networks. Optica 5(7):803–813
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–8
Routray S, Malla PP, Sharma SK, Panda SK, Palai G (2020) A new image denoising framework using bilateral filtering based non-subsampled Shearlet transform. Optik 216:164903
Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B (2018) Attention u-net: learning where to look for the pancreas, arXiv preprint arXiv: 1804.03999
Sutskever KI, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 2012:1097–1105
Zhang Q, Yuan Q, Li J, Sun F, Zhang L (2020) Deep spatio-spectral Bayesian posterior for hyperspectral image non-i.i.d.noise removal. ISPRS J Photogramm Remote Sens 164:125–137
Xu Z et al (2019) Deep gradient prior network for DEM super-resolution: transfer learning from image to DEM. ISPRS J Photogramm Remote Sens 150:80–90
Guan J, Lai R, Xiong A, Liu Z, Gu L (2020) Fixed pattern noise reduction for infrared images based on cascade residual attention CNN. Neurocomputing 377:301–313
Giannatoua E, Papavieros G, Constantoudis V, Papageorgiou H, Gogolides E (2019) Deep learning denoising of SEM images towards noise-reduced LER measurements. Microelectron Eng 216:111051
Jiang Q, Chen Y, Wang G, Ji T (2020) A novel deep neural network for noise removal from underwater image. Signal Process Image Commun 87:115921
Elhoseny M, Shankar K (2019) Optimal bilateral filter and convolutional neural network based denoising method of medical image measurements. Measurement 143:125–135
Ilesanmi AE, Idowu OP, Makhanov SS (2020) Multiscalesuperpixel method for segmentation of breast ultrasound. Comp Biol Med 125:103879
Mafarja M, Aljarah I, Heidari AA, Faris H, Fournier-Viger P, Li X, Mirjalili S (2018) Binary dragonfly optimization for feature selection using time-varying transfer functions. Knowl-Based Syst 161(1):185–204
Singh K, Ranade SK, Singh C (2017) A hybrid algorithm for speckle noise reduction of ultrasound images. Comp Methods Progr Biomed 148:55–69
Feng X, Huang Q, Li X (2020) Ultrasound image de-speckling by a hybrid deep network with transferred filtering and structural prior. Neurocomputing 414:346–355
Kokil P, Sudharson S (2020) Despeckling of clinical ultrasound images using deep residual learning. Comp Methods Programs Biomed 194:105477
Kim H-J, Lee D (2020) Image denoising with conditional generative adversarial networks (CGAN) in low dose chest images. Nuclear Inst Methods Phys Res A 954:161914
Kunfeng W, Xuan L, Lan Y et al (2017) Generative adversarial networks for parallel vision. In: Proc., Chinese Autom. Cong., Jinan, China
Burlingame EA, Margolin A, Gray JW et al (2018) SHIFT: speedy histopathological to-immunofluorescent translation of whole slide images using conditional generative adversarial networks. In: Proc., SPIE Medical Imaging, Houston, Texas, United States
Li S, Zhou J, Liang D, Liu Q (2020) MRI denoising using progressively distribution-based neural network. Magn Reson Imaging 71:55–68
Tripathi PC, Bag S (2020) CNN-DMRI: a convolutional neural network for denoising of magnetic resonance images. Pattern Recogn Lett 135:57–63
Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386
Garzelli A (2016) A review of image fusion algorithms based on the super-resolution paradigm. Remote Sens 8:797. https://doi.org/10.3390/rs8100797
Setiadi DIM (2021) PSNR vs SSIM: imperceptibility quality assessment for image steganography. Multimed Tools Appl 80:8423–8444
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252
Zhou B, Khosla A, Lapedriza A, Torralba A, Oliva A. Places2: A large-scale database for scene understanding, ArXiV preprint. arXiv: 1610.02055
Martin D, Fowlkes C, Tal D, Malik J et al. (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV Vancouver
Ma K, Duanmu Z, Wu Q, Wang Z, Yong H, Li H et al (2016) Waterlooexploration database: new challenges for image quality assessment models. IEEE Trans Image Process 26(2):1004–1016
Cohen G, Afshar S, Tapson J, van Schaik A (2017) EMNIST: an extension of MNIST to handwritten letters. arXiv: 1702.05373
Lin T-Y, Maire M, Belongie S, Bourdev L, Girshick R, Hays J et al (2014) Microsoft COCO: common objects in context. In: European conference on computer vision. Springer, Cham, pp 740–55
Bychkovsky V et al (2011) Learning photographic global tonal adjustment with a database of input/output image Pairs. IEEE Conf Comp Vis Pattern Recogn (Cvpr) 2011:97–104
J. Deng et al (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition (Cvpr), pp 248–255
Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse representations. In: International conference on curves and surfaces, pp 711–730
Roth S, Black MJ (2009) Fields of experts. Int J Comput Vision 82(2):205–229
Anaya J, Barbu A (2018) RENOIR—a dataset for real low-light image noise reduction. J Vis Commun Image Represent 51:144–154
Lebrun M, Colom M, Morel JM (2014) The noise clinic: a universal blind denoising algorithm. In 2014 IEEE International Conference on Image Processing (Icip), pp 2674–2678
Nam S et al (2016) A holistic approach to cross-channel image noise modeling and its application to image denoising. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (Cvpr), pp 1683–1691
Abdelhamed A, Lin S, Brown MS (2018) A high-quality denoising dataset for smartphone cameras. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (Cvpr), pp 1692–1700
Xiao J, Ehinger KA, Hays J, Torralba A, Oliva A (2016) SUN database: exploring a large collection of scene categories. Int J Comput Vis 119(1):3–22
Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. BMVA Press
Chakrabarti YA, Zickler T (2011) Statistics of real-world hyperspectral images. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA, pp 193–200
Yasuma F, Mitsunaga T, Iso D, Nayar SK (2010) Generalized assorted pixel camera: post capture control of resolution, dynamic range, and spectrum. IEEE Trans Image Process 19(9):2241–2253
https://www.kaggle.com/ilknuricke/neurohackinginrimages. Accessed 15 Mar 2021
Timofte R, Gu S, Wu J, Van Gool L (2018) Ntire 2018 challenge on single image super-resolution: methods and results. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018. pp 852–63
Tay PC, Acton ST, Hossack JA (2006) Ultrasound despeckling using an adaptive window stochastic approach. In: Proceedings of the International Conference on Image Processing, 2006, pp 2549–2552
Geertsma T (2011) Ultrasoundcases.info
Antony J (2015), Ultrasound-images.com
Cancer image archive database, Available at: https://www.cancerimagingarchive.net/ Accessed 15 Mar 2021
Brain web, Simulated brain database, McConnell Brain Imaging Centre, Montreal Neurological Institute, McGill, 2004. https://www.mcgill.ca/bic/software/brainweb-mri-simulator. Accessed 15 Mar 2021
Kwan RK-S, Evans AC, Pike GB (1996) An extensible MRI simulator for post-processing evaluation. In: Visualization in Biomedical Computing (VBC’96). Lecture Notes in Computer Science, vol. 1131. Springer Verlag, pp 135–140
IXI MRI, Brain MRI database, Imperial College London (2018).https://brain-development.org/ixi-dataset/ Accessed 15 Mar 2021
Loizou CP, Murray V, Pattichis MS, Seimenis I, Pantziaris M, Pattichis CS (2010) Multiscale amplitude-modulation frequency-modulation (am–fm) texture analysis of multiple sclerosis in brain MRI images. IEEE Trans Inform Technol Biomed 15(1):119–129
Prostate MRI, Prostate MR image database, National Center for Image Guided Therapy (2008). https://prostatemrimagedatabase.com/. Accessed 15 Mar 2021
Rodtook A, Kirimasthong K, Lohitvisate W, Makhanov SS (2018) Automatic initialization of active contours and level set method in ultrasound images of breast abnormalities. Pattern Recogn 79:172–182
Koesten LM, Kacprzak E, Tennison JFA, Simperl E (2017) The trials and tribulations of working with structured data: a study on information seeking behavior. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, pp. 1277–1289, https://doi.org/10.1145/3025453.3025838
Acknowledgements
This research is supported by Thailand Research Fund grant RSA6280098 and the Center of Excellence in Biomedical Engineering of Thammasat University. Special thanks to the management of Alex Ekwueme Federal University, Ndufu-Alike for support during this research. The authors express thanks to Professor Stanislav S. Makhanov for his encouragement and support during this review paper.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors in this paper have no potential conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Ilesanmi, A.E., Ilesanmi, T.O. Methods for image denoising using convolutional neural network: a review. Complex Intell. Syst. 7, 2179–2198 (2021). https://doi.org/10.1007/s40747-021-00428-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40747-021-00428-4