Skip to main content

Tackling Multiple Visual Artifacts: Blind Image Restoration Using Conditional Adversarial Networks

  • Conference paper
  • First Online:
Computer Vision and Image Processing (CVIP 2019)

Abstract

Restoring images that are degraded by visual artifacts like noise, blurness and other environmental visual artifacts like shadow, snow, rain, and haze is a challenging task. From literature, it can be seen that there are many model-based as well as blind restoration methods that have been proposed to restore an image degraded by a single artifact. In most practical cases, the image is degraded by more than one artifact. Complexity arises while trying to estimate degradation function using conventional techniques where images are degraded by multiple visual artifacts. To the best of our knowledge, there has not been any generalized method proposed to tackle this problem. In this paper, we propose a methodology using conditional adversarial networks for blind image restoration of images that are degraded by multiple artifacts. To analyze the performance, ISTD dataset (meant originally for shadow removal) is used by augmenting it with different types of noises and blurness. The network has been trained on this data and has been analyzed how it behaves during the addition of each artifact. Various image quality metrics like Peak signal-to-noise ratio (PSNR), Mean squared error (MSE), Structural similarity index (SSIM), Blind/reference-less image spatial quality evaluator (BRISQUE) and Naturalness image quality evaluator (NIQE) have been evaluated to validate the performance of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Banham, M.R., Katsaggelos, A.K.: Digital image restoration. IEEE Sig. Process. Mag. 14(2), 24–41 (1997)

    Article  Google Scholar 

  2. Kaur, A., Verma, D.: Blind and non-blind image restoration techniques. Int. J. Adv. Res. Comput. Sci. 4(8), 315–317 (2013)

    Google Scholar 

  3. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)

    Article  Google Scholar 

  4. Sünderhauf, N., et al.: The limits and potentials of deep learning for robotics. Int. J. Robot. Res. 37(4-5), 405–420 (2018)

    Article  Google Scholar 

  5. Bakas, S., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629 (2018)

  6. Islam, M., Jose, V.J.M., Ren, H.: Glioma prognosis: segmentation of the tumor and survival prediction using shape, geometric and clinical information. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11384, pp. 142–153. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11726-9_13

    Chapter  Google Scholar 

  7. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  8. Junginger, A., Hanselmann, M., Strauss, T., Boblest, S., Buchner, J., Ulmer, H.: Unpaired high-resolution and scalable style transfer using generative adversarial networks. arXiv preprint arXiv:1810.05724 (2018)

  9. Demir, U., Unal, G.: Patch-based image inpainting with generative adversarial networks. arXiv preprint arXiv:1803.07422 (2018)

  10. Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)

    Google Scholar 

  11. Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., Matas, J.: DeblurGAN: blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8183–8192 (2018)

    Google Scholar 

  12. Islam, M., Vaidyanathan, N.R., Jose, V.J.M., Ren, H.: Ischemic stroke lesion segmentation using adversarial learning. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 292–300. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_29

    Chapter  Google Scholar 

  13. Tripathi, S., Lipton, Z.C., Nguyen, T.Q.: Correction by projection: denoising images with generative adversarial networks. arXiv preprint arXiv:1803.04477 (2018)

  14. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)

    Google Scholar 

  15. Donoho, D.L., Johnstone, I.M.: Adapting to unknown smoothness via wavelet shrinkage. J. Am. Stat. Assoc. 90(432), 1200–1224 (1995)

    Article  MathSciNet  Google Scholar 

  16. Chan, R.H., Ho, C.-W., Nikolova, M.: Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14(10), 1479–1485 (2005)

    Article  Google Scholar 

  17. Zhong, H., Li, Y., Jiao, L.: SAR image despeckling using bayesian nonlocal means filter with sigma preselection. IEEE Geosci. Remote Sens. Lett. 8(4), 809–813 (2011)

    Article  Google Scholar 

  18. Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing shadows from images. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 823–836. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47979-1_55

    Chapter  Google Scholar 

  19. Fredembach, C., Finlayson, G.D.: Fast re-integration of shadow free images. In: Twelfth Color Imaging Conference: Color Science and Engineering Systems, Technologies, and Applications, pp. 117–122 (2004)

    Google Scholar 

  20. Zhang, L., Zhang, Q., Xiao, C.: Shadow remover: image shadow removal based on illumination recovering optimization. IEEE Trans. Image Process. 24(11), 4623–4636 (2015)

    Article  MathSciNet  Google Scholar 

  21. Wang, J., Li, X., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1788–1797 (2018)

    Google Scholar 

  22. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  23. https://affinelayer.com/pix2pix/

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to S. Deivalakshmi .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Anand, M. et al. (2020). Tackling Multiple Visual Artifacts: Blind Image Restoration Using Conditional Adversarial Networks. In: Nain, N., Vipparthi, S., Raman, B. (eds) Computer Vision and Image Processing. CVIP 2019. Communications in Computer and Information Science, vol 1148. Springer, Singapore. https://doi.org/10.1007/978-981-15-4018-9_30

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-4018-9_30

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-4017-2

  • Online ISBN: 978-981-15-4018-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics