Skip to main content
SpringerLink
Log in

Image Completion of Highly Noisy Images Using Deep Learning

  • Conference paper
  • First Online:
Computational Vision and Bio-Inspired Computing ( ICCVBIC 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1108))

  • 1868 Accesses

Abstract

Generating images from a noisy or blurred image is an active research problem in the field of computer vision that aims to regenerate refined images automatically in a content-aware manner. Various approaches have been developed by academia and industry which includes modern ones applying convolution neural networks and many other approaches to have more realistic images. In this paper, we present a novel approach that leverages the combination of capsule networks and generative adversarial networks that is used for generating images in a more realistic manner. This approach tries to generate images which are locally and globally consistent in nature by leveraging the low level and high-level features of an image.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lagendijk, R.L., Biemond, J., Boekee, D.E.: Identification and restoration of noisy blurred images using the expectation-maximization algorithm. IEEE Trans. Acoust. Speech Signal Process. 38(7), 1180–1191 (1990)

    Article  Google Scholar 

  2. Gregor, K., Danihelka, I., Graves, A., Wierstra, D.: DRAW: a recurrent neural network for image generation. CoRR abs/1502.04623, arXiv:1502.04623

  3. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  4. Im, D.J., Kim, C.D., Jiang, H., Memisevic, R.: Generating images with recurrent adversarial networks. CoRR abs/1602.05110, arXiv:1602.05110

  5. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. CoRR abs/1311.2901, arXiv:1311.2901

  6. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. CoRR abs/1602.05110. https://arxiv.org/abs/1406.2661

  7. Theis, L., van den Oord, A., Bethge, M.: A note on the evaluation of generative models (2015). arXiv:arXiv:1511.01844

  8. Dosovitskiy, A., Springenberg, J.T., Brox, T.: Learning to generate chairs with convolutional neural networks, CoRR abs/1411.5928, arXiv:1411.5928

  9. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks, CoRR abs/1511.06434. arXiv:1511.06434

  10. Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784

  11. Amos, B.: Image Completion with Deep Learning in TensorFlow. http://bamos.github.io/2016/08/09/deep-completion

  12. Gorijala, M., Dukkipati, A.: Image generation and editing with variational info generative adversarial networks. arXiv preprint arXiv:1701.04568

  13. Sun, J., Yuan, L., Jia, J., Shum, H.-Y.: Image completion with structure propagation. In: ACM Transactions on Graphics (ToG), vol. 24, pp. 861–868. ACM (2005)

    Google Scholar 

  14. Darabi, S., Shechtman, E., Barnes, C., Goldman, D.B., Sen, P.: Image melding: combining inconsistent images using patch-based synthesis. ACM Trans. Graph. 31(4), 82-1 (2012)

    Article  Google Scholar 

  15. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv preprint arXiv:1701.07875

  16. Nagarajan, V., Kolter, J.Z.: Gradient descent GAN optimization is locally stable. In: Advances in Neural Information Processing Systems, pp. 5585–5595 (2017)

    Google Scholar 

  17. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules, CoRR abs/1710.09829. arXiv:1710.09829

  18. Suárez, P.L., Sappa, A.D., Vintimilla, B.X.: Infrared image colorization based on a triplet DCGAN architecture. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 212–217. IEEE (2017)

    Google Scholar 

  19. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167

  20. Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol. 30, p. 3 (2013)

    Google Scholar 

  21. Wazwaz, A.-M.: The tan h method: solitons and periodic solutions for the Dodd-Bullough-Mikhailov and the Tzitzeica-Dodd-Bullough equations. Chaos Solitons Fractals 25(1), 55–63 (2005)

    Article  MathSciNet  Google Scholar 

  22. Liu, Z., Luo, P., Wang, X., Tang, X.: Large-scale CelebFaces Attributes (CelebA) dataset (2018). Accessed 15 Aug 2018

    Google Scholar 

  23. Goodfellow, I.J., Bulatov, Y., Ibarz, J., Arnoud, S., Shet, V.: Multi-digit number recognition from street view imagery using deep convolutional neural networks. arXiv preprint arXiv:1312.6082

  24. Ng, H.W., Winkler, S.: A data-driven approach to cleaning large face datasets. In. IEEE International Conference on Image Processing (ICIP 2014), pp. 343–347 (2014). https://doi.org/10.1109/ICIP.2014.7025068

  25. Amos, B., Ludwiczuk, B., Satyanarayanan, M.: OpenFace: a general-purpose face recognition library with mobile applications. Technical report, CMU-CS-16-118, CMU School of Computer Science (2016)

    Google Scholar 

  26. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X., Chen, X.: Improved techniques for training GANs. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29, pp. 2234–2242. Curran Associates, Inc. (2016)

    Google Scholar 

  27. Dziugaite, G.K., Roy, D.M., Ghahramani, Z.: Training generative neural networks via maximum mean discrepancy optimization. arXiv:1505.03906 (2015)

  28. Yeh, R.A., Chen, C., Lim, T., Hasegawa-Johnson, M., Do, M.N.: Semantic image inpainting with perceptual and contextual losses, CoRR abs/1607.07539, arXiv:1607.07539

  29. Hinton, G.E., Sabour, S., Frosst, N.: Matrix capsules with EM routing. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=HJWLfGWRb

  30. Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: InfoGan: interpretable representation learning by information maximizing generative adversarial nets. CoRR abs/1606.03657, arXiv:1606.03657

Download references

Author information

Authors and Affiliations

  1. Bharati Vidyapeeth’s College of Engineering, New Delhi, India

    Piyush Agrawal & Sajal Kaushik

Authors
  1. Piyush Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sajal Kaushik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sajal Kaushik .

Editor information

Editors and Affiliations

  1. Department of CSE, RVS Technical Campus, Coimbatore, India

    S. Smys

  2. Faculty of Engineering, Faculdade de Engenharia da Universidade do Porto, Porto, Portugal

    João Manuel R. S. Tavares

  3. Faculty of Engineering, Aurel Vlaicu University of Arad, Arad, Romania

    Valentina Emilia Balas

  4. School of Computing, Tokyo Institute of Technology, Tokyo, Japan

    Abdullah M. Iliyasu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Agrawal, P., Kaushik, S. (2020). Image Completion of Highly Noisy Images Using Deep Learning. In: Smys, S., Tavares, J., Balas, V., Iliyasu, A. (eds) Computational Vision and Bio-Inspired Computing. ICCVBIC 2019. Advances in Intelligent Systems and Computing, vol 1108. Springer, Cham. https://doi.org/10.1007/978-3-030-37218-7_108

Download citation

Publish with us

Policies and ethics

Search

Navigation