Skip to main content
SpringerLink
Log in

Image enhancement algorithm based on generative adversarial network in combination of improved game adversarial loss mechanism

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The feed-forward architectures of recently proposed generative adversarial network can learn the non-linear mapping from low-resolution output to high-resolution output. However, this approach does not fully address the mutual dependencies of different resolution images. By analyzing the zero-sum game, the paper proposes an image enhancement algorithm by using conditional generative adversarial networks based on improved non-saturating game. Firstly, the enhancement image obtained by the GAN model is adopted as a condition against the network object image, making the original image learning the network structure of the object image with dim-small. Our proposed generative adversarial networks can obtain a clearer image through improved non-saturating game, and it can still get a large gradient and sufficient learning, which makes up for the deficiencies in the mini-maximum game. In addition, the loss function of the network adds the loss of discriminator to guide discriminator to generate high quality images. We compared the proposed method (SRG) with other methods including SC, SRCNN, VESPCN and ESPCN, and the proposed method resulted in obvious improvements in the peak signal-to-noise ratio (PSNR) by 2.348 dB and in structural similarity index measurement (SSIM) by 1.89% to enhance the visual effects of nature images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Caballero J, Ledig C, Aitken A, et al (2017) Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, p 172-181

  2. Chong F, Chaoyun W, Grand L et al (2017) Projections onto convex sets super-resolution reconstruction based on point spread function estimation of low-resolution remote sensing images[J]. Sensors 17(2):362

    Article  Google Scholar 

  3. Chong F, Xushuai C, Lei Z et al (2017) Improved Wallis dodging algorithm for large-scale super-resolution reconstruction remote sensing images[J]. Sensors 17(3):623

    Google Scholar 

  4. Creswell A, White T, Dumoulin V et al (2017) Generative adversarial networks: an overview[J]. IEEE Signal Process Mag 35(1):53–65

    Article  Google Scholar 

  5. Culley S, Albrecht D, Jacobs C et al (2018) Quantitative mapping and minimization of super-resolution optical imaging artifacts[J]. Nat Methods, 15(4):263–266

  6. Culley S, Albrecht D, Jacobs C et al (2018) NanoJ-SQUIRREL: quantitative mapping and minimisation of super-resolution optical imaging artefacts[J]. Nat Methods 15(4):263–266

    Article  Google Scholar 

  7. Darren P, Rasim L, Jon P et al (2018) Landsat super-resolution enhancement using convolution neural networks and Sentinel-2 for training[J]. Remote Sens 10(3):394

    Article  Google Scholar 

  8. Elad M, Datsenko D (2008) Example-based regularization deployed to super-resolution reconstruction of a single image[J]. Comput J 52(1):15–30

    Article  Google Scholar 

  9. Graf BL, Rojo LE, Delatorre-Herrera J (2017) ChromoTrace reconstruction of 3D chromosome configurations by super-resolution microscopy[J]. Food Chem 131(2):387–396

    Google Scholar 

  10. Huang DT, Huang WQ, Gu PT et al (2017) Image super-resolution reconstruction based on regularization technique and guided filter[J]. Infrared Phys Technol 83:103–113

    Article  Google Scholar 

  11. Huang B, Chen W, Wu X et al (2018) High-quality face image generated with conditional boundary equilibrium generative adversarial networks[J]. Pattern Recogn Lett 111:72–79

    Article  Google Scholar 

  12. Jinsheng X, Enyu L, Li Z et al (2017) Improved image super-resolution algorithm based on convolutional neural network[J]. Acta Opt Sin 32(7):872–890

    Google Scholar 

  13. Ledig C, Theis L, Huszar F, et al (2016) Photo-realistic single image super-resolution using a generative adversarial network[J]. computer vision and pattern recognition 105–114

  14. Ledig C, Theis L, Huszar F, et al (2017) Photo-realistic single image super-resolution using a generative adversarial network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE

  15. Lei J, Li L, Yue H et al (2017) Depth map super-resolution considering view synthesis quality[J]. IEEE Trans Image Process 26(4):1732–1745

    Article  MathSciNet  MATH  Google Scholar 

  16. Li D, Wang Z (2017) Face video super-resolution with identity guided generative adversarial networks[C]// Ccf Chinese Conference on Computer Vision. Springer, Singapore

  17. Lucas A, Tapia SL, Molina R, et al (2018) Generative adversarial networks and perceptual losses for video super-resolution[J]. international conference on image processing 2018:51–55

  18. Mahapatra D, Bozorgtabar B (2017) Retinal vasculature segmentation using local saliency maps and generative adversarial networks for image super resolution[J]

  19. Mahapatra D, Bozorgtabar B, Hewavitharanage S, et al (2017) Image super resolution using generative adversarial networks and local saliency maps for retinal image analysis[C]// International Conference on Medical Image Computing & Computer-assisted Intervention. Springer, Cham

  20. Okanovic M, Hillig B, Breuer F et al (2018) Time-of-flight MR-angiography with a helical trajectory and slice-super-resolution reconstruction[J]. Magn Reson Med

  21. Sanchez I, Vilaplana V (2018) Brain MRI super-resolution using 3D generative adversarial networks[J]. Computer Vision and Pattern Recognition 2018:1–8

  22. Shi W, Caballero J, Huszár, F, et al (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[J]. 12(2):722-739

  23. Shi Y, Li Q, Zhu XX (2018) Building footprint generation using improved generative adversarial networks[J]. IEEE Geosci Remote Sens Lett

  24. Wang X, Yu K, Wu S, et al (2018) ESRGAN: enhanced super-resolution generative adversarial networks[J]. european conference on computer vision 2018:63–79

  25. Ying C, Zhao P, Li Y (2018) Low-light-level image super-resolution reconstruction based on iterative projection photon localization algorithm[J]. J Electron Imaging 27(1):1

    Article  Google Scholar 

  26. Yisheng L, Yuanyuan C, Li L, et al (2018) Generative adversarial networks for parallel transportation systems[J]. IEEE Intell Transp Syst Mag 1-1

  27. Yuan Y, Liu S, Zhang J, et al (2018) Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks[J]. computer vision and pattern recognition 701–710.

  28. Zhang D, He J (2017) Hybrid sparse-representation-based approach to image super-resolution reconstruction[J]. J Electron Imaging 26(2):023008

    Article  Google Scholar 

  29. Zhang DX, Lu L, Li CH et al (2014) Super-resolution image reconstruction algorithm based on sub-pixel shift[J]. Acta Automat Sin 40(12):2851–2861

    MATH  Google Scholar 

  30. Zhang D, Shao J, Hu G, et al (2017) Sharp and real image super-resolution using generative adversarial network[C]// International Conference on Neural Information Processing 217–226

  31. Zhang M, Hu X, Zhao L et al (2017) Translation-aware semantic segmentation via conditional Least Square generative adversarial networks[J]. J Appl Remote Sens (4):11

  32. Zhao L, Bai H, Liang J, et al (2017) Simultaneously color-depth super-resolution with conditional generative adversarial network[J]. Pattern Recogn 356–369

Download references

Acknowledgments

This work was financially supported by the Natural Science Foundation of Zhejiang Province, China (No. LGF18F020015).

Author information

Authors and Affiliations

  1. Faculty of Engineering, University of Yamanashi, Kofu, 4000016, Japan

    Caie Xu

  2. School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, 310000, Zhejiang, China

    Yang Cui, Yunhui Zhang, Peng Gao & Jiayi Xu

Authors
  1. Caie Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yunhui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiayi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Caie Xu or Jiayi Xu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, C., Cui, Y., Zhang, Y. et al. Image enhancement algorithm based on generative adversarial network in combination of improved game adversarial loss mechanism. Multimed Tools Appl 79, 9435–9450 (2020). https://doi.org/10.1007/s11042-019-07776-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07776-x

Keywords

Search

Navigation