Advertisement

Image Up-Sampling for Super Resolution with Generative Adversarial Network

  • Shohei Tsunekawa
  • Katsufumi Inoue
  • Michifumi Yoshioka
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11320)

Abstract

In case, we carry out single image Super Resolution (SR) utilizing deep learning, we utilize bicubic interpolation for up-sampling of low resolution images before input them into SR methods. In the preprocessing, these basic interpolation methods cause blur and noise effects for after processed images. These noise images may affect the SR results. In this research, by focusing on this point, we propose a new image up-sampling method utilizing Generative Adversarial Network (GAN). In this work, we improve an image evaluation criterion in generator part of GAN by combining Multi-Scale Structural Similarity (MS-SSIM) and L1 norm. From experiments, we have confirmed that our method allows us to create more qualitatively up-sampling images. As the quantitative results, our proposed method have achieved 0.90 [dB] of average PSNR, 3.35 [%] of average SSIM, and 1.28 [%] of average MS-SSIM improvement using Set 5 and Set 14 dataset compared with bicubic interpolation.

Keywords

Image up-sampling GAN Super resolution 

References

  1. 1.
    Irani, M.: Improving resolution by image registration. CVGIP: Graph. Model. Image Process. 53, 231–239 (1991)Google Scholar
  2. 2.
    Elad, M., Feuer, A.: Superresolution restoration of an image sequence: adaptive filtering approach. IEEE Trans. Image Process. 8(3), 387–395 (1999)CrossRefGoogle Scholar
  3. 3.
    Cruz, C., Mehta, R., Katkovnik, V., Egiazarian, K.O.: Single image super-resolution based on wiener filter in similarity domain. CoRR abs/1704.04126 (2017)Google Scholar
  4. 4.
    Wang, L., Xiang, S., Meng, G., Wu, H., Pan, C.: Edge-directed single-image super-resolution via adaptive gradient magnitude self-interpolation. IEEE Trans. Circuits Syst. Video Technol. 23(8), 1289–1299 (2013)CrossRefGoogle Scholar
  5. 5.
    Fattal, R.: Upsampling via imposed edges statistics. ACM Transactions on Graphics 26(3) (2007). (Proceedings of SIGGRAPH 2007)Google Scholar
  6. 6.
    Freeman, W., Jones, T., Pasztor, E.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)CrossRefGoogle Scholar
  7. 7.
    Sun, J., Zheng, N.N., Tao, H., Shum, H.Y.: Image hallucination with primal sketch priors. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Proceedings, vol. 2, pp. II-729-736 (2003)Google Scholar
  8. 8.
    Dong, C., Loy, C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)CrossRefGoogle Scholar
  9. 9.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10593-2_13CrossRefGoogle Scholar
  10. 10.
    Zhao, Y., et al.: GUN: gradual upsampling network for single image super-resolution. CoRR abs/1703.04244 (2017)Google Scholar
  11. 11.
    Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates, Inc. (2014)Google Scholar
  12. 12.
    Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. CoRR abs/1609.04802 (2016)Google Scholar
  13. 13.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)Google Scholar
  14. 14.
    Nguyen, A.M., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. CoRR abs/1412.1897 (2014)Google Scholar
  15. 15.
    Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems Computers, vol. 2, pp. 1398–1402, November 2003Google Scholar
  16. 16.
    Cai, L., Gao, H., Ji, S.: Multi-stage variational auto-encoders for coarse-to-fine image generation. CoRR abs/1705.07202 (2017)Google Scholar
  17. 17.
    van den Oord, A., Kalchbrenner, N., Kavukcuoglu, K.: Pixel recurrent neural networks. CoRR abs/1601.06759 (2016)Google Scholar
  18. 18.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, June 2016Google Scholar
  19. 19.
    Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration with neural networks. IEEE Trans. Comput. Imaging 3(1), 47–57 (2017)CrossRefGoogle Scholar
  20. 20.
    Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  21. 21.
    Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Prentice Hall, Upper Saddle River (2007)Google Scholar
  22. 22.
    Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. CoRR abs/1511.04587 (2015)Google Scholar
  23. 23.
    Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. CoRR abs/1707.02921 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Shohei Tsunekawa
    • 1
  • Katsufumi Inoue
    • 1
  • Michifumi Yoshioka
    • 1
  1. 1.Osaka Prefecture UniversityOsakaJapan

Personalised recommendations