Advertisement

Improved Algorithms for Zero Shot Image Super-Resolution with Parametric Rectifiers

  • Jiayi ZhuEmail author
  • Senjian An
  • Wanquan Liu
  • Ling Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11888)

Abstract

Recently, a novel Zero-Shot Super-Resolution (ZSSR) method is proposed to generate high-resolution (HR) images from their low-resolution (LR) counterparts. ZSSR employs a convolutional neural network (CNN) to represent transformations from LR images to HR images and is trained on a single image. ZSSR achieves state-of-the-art performance on both real low-resolution images (i.e., historic images, and images taken with a mobile phone) and several benchmark datasets (e.g., Set 5 and Set 14 to name a few). However, the training of the CNN network of ZSSR is not stable since rectifier is used as the activation function and a custom learning rate adjustment policy is proposed in ZSSR. In this paper, we use parametric rectifier as the activation function and present an improved algorithm for the training of ZSSR. Experimental results demonstrate that the proposed method outperforms ZSSR in terms of both reconstruction accuracy and speed on two benchmark datasets: Set 5 and Set 14, respectively.

Keywords

Single Image Super-Resolution Unsupervised Computer vision 

Notes

Acknowledgment

This work is supported by a Faculty of Science and Engineering Research and Development Committee Small Grants Program of Curtin University.

References

  1. 1.
    Shocher, A., Cohen, N., Irani, M.: “Zero-shot” super-resolution using deep internal learning. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018Google Scholar
  2. 2.
    Bevilacqua, M., Roumy, A., Guillemot, C., Alberi Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the British Machine Vision Conference, pp. 135.1–135.10. BMVA Press (2012).  https://doi.org/10.5244/C.26.135
  3. 3.
    Chowdhuri, D., Sendhil Kumar, K.S., Babu, M.R., Reddy, C.P.: Very low resolution face recognition in parallel environment. Int. J. Comput. Sci. Inf. Technol. 3, 4408–4410 (2012)Google Scholar
  4. 4.
    Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). CoRR abs/1511.07289 (2016)Google Scholar
  5. 5.
    Epstein, J.: Chapter 3 - elements of voice quality. In: Epstein, J. (ed.) Scalable VoIP Mobility, pp. 57 – 72. Newnes, Boston (2009).  https://doi.org/10.1016/B978-1-85617-508-1.00003-7, http://www.sciencedirect.com/science/article/pii/B9781856175081000037CrossRefGoogle Scholar
  6. 6.
    Ferwerda, J.A.: Three varieties of realism in computer graphics. In: Proceedings of SPIE, Human Vision and Electronic Imaging VIII, vol. 5007 (2003).  https://doi.org/10.1117/12.473899
  7. 7.
    Freedman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Trans. Graph. 30(2), 12:1–12:11 (2011).  https://doi.org/10.1145/1944846.1944852CrossRefGoogle Scholar
  8. 8.
    Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 349–356, September 2009.  https://doi.org/10.1109/ICCV.2009.5459271
  9. 9.
    Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates, Inc. (2014). http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. CoRR abs/1502.01852 (2015). http://arxiv.org/abs/1502.01852
  11. 11.
    Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015Google Scholar
  12. 12.
    Irani, M., Peleg, S.: Improving resolution by image registration. CVGIP: Graph. Models Image Process. 53(3), 231–239 (1991).  https://doi.org/10.1016/1049-9652(91)90045-L. http://www.sciencedirect.com/science/article/pii/104996529190045LCrossRefGoogle Scholar
  13. 13.
    Keys, R.: Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust. Speech Sig. Process. 29(6), 1153–1160 (1981).  https://doi.org/10.1109/TASSP.1981.1163711MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016Google Scholar
  15. 15.
    Kim, J., Kwon Lee, J., Mu Lee, K.: Deeply-recursive convolutional network for image super-resolution. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016Google Scholar
  16. 16.
    Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. CoRR abs/1511.04587 (2015). http://arxiv.org/abs/1511.04587
  17. 17.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations, December 2014Google Scholar
  18. 18.
    LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989).  https://doi.org/10.1162/neco.1989.1.4.541CrossRefGoogle Scholar
  19. 19.
    Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  20. 20.
    Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. CoRR abs/1707.02921 (2017). http://arxiv.org/abs/1707.02921
  21. 21.
    Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: ICML Workshop on Deep Learning for Audio, Speech and Language Processing (2013)Google Scholar
  22. 22.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 2, pp. 416–423 (2001)Google Scholar
  23. 23.
    Matsui, Y., et al.: Sketch-based manga retrieval using manga109 dataset. Multimed. Tools Appl. 76(20), 21811–21838 (2017).  https://doi.org/10.1007/s11042-016-4020-zCrossRefGoogle Scholar
  24. 24.
    McCann, M.T., Jin, K.H., Unser, M.: Convolutional neural networks for inverse problems in imaging: a review. IEEE Sig. Process. Mag. 34(6), 85–95 (2017).  https://doi.org/10.1109/MSP.2017.2739299CrossRefGoogle Scholar
  25. 25.
    Michaeli, T., Irani, M.: Nonparametric blind super-resolution. In: The IEEE International Conference on Computer Vision (ICCV), December 2013Google Scholar
  26. 26.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)Google Scholar
  27. 27.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004).  https://doi.org/10.1109/TIP.2003.819861CrossRefGoogle Scholar
  28. 28.
    Yang, Q., Yang, R., Davis, J., Nister, D.: Spatial-depth super resolution for range images. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, June 2007.  https://doi.org/10.1109/CVPR.2007.383211
  29. 29.
    Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J., Liao, Q.: Deep learning for single image super-resolution: a brief review. IEEE Trans. Multimed. 1 (2019).  https://doi.org/10.1109/TMM.2019.2919431
  30. 30.
    Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., et al. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-27413-8_47CrossRefGoogle Scholar
  31. 31.
    Zhang, S.: Application of super-resolution image reconstruction to digital holography. EURASIP J. Adv. Sig. Process. 2006(1), 090358 (2006).  https://doi.org/10.1155/ASP/2006/90358CrossRefGoogle Scholar
  32. 32.
    Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018Google Scholar
  33. 33.
    Zontak, M., Mosseri, I., Irani, M.: Separating signal from noise using patch recurrence across scales. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Electrical Engineering, Computing and Mathematical SciencesCurtin UniversityBentleyAustralia

Personalised recommendations