Skip to main content
SpringerLink
Log in

Brief Survey of Single Image Super-Resolution Reconstruction Based on Deep Learning Approaches

  • Overview / Summary paper
  • Published:
Sensing and Imaging Aims and scope Submit manuscript

Abstract

With the presentation of super-resolution convolutional neural network, deep learning approach was applied to image super-resolution reconstruction for the first time. By using convolutional neural network, the deep learning approaches can directly learn the mapping relationship between the low-resolution image and high-resolution image, and have achieved better reconstruction effects than the traditional image super-resolution reconstruction methods. Subsequently, a series of improved deep learning approaches have been proposed, and the reconstruction effects have been improved continuously. This paper systematically summa rizes the image super-resolution reconstruction approaches based on deep learning, analyzes the characteristics of different models, and compares the main deep learning models based on the experiments. Furthermore, based on deep learning model, the future research directions of the image super-resolution reconstruction methods based on deep learning models are reasonably predicted.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9.
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Shi, W., Caballero, J., Ledig C, et al. (2013). Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In International conference on medical image computing and computer-assisted intervention (pp. 9–16). Springer, Berlin, Heidelberg.

  2. Zou, W., & Yuen, P. (2011). Very low resolution face recognition problem. IEEE Transactions on Image Processing, 21(1), 327–340.

    Article  MathSciNet  Google Scholar 

  3. Sun, J., Xu, Z., & Shum, H. Y. (2008). Image super-resolution using gradient profile prior. In 2008 IEEE conference on computer vision and pattern recognition (pp. 1–8). IEEE.

  4. Kim, K. I., & Kwon, Y. (2010). Single-image super-resolution using sparse regression and natural image prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(6), 1127–1133.

    Article  Google Scholar 

  5. Glasner, D., Bagon, S., & Irani, M. (2009). Super-resolution from a single image. In 2009 IEEE 12th international conference on computer vision (pp. 349–356). IEEE.

  6. Huang, J. B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5197–5206).

  7. Pérez-Pellitero, E., Salvador, J., Ruiz-Hidalgo, J., et al. (2016). Psyco: Manifold span reduction for super resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1837–1845).

  8. Salvador, J., & Perez-Pellitero, E. (2015). Naive bayes super-resolution forest. In Proceedings of the IEEE international conference on computer vision (pp. 325–333).

  9. Schulter, S., Leistner, C., & Bischof, H. (2015). Fast and accurate image upscaling with super-resolution forests. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3791-3799).

  10. Wang, W., Yang, Y., Wang, X., et al. (2019). Development of convolutional neural network and its application in image classification: A survey. Optical Engineering, 58(4), 040901.

    Google Scholar 

  11. Wei, W., Can, T., Xin, W., Yanhong, L., Yongle, H., & Ji, L. (2019). Image object recognition via deep feature-based adaptive joint sparse representation. Computational Intelligence and Neuroscience. https://doi.org/10.1155/2019/8258275.

    Article  Google Scholar 

  12. Wang, W., Li, Y., Zou, T., Wang, X., You, J., & Luo, Y. (2010). A novel image classification approach via dense-MobileNet models. Mobile Information Systems. https://doi.org/10.1155/2020/7602384.

    Article  Google Scholar 

  13. Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527–1554.

    Article  MathSciNet  Google Scholar 

  14. Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of Physiology, 160(1), 106–154.

    Article  Google Scholar 

  15. LeCun, Y., Bottou, L., Bengio, Y., et al. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

    Article  Google Scholar 

  16. He, K., Zhang, X., Ren, S., et al. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp 770–778).

  17. Yu, F., Koltun, V., & Funkhouser, T. (2017). Dilated residual networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. (pp. 472–480).

  18. Zhong, Z., Li, J., Ma, L., et al. (2017) Deep residual networks for hyperspectral image classification. In 2017 IEEE international geoscience and remote sensing symposium (IGARSS) (pp. 1824–1827). IEEE.

  19. Lee, S. H., Hosseini, S., Kwon, H. J., et al. (2018). Age and gender estimation using deep residual learning network. In 2018 International workshop on advanced image technology (IWAIT) (pp. 1–3). IEEE.

  20. Li, N., Huang, N., & Xiao, L. (2017). Pan-sharpening via residual deep learning. In 2017 IEEE international geoscience and remote sensing symposium (IGARSS) (pp. 5133–5136). IEEE.

  21. Huang, G., Liu, Z., Van Der Maaten, L., et al. (2017) Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4700–4708).

  22. Krešo, I., Oršić, M., Bevandić, P., et al. (2018). Robust semantic segmentation with ladder-densenet models. arXiv preprint arXiv:1806.03465.

  23. Park, S., Jeong, Y., & Kim, H. S. (2018). Multi-resolution DenseNet based acoustic models for reverberant speech recognition. Phonetics and Speech Sciences, 10(1), 33–38.

    Article  Google Scholar 

  24. Liu, W., & Zeng, K. (2018). SparseNet: A sparse DenseNet for image classification. arXiv preprint arXiv:1804.05340.

  25. Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al. (2014). Generative adversarial nets. In Advances in neural information processing systems (pp. 2672–2680).

  26. Dong, C., Loy, C. C., He, K., et al. (2015). Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 295–307.

    Article  Google Scholar 

  27. Shi, W., Caballero, J., Huszár, F., et al. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition. (pp. 1874–1883).

  28. Dong, C., Loy, C. C., & Tang, X. (2016). Accelerating the super-resolution convolutional neural network. In B. Leibe, J. Matas, N. Sebe, M. Welling (Eds.), European Conference on Computer Vision (pp. 391–407). Cham: Springer. https://doi.org/10.1007/978-3-319-46475-6_25.

    Chapter  Google Scholar 

  29. Zeiler, M. D., Taylor, G. W., & Fergus, R. (2011). Adaptive deconvolutional networks for mid and high level feature learning. In 2011 international conference on computer vision (pp. 2018–2025). IEEE.

  30. Kang, E., Yoo, J., Ye, J. C. (2017). Wavelet residual network for low-dose CT via deep convolutional framelets. arXiv preprint arXiv:1707.0993827, 679–694.

  31. Sinha, A., Unmesh, A., Huang, Q., et al. (2017). Surfnet: Generating 3d shape surfaces using deep residual networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6040–6049).

  32. Zhong, Z., Zhang, X. Y., Yin, F., et al. (2016). Handwritten Chinese character recognition with spatial transformer and deep residual networks. In 2016 23rd international conference on pattern recognition (ICPR) (pp. 3440–3445). IEEE.

  33. Kim, J., Kwon Lee, J., & Mu Lee, K. (2016). Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1646–1654).

  34. Wang, Y., Wang, L., Wang, H., et al. (2019). End-to-end image super-resolution via deep and shallow convolutional networks. IEEE Access, 7, 31959–31970.

    Article  Google Scholar 

  35. Kim, J., Kwon Lee, J., Mu Lee, K. (2016) Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1637–1645).

  36. Tai, Y., Yang, J., Liu, X. (2017) Image super-resolution via deep recursive residual network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3147–3155).

  37. Wei, W., Yongbin, J., Yanhong, L., et al. (2019). An advanced deep residual dense network (DRDN) approach for image super-resolution. International Journal of Computational Intelligence Systems, 12(2), 1592–1601.

    Article  Google Scholar 

  38. Tai, Y., Yang, J., Liu, X., et al. (2017) Memnet: A persistent memory network for image restoration. In Proceedings of the IEEE international conference on computer vision (pp. 4539–4547).

  39. Tong, T., Li, G., Liu, X., et al. (2017). Image super-resolution using dense skip connections. In Proceedings of the IEEE international conference on computer vision. (pp. 4799–4807).

  40. Zhang, Y., Tian, Y., Kong, Y., et al. (2018) Residual dense network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2472–2481).

  41. Ledig, C., Theis, L., Huszár, F., et al. (2017) Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4681–4690).

  42. Gross, S., & Wilber, M. (2016) Training and investigating residual nets, https://torch.ch/blog/2016/02/04/resnets.html. Accessed 5 Mar 2019.

  43. Xu, T., Zhang, P., Huang, Q., et al. (2018) Attngan: Fine-grained text to image generation with attentional generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1316–1324).

  44. Bulat, A., Yang, J., Tzimiropoulos, G. (2018). To learn image super-resolution, use a GAN to learn how to do image degradation first. In Proceedings of the European conference on computer vision (ECCV) (pp. 185–200).

  45. Haris, M., Shakhnarovich, G., Ukita, N. (2018) Deep back-projection networks for super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp.1664–1673).

  46. Wang, X., Yu, K., Dong, C., et al. (2018) Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 606–615).

Download references

Funding

National Defense Pre-Research Foundation of China (7301506); National Natural Science Foundation of China (61070040); Education Department of Hunan Province (17C0043); Hunan Provincial Natural Science Fund (2019JJ80105).

Author information

Authors and Affiliations

  1. School of Computer and Communication Engineering, Changsha University of Science and Technology, Changsha, 410114, China

    Wei Wang, Yihui Hu & Tong Zhang

  2. Hunan Children’s Hospital, Changsha, 410007, China

    Yanhong Luo

Authors
  1. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yihui Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanhong Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wei Wang or Tong Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, W., Hu, Y., Luo, Y. et al. Brief Survey of Single Image Super-Resolution Reconstruction Based on Deep Learning Approaches. Sens Imaging 21, 21 (2020). https://doi.org/10.1007/s11220-020-00285-4

Download citation

  • Received:

  • Revised:

  • Published:

  • DOI: https://doi.org/10.1007/s11220-020-00285-4

Keywords

Search

Navigation