A Universal Fusion Strategy for Image Super-Resolution Jointly from External and Internal Examples

  • Wei Wang
  • Xuesen Shang
  • Wenming YangEmail author
  • Canrong Zhang
  • Qingmin Liao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11901)


The validity of learning-based image super-resolution is largely limited by supporting dataset. Neither external-based nor internal-based super-resolution methods can perform well in real applications such as medical endoscopic images. This paper studies the strategy of joint learning of two kinds of methods. We first build sub-dictionaries and study the corresponding mapping matrices on the respective samples. Due to the consistency of learning strategies, we establish joint mapping matrices based on the distance between the input low-resolution image patches and the dictionary atoms in the reconstruction phase. We adopt the nearest neighbor strategy and the weighted joint strategy to obtain the new mapping matrix. The high-resolution image is reconstructed by the new mapping model. The experiments prove the effectiveness of our strategy.


Super-resolution External examples Internal examples Medical endoscopic images Joint learning 



This work was supported by the Natural Science Foundation of China (Nos. 61471216 and 61771276), the National Key Research and Development Program of China (No. 2016YFB0101001) and the Special Foundation for the Development of Strategic Emerging Industries of Shenzhen (Nos. JCYJ20170307153940960 and JCYJ20170817161845824).


  1. 1.
    Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002)CrossRefGoogle Scholar
  2. 2.
    Bevilacqua, M., Roumy, A., Guillemot, C., Morel, M.L.A.: Single-image super-resolution via linear mapping of interpolated self-examples. IEEE Trans. Image Process. 23(12), 5334–5347 (2014)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: 2004 Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, p. I. IEEE (2004)Google Scholar
  4. 4.
    Dai, S., Han, M., Xu, W., Wu, Y., Gong, Y.: Soft edge smoothness prior for alpha channel super resolution. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)Google Scholar
  5. 5.
    Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)CrossRefGoogle Scholar
  6. 6.
    Freedman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Trans. Graph. (TOG) 30(2), 12 (2011)CrossRefGoogle Scholar
  7. 7.
    Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 349–356. IEEE (2009)Google Scholar
  8. 8.
    Hou, H., Andrews, H.: Cubic splines for image interpolation and digital filtering. IEEE Trans. Acoust. Speech Signal Process. 26(6), 508–517 (1978)CrossRefGoogle Scholar
  9. 9.
    Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206 (2015)Google Scholar
  10. 10.
    Jiang, J., Ma, X., Chen, C., Lu, T., Wang, Z., Ma, J.: Single image super-resolution via locally regularized anchored neighborhood regression and nonlocal means. IEEE Trans. Multimed. 19(1), 15–26 (2017)CrossRefGoogle Scholar
  11. 11.
    Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)Google Scholar
  12. 12.
    Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10(10), 1521–1527 (2001)CrossRefGoogle Scholar
  13. 13.
    Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)CrossRefGoogle Scholar
  14. 14.
    Sun, J., Xu, Z., Shum, H.Y.: Image super-resolution using gradient profile prior. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)Google Scholar
  15. 15.
    Timofte, R.: Anchored fusion for image restoration. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 1412–1417. IEEE (2016)Google Scholar
  16. 16.
    Timofte, R., De Smet, V., Van Gool, L.: Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1920–1927 (2013)Google Scholar
  17. 17.
    Timofte, R., De Smet, V., Van Gool, L.: A+: adjusted anchored neighborhood regression for fast super-resolution. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 111–126. Springer, Cham (2015). Scholar
  18. 18.
    Wang, Z., Yang, Y., Wang, Z., Chang, S., Yang, J., Huang, T.S.: Learning super-resolution jointly from external and internal examples. IEEE Trans. Image Process. 24(11), 4359–4371 (2015)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Wang, Z., Liu, D., Yang, J., Han, W., Huang, T.: Deep networks for image super-resolution with sparse prior. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 370–378 (2015)Google Scholar
  20. 20.
    Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. 21(8), 3467–3478 (2012)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., et al. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Wei Wang
    • 1
  • Xuesen Shang
    • 1
  • Wenming Yang
    • 1
    Email author
  • Canrong Zhang
    • 2
  • Qingmin Liao
    • 1
  1. 1.Department of Electronic Engineering, Graduate School at ShenzhenTsinghua UniversityShenzhenChina
  2. 2.Research Center for Modern Logistics, Graduate School at ShenzhenTsinghua UniversityShenzhenChina

Personalised recommendations