Image Super Resolution Using Generative Adversarial Networks and Local Saliency Maps for Retinal Image Analysis

  • Dwarikanath MahapatraEmail author
  • Behzad Bozorgtabar
  • Sajini Hewavitharanage
  • Rahil Garnavi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10435)


We propose an image super resolution (ISR) method using generative adversarial networks (GANs) that takes a low resolution input fundus image and generates a high resolution super resolved (SR) image upto scaling factor of 16. This facilitates more accurate automated image analysis, especially for small or blurred landmarks and pathologies. Local saliency maps, which define each pixel’s importance, are used to define a novel saliency loss in the GAN cost function. Experimental results show the resulting SR images have perceptual quality very close to the original images and perform better than competing methods that do not weigh pixels according to their importance. When used for retinal vasculature segmentation, our SR images result in accuracy levels close to those obtained when using the original images.


  1. 1.
  2. 2.
  3. 3.
    Bhatia, K., Price, A., Shi, W., Hajnal, J., Rueckert., D.: Super-resolution reconstruction of cardiac mri using coupled dictionary learning. In: Proceedings of ISBI, pp. 947–950 (2014)Google Scholar
  4. 4.
    Dong, C., Loy, C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Patt. Anal. Mach. Intell. 38(2), 295–307 (2016)CrossRefGoogle Scholar
  5. 5.
    Fraz, M., Remagnino, P., Hoppe, A., Uyyanonvara, B., Rudnicka, A., Owen, C., Barman, S.: An ensemble classification-based approach applied to retinal blood vessel segmentation. IEEE Trans. Biomed. Engg. 59(9), 2538–2548 (2012)CrossRefGoogle Scholar
  6. 6.
    Hoover, A., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imag. 19(3), 203–210 (2000)CrossRefGoogle Scholar
  7. 7.
    Jog, A., Carass, A., Prince, J.L.: Self super-resolution for magnetic resonance images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 553–560. Springer, Cham (2016). doi: 10.1007/978-3-319-46726-9_64 CrossRefGoogle Scholar
  8. 8.
    Ledig, C., et. al.: Photo-realistic single image super-resolution using a generative adversarial network. CoRR abs/1609.04802 (2016)Google Scholar
  9. 9.
    Manjon, J., Coupe, P., Buades, A., Fonov, V., Collins, D., Robles, M.: Non-local MRI upsampling. Med. Image Anal. 14(6), 784–792 (2010)CrossRefGoogle Scholar
  10. 10.
    Oktay, O., et al.: Multi-input cardiac image super-resolution using convolutional neural networks. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 246–254. Springer, Cham (2016). doi: 10.1007/978-3-319-46726-9_29 CrossRefGoogle Scholar
  11. 11.
    Perazzi, F., Kraehenbuhl, P., Pritch, Y., Hornung, A.: Saliency filters: contrast based filtering for salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 733–740 (2012)Google Scholar
  12. 12.
    Rousseau, F.: Brain hallucination. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 497–508. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88682-2_38 CrossRefGoogle Scholar
  13. 13.
    Rueda, A., Malpica, N., Romero, E.: Single-image super-resolution of brain mr images using overcomplete dictionaries. Med. Image Anal. 17(1), 113–132 (2013)CrossRefGoogle Scholar
  14. 14.
    Schulter, S., Leistner, C., Bischof., H.: Fast and accurate image upscaling with super-resolution forests. In: Proceedings of CVPR, pp. 3791–3799 (2015)Google Scholar
  15. 15.
    Simonyan, K., Zisserman., A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)Google Scholar
  16. 16.
    Staal, J., Abramoff, M., Niemeijer, M., Viergever, M., Ginneken, B.V.: Ridge based vessel segmentation in color images of the retina. IEEE Trans. Med. Imag. 23(4), 501–509 (2004)CrossRefGoogle Scholar
  17. 17.
    Tanno, R., Ghosh, A., Grussu, F., Kaden, E., Criminisi, A., Alexander, D.C.: Bayesian image quality transfer. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 265–273. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_31 CrossRefGoogle Scholar
  18. 18.
    Vu, C., Phan, T., Chandler, D.: S3: a spectral and spatial measure of local perceived sharpness in natural images. IEEE Trans. Imag. Proc. 21(3), 934–945 (2012)CrossRefzbMATHGoogle Scholar
  19. 19.
    Wang, Z., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Imag. Proc. 13(4), 600–612 (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Dwarikanath Mahapatra
    • 1
    Email author
  • Behzad Bozorgtabar
    • 1
  • Sajini Hewavitharanage
    • 1
  • Rahil Garnavi
    • 1
  1. 1.IBM Research AustraliaMelbourneAustralia

Personalised recommendations