Skip to main content
SpringerLink
Log in

Retinal Vessel Segmentation Using Multi-Scale Residual Convolutional Neural Network (MSR-Net) Combined with Generative Adversarial Networks

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

Retinal fundus images provide valuable diagnostic and clinical information in the diagnosis of ophthalmologic diseases. Retinal blood vessel analysis provides important diagnostic information about thinning of the retinal nerve fiber layer and alteration in the structural appearance of the optic nerve head. Here, an accurate retinal vessel detection method is proposed from fundus images using a generative adversarial network (GAN) utilizing multiple loss functions. The proposed GAN architecture consists of the generator as a segmentation network and the discriminator as a classification network. The generator is a multi-scale residual convolutional neural network with skip connection and up-sampling, while the discriminator is a vision transformer that acts as a binary classifier. The inception module extracts multi-scale features of vessel segments from different scales and captures fine vessel segments. The discriminator consists of stacked self-attention networks and position-wise fully connected feed-forward networks inferring two-class output. The attention mechanism in the transformer is competent to preserve both global and local information while acting as a discriminator. The proposed GAN model segments the blood vessels more accurately through the adversarial learning process to produce state-of-the-art results. In the preprocessing stage, the contrast of blood vessels is enhanced by contrast-limited adaptive histogram equalization algorithm. The robustness and efficacy of the proposed method have been evaluated on publicly available DRIVE, STARE, CHASE_DB1, HRF, ARIA, IOSTAR, and RC-SLO databases. Different performance measures like accuracy, sensitivity, precision, intersection of union, and F1Score are adopted to compare the proposed method with the existing methods available in the literature. The proposed method attains an accuracy of 0.9873 for CHASE_DB1 database, 0.9742 for DRIVE database, 0.9773 for HRF database, and 0.9628 for ARIA database.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Data availability

The authors confirm that the data supporting the findings of this study are available from the corresponding author upon request. The programs and the supporting files will be provided on request.

References

  1. M.D. Abràmoff, M.K. Garvin, M. Sonka, Retinal imaging and image analysis. IEEE Rev. Biomed. Eng. 3, 169–208 (2010)

    Article  Google Scholar 

  2. H. Akil, A.S. Huang, B.A. Francis, S.R. Sadda, V. Chopra, Retinal vessel density from optical coherence tomography angiography to differentiate early glaucoma, pre-perimetric glaucoma and normal eyes. PLoS One (2017)

  3. J. Ba, J.R. Kiros, G.E. Hinton, Layer normalization. arXiv:1607.06450 (2016)

  4. K. Beom, S.H. Choi, J.Y. Lee, M-GAN: Retinal blood vessel segmentation by balancing losses through stacked deep fully convolutional networks. in IEEE Access (2020)

  5. A. Budai, R. Bock, A. Maier, J. Hornegger, G. Michelson, Robust vessel segmentation in fundus images. Int. J. Biomed. Imaging (2013)

  6. D. Chen, W. Yang, L. Wang, S. Tan, J. Lin, W. Bu, PCAT-UNet: UNet-like network fused convolution and transformer for retinal vessel segmentation. PLOS ONE (2022)

  7. L. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, Encoder-Decoder with atrous separable convolution for semantic image segmentation (2018). arXiv:1802.02611

  8. B. Dashtbozorg, J. Zhang, F. Huang, B.M. ter Haar Romeny, Retinal microaneurysms detection using local convergence index features. IEEE Trans. Image Process. 27(7), 3300–3315 (2018). https://doi.org/10.1109/TIP.2018.2815345

    Article  MathSciNet  MATH  Google Scholar 

  9. X. Deng, J. Ye, A retinal blood vessel segmentation based on improved D-MNet and pulse-coupled neural network. Biomedical Signal Processing and Control 73, 103467 (2022). https://doi.org/10.1016/j.bspc.2021.103467

    Article  Google Scholar 

  10. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, N. Houlsby: An image is worth 16 \(\times \) 16 words: transformers for image recognition at scale (2020). arXiv:2010.11929

  11. P. Elangovan, M.K. Nath, Glaucoma assessment from color fundus images using convolutional neural network. Int. J. Imaging Syst. Technol. (2020). https://doi.org/10.1002/ima.22494

  12. H. Emami, M. Dong, S. Nejad-Davarani, C. Glide-Hurst, Generating synthetic CTs from magnetic resonance images using generative adversarial networks. Med. Phys. (2018). https://doi.org/10.1002/mp.13047

  13. D. Farnell, F. Hatfield, P. Knox, M. Reakes, S. Spencer, D. Parry, S. Harding, Enhancement of blood vessels in digital fundus photographs via the application of multi-scale line operators. J. Frankl. Inst. 345(7), 748–765 (2008). https://doi.org/10.1016/j.jfranklin.2008.04.009

    Article  MATH  Google Scholar 

  14. M.M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A.R. Rudnicka, C.G. Owen, S.A. Barman, An ensemble classification-based approach applied to retinal blood vessel segmentation. IEEE Trans. Biomed. Eng. (2012)

  15. H. Fu, Y. Xu, S. Lin, D.W.K. Wong, J. Liu, Deepvessel: retinal vessel segmentation via deep learning and conditional random field. In: International conference on medical image computing and computer- assisted intervention (2016)

  16. I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks (2014)

  17. Z. Gu, J. Cheng, H. Fu, K. Zhou, H. Hao, Y. Zhao, T. Zhang, S. Gao, J. Liu, CE-Net: context encoder network for 2D medical image segmentation. IEEE Trans. Med. Imaging (2019)

  18. X. Guo, C. Chen, Y. Lu, K. Meng, H. Chen, K. Zhou, Z. Wang, R. Xiao, Retinal vessel segmentation combined with generative adversarial networks and dense UNet. IEEE Access (2020)

  19. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

  20. J. Hoffman, E. Tzeng, T. Park, , J.Y. Zhu, P. Isola, K. Saenko, A. Efros, T. Darrell, Cycada: cycle-consistent adversarial domain adaptation. in International Conference on Machine Learning, pp. 1989–1998 (2018)

  21. A. Hoover, V. Kouznetsova, M. Goldbaum, Locating blood vessels in retinal images by piece-wise threhsold probing of a matched filter response. IEEE Trans. Med. Imaging, pp. 203–210 (2000)

  22. K. Hu, Z. Zhang, X. Niu, Y. Zhang, C. Cao, F. Xiao, X. Gao, Retinal vessel segmentation of color fundus images using multi-scale convolutional neural network with an improved cross-entropy loss function. J. Neurocomput. (2018)

  23. Q. Jin, Z. Meng, T.D. Pham, Q. Chen, , L. Wei, R. Su, DUNet: a deformable network for retinal vessel segmentation. Knowl.-Based Syst. (2019)

  24. M. Kar, M. Nath, M. Mishra, Retinal vessel segmentation and disc detection from color fundus images using inception module and residual connection. in 3rd International Conference On Recent Trends In Advanced Computing (2020)

  25. J. Kohler, A. Budai, M.F. Kraus, J. Odstrcilik, G. Michelson, J. Hornegger, Automatic no-reference quality assessment for retinal fundus images using vessel segmentation. in Proceedings of the 26th IEEE International Symposium on Computer-based Medical Systems (2013)

  26. J. Li, X. Liang, Y. Wei, T. Xu, J. Feng, S. Yan, Perceptual generative adversarial networks for small object detection (2017). arXiv:1706.05274

  27. X. Li, Z. Du, Y. Huang, Z. Tan, A deep translation (GAN) based change detection network for optical and SAR remote sensing images. ISPRS J. Photogrammetry Remote Sens. 179, 14–34 (2021). https://doi.org/10.1016/j.isprsjprs.2021.07.007

    Article  Google Scholar 

  28. S. Moccia, E.D. Momi, S.E. Hadji, L.S Mattos, Blood vessel segmentation algorithms: review of methods, datasets and evaluation metrics. Comput. Methods Programs Biomed. (2018)

  29. M. Niemeijer, B. Ginneken, M. Loog, Comparative study of retinal vessel segmentation methods on a new publicly available database, in Proceedings of SPIE - The International Society for Optical Engineering, (2004)

  30. C.G. Owen, A.R. Rudnicka, R. Mullen, S.A. Barman, D.N. Monekosso, P.H. Whincup, J. Ng, C. Paterson, Measuring retinal vessel tortuosity in 10-year-old children: validation of the computer-assisted image analysis of the retina (CAIAR) program. Investig. Ophthalmol. Visual Sci. 50(5), 2004–10 (2009)

    Article  Google Scholar 

  31. K.B. Park, S.H. Choi, J.Y. Lee, M-GAN: Retinal blood vessel segmentation by balancing losses through stacked deep fully convolutional networks. in IEEE Access (2020)

  32. S.M. Pizer, E.P. Amburn, , J.D. Austin, R. Cromartie, A. Geselowitz, T. Greer, B.M. ter Haar Romeny, J.B. Zimmerman, Adaptive histogram equalization and its variations. Comput. Vis. Graphics. Image Process. 39, 355–368 (1987)

  33. O. Ronneberger, P. Fischer, T. Brox, UNet: Convolutional networks for biomedical image segmentation. in International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 1–8 (2017)

  34. K. Santosh, S. Ghosh, M. Bose, Ret-GAN: Retinal image enhancement using generative adversarial networks, pp. 79–84 (2021). https://doi.org/10.1109/CBMS52027.2021.00082

  35. R.A. Shehhi, P.R. Marpu, W.L. Woon, An automatic cognitive graph-based segmentation for detection of blood vessels in retinal images. Math. Probl. Eng. (2016). https://doi.org/10.1155/2016/7906165

    Article  MathSciNet  MATH  Google Scholar 

  36. S.Y. Shin, S. Lee, I.D. Yun, K.M. Lee, Deep vessel segmentation by learning graphical connectivity. Med. Image Anal. 58, 1–14 (2019)

    Article  Google Scholar 

  37. J. Son, S.J. Park, K.H. Jung, Retinal vessel segmentation in fundoscopic images with generative adversarial networks. arXiv:1706.09318 (2017)

  38. E. Uysal, G.E. Guraksin, Computer-aided retinal vessel segmentation in retinal images: convolutional neural networks. Multimedia Tools Appl., pp. 1929–1958 (2020)

  39. A. Vaswani, N. Shazeer, , N. Parmar, , J. Uszkoreit, , L. Jones, A.N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need. (2017). arXiv:1706.03762

  40. D. Vijayalakshmi, M.K. Nath, O.P. Acharya, A comprehensive survey on image contrast enhancement techniques in spatial domain. Sens. Imaging 21 (2020). https://doi.org/10.1007/s11220-020-00305-3

  41. Y. Xue, T. Xu, H. Zhang, L.R. Long, X. Huang, SegAN: Adversarial network with multi-scale L1 loss for medical image segmentation. Neuroinformatics, pp. 383–392 (2018)

  42. Z. Yan, X. Yang, K.T. Cheng, Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation. IEEE Trans. Biomed. Eng., pp. 1912–1923 (2018)

  43. X. Yu, X. Cai, Z. Ying, T.H. Li, G. Li, SingleGAN: image-to-Image translation by a single-generator network using multiple generative adversarial learning. (2018). arXiv:1810.04991

  44. H. Zhang, I. Goodfellow, D. Metaxas, A. Odena, Self-attention generative adversarial networks (2019)

  45. J. Zhang, B. Dashtbozorg, E. Bekkers, J.P.W. Pluim, R. Duits, B.M.T.H. Romeny, Retinal vessel segmentation via locally adaptive derivative frames in orientation scores. IEEE Trans Med Imaging (2016)

  46. Y. Zhang, M. He, Z. Chen, K. Hu, X. Li, X. Gao, Bridge-Net: context-involved UNet with patch-based loss weight mapping for retinal blood vessel segmentation. Expert Syst. Appl. 195, 116526 (2022) https://doi.org/10.1016/j.eswa.2022.116526

  47. Y. Zhao, Y. Liu, X. Wu, S.P. Harding, Y. Zheng, Retinal vessel segmentation: an efficient graph cut approach with retinex and local phase. PLoS ONE 10(4), 1–22 (2015). https://doi.org/10.1371/journal.pone.0122332

    Article  Google Scholar 

  48. Z. Zhou, M.M.R. Siddiquee, N. Tajbakhsh, J. Liang, UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging. (2020)

  49. J. Zhu, T. Park, P. Isola, A.A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks (2017). arXiv:1703.10593

Download references

Acknowledgements

This work is carried out by the authors at National Institute of Technology Puducherry, Karaikal, India. IIT Guwahati (India) has supported for retinal database.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology Puducherry, Karaikal, 609609, India

    Mithun Kumar Kar & Malaya Kumar Nath

  2. Mehta Family School of Data Science and Artificial Intelligence, Indian Institute of Technology Guwahati, Guwahati, 781039, India

    Debanga Raj Neog

Authors
  1. Mithun Kumar Kar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debanga Raj Neog
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Malaya Kumar Nath
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mithun Kumar Kar.

Ethics declarations

Conflict of interest

There is no conflicts of interest.

Ethical approval

Our paper satisfies and fulfills the compliance with ethical standards. The research does not involve human participants and/or animals.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kar, M.K., Neog, D.R. & Nath, M.K. Retinal Vessel Segmentation Using Multi-Scale Residual Convolutional Neural Network (MSR-Net) Combined with Generative Adversarial Networks. Circuits Syst Signal Process 42, 1206–1235 (2023). https://doi.org/10.1007/s00034-022-02190-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-022-02190-5

Keywords

Search

Navigation