Skip to main content
SpringerLink
Log in

Learning multi-scale deep fusion for retinal blood vessel extraction in fundus images

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Segmentation of retinal blood vessels is crucial in the automated diagnosis of many retinal and cardiovascular diseases. The process of precise vessel extraction using fundus images is still a challenge due to spatially varying vessel-width and non-homogeneous retinal backgrounds. This work targets the challenges mentioned above with an adaptive multi-scale decomposition of the input image and a novel characteristic patch-based deep network training. In order to enhance vessels of different widths, we use the observed field of view of the input image to estimate the most significant scales for Gabor decomposition. Enhanced vessel maps corresponding to real, imaginary, and absolute coefficients at the estimated scales are linearly combined using a trainable \(1\times 1\) convolutional layer of U-net. Moreover, the ‘characteristic patch-based training’ uses ‘random’ and ‘specific’ patches to learn vessels in non-homogeneous retinal backgrounds. The proposed algorithm minimizes false negatives and extracts promising vessel maps in various challenging regions of the retina. The significant improvement in accuracy, sensitivity and AUC compared to other state-of-the-art values proves the proposed method’s outstanding performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Chatziralli, I.P., Kanonidou, E.D., Keryttopoulos, P., Dimitriadis, P., Papazisis, L.E.: The value of fundoscopy in general practice. Open Ophthalmol. J. 6, 4 (2012)

    Article  Google Scholar 

  2. Hoover, A.D., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imaging 19(3), 203–210 (2000)

    Article  Google Scholar 

  3. Zhang, B., Zhang, L., Zhang, L., Karray, F.: Retinal vessel extraction by matched filter with first-order derivative of Gaussian. Comput. Biol. Med. 40(4), 438–445 (2010)

    Article  Google Scholar 

  4. Odstrcilik, J., Kolar, R., Budai, A., Hornegger, J., Jan, J., Gazarek, J., Kubena, T., Cernosek, P., Svoboda, O., Angelopoulou, E.: Retinal vessel segmentation by improved matched filtering: evaluation on a new high-resolution fundus image database. IET Image Process. 7(4), 373–383 (2013)

    Article  MathSciNet  Google Scholar 

  5. Roychowdhury, S., Koozekanani, D.D., Parhi, K.K.: Iterative vessel segmentation of fundus images. IEEE Trans. Biomed. Eng. 62(7), 1738–1749 (2015)

    Article  Google Scholar 

  6. Fraz, M.M., Basit, A., Barman, S.A.: Application of morphological bit planes in retinal blood vessel extraction. J. Digit. Imaging 26(2), 274–286 (2013)

    Article  Google Scholar 

  7. Soares, J.V., Leandro, J.J., Cesar, R.M., Jelinek, H.F., Cree, M.J.: Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification. IEEE Trans. Med. Imaging 25(9), 1214–1222 (2006)

    Article  Google Scholar 

  8. Upadhyay, K., Agrawal, M., Vashist, P.: Wavelet based fine-to-coarse retinal blood vessel extraction using U-net model. In: 2020 International Conference on Signal Processing and Communications (SPCOM), pp. 1–5. IEEE (2020)

  9. Delibasis, K.K., Kechriniotis, A.I., Tsonos, C., Assimakis, N.: Automatic model-based tracing algorithm for vessel segmentation and diameter estimation. Comput. Methods Progr. Biomed. 100(2), 108–122 (2010)

    Article  Google Scholar 

  10. Remeseiro, B., Mendonça, A.M., Campilho, A.: Automatic classification of retinal blood vessels based on multilevel thresholding and graph propagation. Vis. Comput. 37(6), 1247–1261 (2021)

    Article  Google Scholar 

  11. Al-Diri, B., Hunter, A., Steel, D.: An active contour model for segmenting and measuring retinal vessels. IEEE Trans. Med. Imaging 28(9), 1488–1497 (2009)

    Article  Google Scholar 

  12. Fraz, M.M., Remagnino, P., Hoppe, A., Uyyanonvara, B., Rudnicka, A.R., Owen, C.G., Barman, S.A.: An ensemble classification-based approach applied to retinal blood vessel segmentation. IEEE Trans. Biomed. Eng. 59(9), 2538–2548 (2012)

    Article  Google Scholar 

  13. Ricci, E., Perfetti, R.: Retinal blood vessel segmentation using line operators and support vector classification. IEEE Trans. Med. Imaging 26(10), 1357–1365 (2007)

    Article  Google Scholar 

  14. Marín, D., Aquino, A., Gegúndez-Arias, M.E., Bravo, J.M.: A new supervised method for blood vessel segmentation in retinal images by using gray-level and moment invariants-based features. IEEE Trans. Med. Imaging 30(1), 146–158 (2010)

    Article  Google Scholar 

  15. Sreejini, K.S., Govindan, V.K.: Improved multiscale matched filter for retina vessel segmentation using PSO algorithm. Egypt. Inform. J. 16(3), 253–260 (2015)

    Article  Google Scholar 

  16. Wang, S., Yin, Y., Cao, G., Wei, B., Zheng, Y., Yang, G.: Hierarchical retinal blood vessel segmentation based on feature and ensemble learning. Neurocomputing 149, 708–717 (2015)

    Article  Google Scholar 

  17. Xue, D.X., Zhang, R., Feng, H., Wang, Y.L.: CNN-SVM for microvascular morphological type recognition with data augmentation. J. Med. Biol. Eng. 36(6), 755–764 (2016)

    Article  Google Scholar 

  18. Liskowski, P., Krawiec, K.: Segmenting retinal blood vessels with deep neural networks. IEEE Trans. Med. Imaging 35(11), 2369–2380 (2016)

    Article  Google Scholar 

  19. Lyu, C., Hu, G., Wang, D.: Attention to fine-grained information: hierarchical multi-scale network for retinal vessel segmentation. Vis. Comput. 19(1), 1–11 (2020)

  20. Lahiri, A., Ayush, K., Kumar Biswas, P., Mitra, P.: Generative adversarial learning for reducing manual annotation in semantic segmentation on large scale miscroscopy images: automated vessel segmentation in retinal fundus image as test case. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 42–48 (2017)

  21. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer, Cham (2015)

  22. Li, X., Chen, H., Qi, X., Dou, Q., Fu, C.W., Heng, P.A.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37(12), 2663–2674 (2018)

    Article  Google Scholar 

  23. Guan, S., Khan, A.A., Sikdar, S., Chitnis, P.V.: Fully dense UNet for 2-D sparse photoacoustic tomography artifact removal. IEEE J. Biomed. Health Inform. 24(2), 568–576 (2019)

    Article  Google Scholar 

  24. Wang, D., Hu, G., Lyu, C.: Frnet: an end-to-end feature refinement neural network for medical image segmentation. Vis. Comput. 37(5), 1101–1112 (2021)

    Article  Google Scholar 

  25. Saranya, P., Prabakaran, S., Kumar, R., Das, E.: Blood vessel segmentation in retinal fundus images for proliferative diabetic retinopathy screening using deep learning. Vis. Comput. 1–16 (2021)

  26. Huang, L., Liu, F.: Retinal vessel segmentation using simple SPCNN model and line connector. Vis. Comput. 1–14 (2020)

  27. Nagashree, N., Patil, P., Patil, S., Kokatanur, M.: Alpha beta pruned UNet-a modified UNet framework to segment MRI brain image to analyse the effects of CNTNAP2 gene towards autism detection. In: 2021 3rd International Conference on Computer Communication and the Internet (ICCCI), pp. 23–26. IEEE (2021)

  28. Zhang, Y., Chung, A.: Deep supervision with additional labels for retinal vessel segmentation task. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 83–91. Springer, Cham (2018)

  29. Jin, Q., Meng, Z., Pham, T.D., Chen, Q., Wei, L., Su, R.: DUNet: a deformable network for retinal vessel segmentation. Knowl. Based Syst. 178, 149–162 (2019)

    Article  Google Scholar 

  30. Li, L., Verma, M., Nakashima, Y., Nagahara, H. , Kawasaki, R.: Iternet: retinal image segmentation utilizing structural redundancy in vessel networks. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3656–3665 (2020)

  31. Kim, J.U., Kim, H.G., Ro, Y.M.: Iterative deep convolutional encoder-decoder network for medical image segmentation. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 685–688. IEEE (2017)

  32. Niemeijer, M., Staal, J., van Ginneken, B., Loog, M., Abramoff, M.D.: Comparative study of retinal vessel segmentation methods on a new publicly available database. In: Medical Imaging 2004: Image Processing, vol. 5370, pp. 648–656. International Society for Optics and Photonics (2004)

  33. Upadhyay, K., Agrawal, M., Vashist, P.: Unsupervised multiscale retinal blood vessel segmentation using fundus images. IET Image Process. 14(11), 2616–2625 (2020)

    Article  Google Scholar 

  34. Pisano, E.D., Zong, S., Hemminger, B.M., DeLuca, M., Johnston, R.E., Muller, K., Braeuning, M.P., Pizer, S.M.: Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms. J. Digit. Imaging 11(4), 193–200 (1998)

    Article  Google Scholar 

  35. Upadhyay, K., Agrawal, M., Vashist, P.: U-Net based multi-level texture suppression for vessel segmentation in low contrast regions. In: 2020 28th European Signal Processing Conference (EUSIPCO), pp. 1304–1308. IEEE January (2021)

  36. Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv preprint arXiv:1802.06955 (2018)

  37. Orlando, J.I., Prokofyeva, E., Blaschko, M.B.: A discriminatively trained fully connected conditional random field model for blood vessel segmentation in fundus images. IEEE Trans. Biomed. Eng. 64(1), 16–27 (2016)

    Article  Google Scholar 

  38. Yan, Z., Yang, X., Cheng, K.T.: Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation. IEEE Trans. Biomed. Eng. 65(9), 1912–1923 (2018)

    Article  Google Scholar 

  39. Azzopardi, G., Strisciuglio, N., Vento, M., Petkov, N.: Trainable COSFIRE filters for vessel delineation with application to retinal images. Med. Image Anal. 19(1), 46–57 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Applied Research in Electronics, IIT Delhi, New Delhi, India

    Kamini Upadhyay & Monika Agrawal

  2. Community Ophthalmology, Dr RP Centre for Ophthalmic Sciences, AIIMS, New Delhi, India

    Praveen Vashist

Authors
  1. Kamini Upadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Monika Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Praveen Vashist
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kamini Upadhyay.

Ethics declarations

Conflict of Interest

No benefits in any form have been or will be received from a commercial party related directly or indirectly to the subject of this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Upadhyay, K., Agrawal, M. & Vashist, P. Learning multi-scale deep fusion for retinal blood vessel extraction in fundus images. Vis Comput 39, 4445–4457 (2023). https://doi.org/10.1007/s00371-022-02600-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02600-4

Keywords

Search

Navigation