Abstract
In the segmentation of retinal vessel, the details of retinal vessel cannot be segmented accurately, and the single-pixel fine retinal vessels were prone to be mistakenly recognized by the unsupervised and supervised approaches. In order to solve this problem, we proposed novel fine retinal vessel segmentation by combining Nest U-net and patch-learning in this study. The special extraction strategy was designed to effectively generate massive training samples including fine retinal vessels, and these samples had a great advantage in the fine retinal vessel segmentation. Nest U-net which directly fast-forwarded high-resolution feature maps from the encoder to the decoder network was designed as a new image segmentation model. This model was trained by the k-fold cross-validation strategy, and the testing samples were predicted, and the final retinal vessel was reconstructed by the sequential reconstruction strategy. This proposed method was tested on the publicly available datasets DRIVE and STARE. Sensitivity (SE), specificity (SP), accuracy (ACC), area under each curve (AUC), F1-score, and jaccard similarity score (JSC) were adopted as evaluation metrics to prove the superiority of this proposed method. The results that we achieved on public datasets (DRIVE: SE = 0.8060, SP = 0.9869, ACC = 0.9512, AUC = 0.9748, F1-score = 0.7863; STARE: SE = 0.8230, SP = 0.9945, ACC = 0.9641, AUC = 0.9620, F1-score = 0.7947) were higher than other state-of-the-art methods. This proposed method can achieve state-of-the-art segmentation results in terms of visual quality and objective assessment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abràmoff MD, Garvin MK, Sonka M (2010) Retinal imaging and image analysis. IEEE Rev Biomed Eng 3:169–208. https://doi.org/10.1109/RBME.2010.2084567
Azzopardi G, Strisciuglio N, Vento M, Petkov N (2015) Trainable COSFIRE filters for vessel delineation with application to retinal images. Med Image Anal 19(1):46–57. https://doi.org/10.1016/j.media.2014.08.002
Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828. https://doi.org/10.1109/TPAMI.2013.50
Brancati N, Frucci M, Gragnaniello D, Riccio D (2017) Retinal vessels segmentation based on a convolutional neural network. Progress Pattern Recognit Image Anal Comput Vision Appl. https://doi.org/10.1007/978-3-319-75193-1_15
Dashtbozorg B, Bekkers E, Pluim JP, Duits R, TerHaar Romeny BM, Zhang J (2016) Robust retinal vessel segmentation via locally adaptive derivative frames in orientation scores. IEEE Trans Med Imaging 35:2631–2644. https://doi.org/10.3788/AOS202040.0410002
Frangi AF, Niessen WJ, Vincken KI, Viergever MA (1998) Multiscale vessel enhancement filtering. In: Medical image computing and computer assisted intervention, Berlin, Germany, pp 130–137. https://doi.org/10.1007/BFb0056195
Fraz MM, Remagnino P, Hoppe A, Uyyanonvara B, Rudnicka AR, Owen CG, Barman SA (2012) An Ensemble classification-based approach applied to retinal blood vessel segmentation. IEEE Trans Biomed Eng 59(9):2538–2548. https://doi.org/10.1109/TBME.2012.2205687
Guo YH, Budak U, Vespa LJ, Khorasania E, Sengür A (2018) A retinal vessel detection approach using convolution neural network with reinforcement sample learning strategy. Measurement 125:586–591. https://doi.org/10.1016/j.measurement.2018.05.003
Hoover A, Kouznetsova V, Goldbaum M (2000) Locating blood vessels in retinal images by piece-wise threshold probing of a matched filter response. IEEE Trans Med Imaging 19(3):203–210. https://doi.org/10.1109/42.845178
Hu K, Zhang ZZ, Niu XR, Zhang Y, Cao CH, Xiao F, Gao XP (2018) Retinal vessel segmentation of color fundus images using multiscale convolutional neural network with an improved cross-entropy loss function. Neurocomputing 309:179–191. https://doi.org/10.1016/j.neucom.2018.05.011
Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: fully convolutional DenseNets for semantic segmentation. In: IEEE conference on computer vision and pattern recognition workshops, pp 1175–1183. https://doi.org/10.1109/CVPRW.2017.156
Jiang XY, Mojon D (2003) Adaptive local thresholding by verification-based multi-threshold probing with application to vessel detection in retinal images. IEEE Trans Pattern Anal Mach Intell 25(1):131–137. https://doi.org/10.1109/TPAMI.2003.1159954
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: International conference on neural information processing systems, 60(2):1097–1105.https://doi.org/10.1145/3065386
Liang MM, Shen LD, Wang YX, Zheng LY (2017) Eye fundus vessel segmentation technology based on SVM. Appl Sci Technol 44(3):67–71
Liang LM, Huang CL, Shi F, Wu J, Jiang HJ, Chen XJ (2018) Retinal vessel segmentation using level set combined with shape priori. Chin J Comput 41(7):1678–1692. https://doi.org/10.11897/SP.J.1016.2018.01678
Long J, Shelhamer E, Darrell T (2015) Fully Convolutional Networks for Semantic Segmentation. In: IEEE conference of computer vision and pattern recognition, pp 3430–3440. https://doi.org/10.1109/TPAMI.2016.2572683
Marín D, Aquino A, Gegundez-Arias ME, Bravo JM (2011) A new supervised method for blood vessel segmentation in retinal images by using gray-level and moment invariants-based features. IEEE Trans Med Imaging 30(1):146–158. https://doi.org/10.1109/TMI.2010.2064333
Milletari F, Navab N, Ahmadi SA (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In Proceedings of the fourth international conference on 3D vision, pp 565–571. https://doi.org/10.1109/3DV.2016.79.
Niemeijer M, Staal J, Van Ginneken B, Loog M, Abramoff MD (2004) Comparative study of retinal vessel segmentation methods on a new publicly available database. Proc SPIE Int Soc Opt Eng 5370:648–656. https://doi.org/10.1117/12.535349
Oliveira AFM, Pereira SRM, Silva CAB (2018) Retinal vessel segmentation based on fully convolutional neural networks. Expert Syst Appl 112:229–242. https://doi.org/10.1016/j.eswa.2018.06.034
Orlando JI, Prokofyeva E, Blaschko MB (2017) A discriminatively trained fully connected conditional random field model for blood vessel segmentation in fundus images. IEEE Trans Biomed Eng 64(1):16–27. https://doi.org/10.1109/TBME.2016.2535311
Ricci E, Perfetti R (2007) Retinal blood vessel segmentation using line operators and support vector classification. IEEE Trans Med Imaging 26(10):1357–1365. https://doi.org/10.1109/TMI.2007.898551
Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241. https://doi.org/10.1007/978-3-319-24574-4_28.
Sato Y, Nakaiima S, Atsumi H, Koller T, Gerig G, Yoshida S, Kikinis R (1998) 3D multi-scale line filter for segmentation and visualization of curvilinear structures in medical images. Med Image Anal 2(2):143–168. https://doi.org/10.1007/BFb0029240
Soares JVB, Leandro JJG, Cesar RM, Jelinek HF, Cree MJ (2006) Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification. IEEE Trans Med Imaging 25(9):1214–1222. https://doi.org/10.1109/TMI.2006.879967
Staal JJ, Abramoff MD, Niemeijer M, Viergever MA, Van Ginneken B (2004) Ridge based vessel segmentation in color images of the retina. IEEE Trans Med Imaging 23(4):501–509. https://doi.org/10.1109/tmi.2004.825627
Wang S, Yin Y, Cao G, Wei B, Zheng Y, Yang G (2015) Hierarchical retinal blood vessel segmentation based on feature and ensemble learning. Neurocomputing 149:708–717. https://doi.org/10.1016/j.neucom.2014.07.059
Wang C, Zhao ZY, Ren QQ, Xu YT, Yu Y (2019) Dense U-net based on patch-based learning for retinal vessel segmentation. Entropy 21(2):168. https://doi.org/10.3390/e21020168
Zhang J, Chen Y, Bekkers E, Wang ML, Dashtbozorg B, Ter Haar Romeny BM (2017) Retinal vessel delineation using a brain-inspired wavelet transform and random forest. Pattern Recogn 69:107–123. https://doi.org/10.1016/j.patcog.2017.04.008
Zhao Y, Rada L, Chen K, Harding SP, Zheng Y (2015) Automated vessel segmentation using infinite perimeter active contour model with hybrid region information with application to retinal images. IEEE Trans Med Imaging 34(9):1797–1807. https://doi.org/10.1109/TMI.2015.2409024
Zhou ZW, Rahman Siddiquee MM, Tajbakhsh N, Liang JM (2018) UNet++: a nested U-net architecture for medical image segmentation. Comput Vis Pattern Recognit. https://doi.org/10.1007/978-3-030-00889-5_1
Funding
This work was supported by the Scientific and Technological Project of Henan Province (Grants nos. 202102310536, 192102310455, and 172102310541), the Disciplinary group of Psychology and Neuroscience, Xinxiang Medical University (2016PN-KFKT-19, 2016PN-KFKT-24), and the Research Project on educational and teaching reform, Xinxiang Medical University (2019-XYJG-21, 2019-XYJG-68).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, C., Zhao, Z. & Yu, Y. Fine retinal vessel segmentation by combining Nest U-net and patch-learning. Soft Comput 25, 5519–5532 (2021). https://doi.org/10.1007/s00500-020-05552-w
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-020-05552-w