Representation Learning for Retinal Vasculature Embeddings

  • Luca GiancardoEmail author
  • Kirk Roberts
  • Zhongming Zhao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10554)


The retinal vasculature imaged with fundus photography has the potential of encoding precious information for image-based retinal biomarkers, however, progress in their development is slow due to the need of defining vasculature morphology variables a priori and developing algorithms specific to these variables. In this paper, we introduce a novel approach to learn a general descriptor (or embedding) that captures the vasculature morphology in a numerically compact vector with minimal feature engineering. The vasculature embedding is computed by leveraging the internal representation of a new encoder-enhanced fully convolutional neural network, trained end-to-end with the raw pixels and manually segmented vessels. This approach effectively transfers the vasculature patterns learned by the network into a general purpose vasculature embedding vector. Using Messidor and Messidor-2, two publicly available datasets, we test the vasculature embeddings on two tasks: (1) an image retrieval task, which retrieved similar images according to their vasculature; (2) a diabetic retinopathy classification task, where we show how the vasculature embeddings improve the classification of an algorithm based on microaneurysms detection by 0.04 AUC on average.



This work has been supported by the Center for Precision Health and School of Biomedical Informatics at University of Texas Health Science Center at Houston. We would like to thank Daniele Cortinovis for the initial implementation of U-Net on The Messidor and Messidor-2 datasets are kindly provided by the LaTIM laboratory (see and the Messidor program partners (see”.


  1. 1.
    MacGillivray, T.J., Cameron, J.R., Zhang, Q., El-Medany, A., Mulholland, C., Sheng, Z., Dhillon, B., Doubal, F.N., Foster, P.J., Trucco, E., Sudlow, C.: Suitability of UK biobank retinal images for automatic analysis of morphometric properties of the vasculature. PLoS ONE 10(5), 1–10 (2015)CrossRefGoogle Scholar
  2. 2.
    Trucco, E., Giachetti, A., Ballerini, L., Relan, D., Cavinato, A., MacGillivray, T.: Morphometric measurements of the retinal vasculature in fundus images with vampire. In: Biomedical Image Understanding: Methods and Applications, pp. 91–111 (2015)Google Scholar
  3. 3.
    Fiorin, D., Ruggeri, A.: Computerized analysis of narrow-field ROP images for the assessment of vessel caliber and tortuosity. In: Proceedings of EMBS, pp. 2622–2625 (2011)Google Scholar
  4. 4.
    Xu, X., Niemeijer, M., Song, Q., Sonka, M., Garvin, M.K., Reinhardt, J.M., Abramoff, M.D.: Vessel boundary delineation on fundus images using graph-based approach. IEEE Trans. Med. Imaging 30(6), 1184–1191 (2011)CrossRefGoogle Scholar
  5. 5.
    Azemin, M.Z.C., Kumar, D.K., Wong, T.Y., Kawasaki, R., Mitchell, P., Wang, J.J.: Robust methodology for fractal analysis of the retinal vasculature. IEEE Trans. Med. Imaging 30(2), 243–250 (2011)CrossRefGoogle Scholar
  6. 6.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–828 (2013)CrossRefGoogle Scholar
  7. 7.
    Fu, H., Xu, Y., Lin, S., Kee Wong, D.W., Liu, J.: DeepVessel: Retinal vessel segmentation via deep learning and conditional random field. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 132–139. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_16 CrossRefGoogle Scholar
  8. 8.
    Maninis, K.-K., Pont-Tuset, J., Arbeláez, P., Van Gool, L.: Deep retinal image understanding. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 140–148. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_17 CrossRefGoogle Scholar
  9. 9.
    Pizer, S.M., Amburn, E.P., Austin, J.D., Cromartie, R., Geselowitz, A., Greer, T., Ter Haar Romeny, B., Zimmerman, J.B., Zuiderveld, K.: Adaptive histogram equalization and its variations. Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987)CrossRefGoogle Scholar
  10. 10.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  11. 11.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of CVPR, pp. 3431–3440 (2015)Google Scholar
  12. 12.
    Staal, J., Abramoff, M.D., Niemeijer, M., Viergever, M.A., van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 23(4), 501–509 (2004)CrossRefGoogle Scholar
  13. 13.
    Decencière, E., Zhang, X., Cazuguel, G., Laÿ, B., Cochener, B., Trone, C., Gain, P., Ordóñez-Varela, J.R., Massin, P., Erginay, A., Charton, B., Klein, J.C.: Feedback on a publicly distributed image database: The Messidor database. Image Anal. Stereology 33(3), 231–234 (2014)CrossRefzbMATHGoogle Scholar
  14. 14.
    Giancardo, L., Meriaudeau, F., Karnowski, T.P., Tobin, K.W., Chaum, E.: Validation of microaneurysm-based diabetic retinopathy screening across retina fundus datasets. In: Proceedings of CBMS, pp. 125–130 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Luca Giancardo
    • 1
    • 2
    Email author
  • Kirk Roberts
    • 1
  • Zhongming Zhao
    • 1
    • 2
  1. 1.School of Biomedical InformaticsUniversity of Texas Health Science Center at Houston (UTHealth)HoustonUSA
  2. 2.Center for Precision HealthUTHealthHoustonUSA

Personalised recommendations