TopNet: Topology Preserving Metric Learning for Vessel Tree Reconstruction and Labelling

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12266)


Reconstructing Portal Vein and Hepatic Vein trees from contrast enhanced abdominal CT scans is a prerequisite for preoperative liver surgery simulation. Existing deep learning based methods treat vascular tree reconstruction as a semantic segmentation problem. However, vessels such as hepatic and portal vein look very similar locally and need to be traced to their source for robust label assignment. Therefore, semantic segmentation by looking at local 3D patch results in noisy misclassifications. To tackle this, we propose a novel multi-task deep learning architecture for vessel tree reconstruction. The network architecture simultaneously solves the task of detecting voxels on vascular centerlines (i.e. nodes) and estimates connectivity between center-voxels (edges) in the tree structure to be reconstructed. Further, we propose a novel connectivity metric which considers both inter-class distance and intra-class topological distance between center-voxel pairs. Vascular trees are reconstructed starting from the vessel source using the learned connectivity metric using the shortest path tree algorithm. A thorough evaluation on public IRCAD dataset shows that the proposed method considerably outperforms existing semantic segmentation based methods. To the best of our knowledge, this is the first deep learning based approach which learns multi-label tree structure connectivity from images.


Deep learning Vessel tree reconstruction Vessel segmentation Liver vessel Centerline detection Computed tomography 



This research was done in cooperation with the Hepato-Biliary-Pancreatic Surgery Division, the University of Tokyo Hospital. We would like to thank Prof. Kiyoshi Hasegawa, Dr. Junichi Kaneko, Dr. Ryugen Takahashi, and Dr. Yusuke Kazami for their valuable advice and curating a liver vessel dataset.

Supplementary material

505219_1_En_2_MOESM1_ESM.pdf (582 kb)
Supplementary material 1 (pdf 582 KB)


  1. 1.
    Bauer, C., Pock, T., Sorantin, E., Bischof, H., Beichel, R.: Segmentation of interwoven 3D tubular tree structures utilizing shape priors and graph cuts. Med. Image Anal. 14(2), 172–184 (2010)CrossRefGoogle Scholar
  2. 2.
    Center, M.M., Jemal, A.: International trends in liver cancer incidence rates. Cancer Epidemiol. Prev. Biomark. 20(11), 2362–2368 (2011)CrossRefGoogle Scholar
  3. 3.
    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). Scholar
  4. 4.
    De Brabandere, B., Neven, D., Van Gool, L.: Semantic instance segmentation with a discriminative loss function (2017). arXiv preprint arXiv:1708.02551
  5. 5.
    Fathi, A., et al.: Semantic instance segmentation via deep metric learning (2017). arXiv preprint arXiv:1703.10277
  6. 6.
    Fraz, M.M., et al.: Blood vessel segmentation methodologies in retinal images-a survey. Comput. Methods Programs Biomed. 108(1), 407–433 (2012)CrossRefGoogle Scholar
  7. 7.
    Gegúndez-Arias, M.E., Aquino, A., Bravo, J.M., Marín, D.: A function for quality evaluation of retinal vessel segmentations. IEEE Trans. Med. Imaging 31(2), 231–239 (2011)CrossRefGoogle Scholar
  8. 8.
    Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)Google Scholar
  9. 9.
    Ibragimov, B., Toesca, D., Chang, D., Koong, A., Xing, L.: Combining deep learning with anatomical analysis for segmentation of the portal vein for liver sbrt planning. Phys. Med. Biol. 62(23), 8943 (2017)CrossRefGoogle Scholar
  10. 10.
    IRCAD: Ircad dataset for liver vessel segmentation, March 2020.
  11. 11.
    Kazaryan, A.M., et al.: Laparoscopic resection of colorectal liver metastases: surgical and long-term oncologic outcome. Ann. Surg. 252(6), 1005–1012 (2010)CrossRefGoogle Scholar
  12. 12.
    Keshwani, D., Kitamura, Y., Li, Y.: Computation of total kidney volume from CT images in autosomal dominant polycystic Kidney disease using multi-task 3D convolutional neural networks. In: Shi, Y., Suk, H.-I., Liu, M. (eds.) MLMI 2018. LNCS, vol. 11046, pp. 380–388. Springer, Cham (2018). Scholar
  13. 13.
    Kitrungrotsakul, T., Han, X.H., Iwamoto, Y., Foruzan, A.H., Lin, L., Chen, Y.W.: Robust hepatic vessel segmentation using multi deep convolution network. In: Medical Imaging 2017: Biomedical Applications in Molecular, Structural, and Functional Imaging, vol. 10137, p. 1013711. International Society for Optics and Photonics (2017)Google Scholar
  14. 14.
    Kitrungrotsakul, T., et al.: Vesselnet: a deep convolutional neural network with multi pathways for robust hepatic vessel segmentation. Comput. Med. Imaging Graph. 75, 74–83 (2019)CrossRefGoogle Scholar
  15. 15.
    Kong, S., Fowlkes, C.C.: Recurrent pixel embedding for instance grouping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 9018–9028 (2018)Google Scholar
  16. 16.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)Google Scholar
  17. 17.
    Mise, Y., et al.: How has virtual hepatectomy changed the practice of liver surgery?: experience of 1194 virtual hepatectomy before liver resection and living donor liver transplantation. Ann. Surg. 268(1), 127–133 (2018)CrossRefGoogle Scholar
  18. 18.
    Payer, C., Štern, D., Neff, T., Bischof, H., Urschler, M.: Instance segmentation and tracking with cosine embeddings and recurrent hourglass networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 3–11. Springer, Cham (2018). Scholar
  19. 19.
    Wakabayashi, G., et al.: Recommendations for laparoscopic liver resection: a report from the second international consensus conference held in morioka. Ann. Surg. 261(4), 619–629 (2015)Google Scholar
  20. 20.
    Zeng, Y.Z., et al.: Liver vessel segmentation and identification based on oriented flux symmetry and graph cuts. Comput. Methods Programs Biomed. 150, 31–39 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Imaging Technology Center, Fujifilm CorporationMinatoJapan
  2. 2.Center for Artificial Intelligence ResearchUniversity of TsukubaTsukubaJapan
  3. 3.Department of Computer Science and EngineeringWaseda UniversityShinjukuJapan

Personalised recommendations