Dense Volume-to-Volume Vascular Boundary Detection

  • Jameson MerkowEmail author
  • Alison Marsden
  • David Kriegman
  • Zhuowen Tu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9902)


In this work, we tackle the important problem of dense 3D volume labeling in medical imaging. We start by introducing HED-3D, a 3D extension of the state-of-the-art 2D edge detector (HED). Next, we develop a novel 3D-Convolutional Neural Network (CNN) architecture, I2I-3D, that predicts boundary location in volumetric data. Our fine-to-fine, deeply supervised framework addresses three critical issues to 3D boundary detection: (1) efficient, holistic, end-to-end volumetric label training and prediction (2) precise voxel-level prediction to capture fine scale structures prevalent in medical data and (3) directed multi-scale, multi-level feature learning. We evaluate our approaches on a dataset consisting of 93 medical image volumes with a wide variety of anatomical regions and vascular structures. We show that our deep learning approaches out-perform the current state-of-the-art in 3D vascular boundary detection (structured forests 3D), by a large margin, as well as HED applied to slices. Prediction takes about one minute on a typical \(512\,\times \,512\,\times \,512\) volume, when using GPU.



Z.T. is supported by NSF IIS-1360566, NSF IIS-1360568, and a Northrop Grumman Contextual Robotics grant. We are grateful for the generous donation of the GPUs by NVIDIA.


  1. 1.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. PAMI 33(5), 898–916 (2011)CrossRefGoogle Scholar
  2. 2.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. In: BMVC (2014)Google Scholar
  3. 3.
    Dollár, P., Zitnick, C.L.: Fast edge detection using structured forests. In: PAMI (2015)Google Scholar
  4. 4.
    Hariharan, B., Arbeláez, P., Girshick, R., Malik, J.: Hypercolumns for object segmentation and fine-grained localization. In: CVPR (2014)Google Scholar
  5. 5.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. preprint arXiv:1408.5093 (2014)
  6. 6.
    Lee, C.Y., Xie, S., Gallagher, P., Zhang, Z., Tu, Z.: Deeply-supervised nets. In: AISTATS (2015)Google Scholar
  7. 7.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2014)Google Scholar
  8. 8.
    Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. PAMI 26(5), 530–549 (2004)CrossRefGoogle Scholar
  9. 9.
    Merkow, J., Tu, Z., Kriegman, D., Marsden, A.: Structural edge detection for cardiovascular modeling. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 735–742. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24574-4_88CrossRefGoogle Scholar
  10. 10.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  11. 11.
    Roth, H.R., Lu, L., Farag, A., Shin, H.-C., Liu, J., Turkbey, E.B., Summers, R.M.: DeepOrgan: multi-level deep convolutional networks for automated pancreas segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 556–564. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24553-9_68CrossRefGoogle Scholar
  12. 12.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  13. 13.
    Xie, S., Tu, Z.: Holistically-nested edge detection. In: ICCV (2015)Google Scholar
  14. 14.
    Zheng, Y., Liu, D., Georgescu, B., Nguyen, H., Comaniciu, D.: 3D deep learning for efficient and robust landmark detection in volumetric data. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 565–572. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24553-9_69CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Jameson Merkow
    • 1
    Email author
  • Alison Marsden
    • 2
  • David Kriegman
    • 1
  • Zhuowen Tu
    • 1
  1. 1.University of CaliforniaSan DiegoUSA
  2. 2.Stanford UniversityStanfordUSA

Personalised recommendations