Segmentation of Renal Structures for Image-Guided Surgery

  • Junning Li
  • Pechin Lo
  • Ahmed Taha
  • Hang Wu
  • Tao Zhao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11073)


Anatomic models of kidneys may help surgeons make plans or guide surgical procedures, in which segmentation is a prerequisite. We develop a convolutional neural network to segment multiple renal structures from arterial-phase CT images, including parenchyma, arteries, veins, collecting systems, and abnormal structures. To the best of our knowledge, this is the first work dedicated to jointly segment these five renal structures. We introduce two novel techniques. First, we generalize the sequential residual architecture to residual graphs. With this generalization, we convert a popular multi-scale architecture (U-Net) to a residual U-Net. Second, we solve the unbalanced data problem which commonly exists in medical image segmentation by weighting pixels with multi-scale entropy. Our multi-scale entropy map combines information theory and scale analysis to capture spatial complexity of a multi-class label map. The two techniques significantly improve segmentation accuracy. Trained on 400 CT scans and tested on another 100, our algorithm achieves median Dice indices 0.96, 0.86, 0.8, 0.62, and 0.29 respectively for renal parenchyma, arteries, veins, collecting systems and abnormal structures.


  1. 1.
    Cancer Stat Facts: Kidney and Renal Pelvis Cancer.
  2. 2.
    Hughes-Hallett, A., et al.: Augmented reality partial nephrectomy: examining the current status and future perspectives. Urology 83, 266–273 (2014)CrossRefGoogle Scholar
  3. 3.
    Su, L., Vagvolgyi, B., Agarwal, R., Reiley, C., Taylor, R., Hager, G.: Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73, 896–900 (2009)CrossRefGoogle Scholar
  4. 4.
    Cuingnet, R., Prevost, R., Lesage, D., Cohen, L.D., Mory, B., Ardon, R.: Automatic detection and segmentation of kidneys in 3D CT images using random forests. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7512, pp. 66–74. Springer, Heidelberg (2012). Scholar
  5. 5.
    Khalifa, F., Soliman, A., Dwyer, A., Gimelfarb, G., ElBaz, A.: A random forest-based framework for 3D kidney segmentation from dynamic contrast-enhanced CT images. In IEEE International Conference on Image Processing (2016)Google Scholar
  6. 6.
    Yang, G., et al.: Automatic kidney segmentation in CT images based on multi-atlas image registration. In: Annual International Conference of the IEEE on Engineering in Medicine and Biology Society (2014)Google Scholar
  7. 7.
    Sharma, K., et al.: Automatic segmentation of kidneys using deep learning for total kidney volume quantification in autosomal dominant polycystic kidney disease. Sci. Rep. 7, 2049 (2017)CrossRefGoogle Scholar
  8. 8.
    Taha, A., Lo, P., Li, J., Zhao, T.: Kid-Net: convolution networks for kidney vessels segmentation from CT-volumes. In: Medical Image Computing and Computer-Assisted Intervention (2018)Google Scholar
  9. 9.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  10. 10.
    Merkow, J., Marsden, A., Kriegman, D., Tu, Z.: Dense volume-to-volume vascular boundary detection. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 371–379. Springer, Cham (2016). Scholar
  11. 11.
    Milletari, F., Navab, N., Ahmadi, S.-A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: International Conference on 3D Vision (2016)Google Scholar
  12. 12.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  13. 13.
    Huang, C., Li, Y., Loy, C.C., Tang, X.: Learning deep representation for imbalanced classification. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  14. 14.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference for Learning Representations Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Junning Li
    • 1
  • Pechin Lo
    • 1
  • Ahmed Taha
    • 1
  • Hang Wu
    • 1
  • Tao Zhao
    • 1
  1. 1.Intuitive SurgicalSunnyvaleUSA

Personalised recommendations