Deep Learning Based Instance Segmentation in 3D Biomedical Images Using Weak Annotation

  • Zhuo Zhao
  • Lin Yang
  • Hao Zheng
  • Ian H. Guldner
  • Siyuan Zhang
  • Danny Z. ChenEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11073)


Instance segmentation in 3D images is a fundamental task in biomedical image analysis. While deep learning models often work well for 2D instance segmentation, 3D instance segmentation still faces critical challenges, such as insufficient training data due to various annotation difficulties in 3D biomedical images. Common 3D annotation methods (e.g., full voxel annotation) incur high workloads and costs for labeling enough instances for training deep learning 3D instance segmentation models. In this paper, we propose a new weak annotation approach for training a fast deep learning 3D instance segmentation model without using full voxel mask annotation. Our approach needs only 3D bounding boxes for all instances and full voxel annotation for a small fraction of the instances, and uses a novel two-stage 3D instance segmentation model utilizing these two kinds of annotation, respectively. We evaluate our approach on several biomedical image datasets, and the experimental results show that (1) with full annotated boxes and a small amount of masks, our approach can achieve similar performance as the best known methods using full annotation, and (2) with similar annotation time, our approach outperforms the best known methods that use full annotation.



This research was supported in part by NSF grant CCF-1640081 and the Nanoelectronics Research Corporation (NERC), a wholly-owned subsidiary of the Semiconductor Research Corporation (SRC), through Extremely Energy Efficient Collective Electronics (EXCEL), an SRC-NRI Nanoelectronics Research Initiative under Research Task ID 2698.005, NSF grants CCF-1617735 and CNS-1629914, and NIH grant R01 R01CA194697.


  1. 1.
    Chen, H., Dou, Q., Yu, L., Heng, P.A.: VoxResNet: deep voxelwise residual networks for volumetric brain segmentation. arXiv preprint arXiv:1608.05895 (2016)
  2. 2.
    Girshick, R.: Fast R-CNN. arXiv preprint arXiv:1504.08083 (2015)
  3. 3.
    He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: ICCV, pp. 2980–2988 (2017)Google Scholar
  4. 4.
    Hu, R., Dollár, P., He, K., Darrell, T., Girshick, R.: Learning to segment every thing. arXiv preprint arXiv:1711.10370 (2017)
  5. 5.
    Khoreva, A., Benenson, R., Hosang, J., Hein, M., Schiele, B.: Simple does it: weakly supervised instance and semantic segmentation. In: CVPR (2017)Google Scholar
  6. 6.
    Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). Scholar
  7. 7.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS, pp. 91–99 (2015)Google Scholar
  8. 8.
    Ulman, V., et al.: An objective comparison of cell-tracking algorithms. Nat. Methods 14, 1141 (2017)CrossRefGoogle Scholar
  9. 9.
    Yang, L., Zhang, Y., Chen, J., Zhang, S., Chen, D.Z.: Suggestive annotation: a deep active learning framework for biomedical image segmentation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 399–407. Springer, Cham (2017). Scholar
  10. 10.
    Yang, L., Zhang, Y., Guldner, I.H., Zhang, S., Chen, D.Z.: 3D segmentation of glial cells using fully convolutional networks and k-terminal cut. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 658–666. Springer, Cham (2016). Scholar
  11. 11.
    Zhang, Y., Yang, L., Chen, J., Fredericksen, M., Hughes, D.P., Chen, D.Z.: Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 408–416. Springer, Cham (2017). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Zhuo Zhao
    • 1
  • Lin Yang
    • 1
  • Hao Zheng
    • 1
  • Ian H. Guldner
    • 2
  • Siyuan Zhang
    • 2
  • Danny Z. Chen
    • 1
    Email author
  1. 1.Department of Computer Science and EngineeringUniversity of Notre DameNotre DameUSA
  2. 2.Department of Biological Sciences, Harper Cancer Research InstituteUniversity of Notre DameNotre DameUSA

Personalised recommendations