Abstract
Medical image segmentation has been so far achieving promising results with Convolutional Neural Networks (CNNs). However, it is arguable that in traditional CNNs, its pooling layer tends to discard important information such as positions. Moreover, CNNs are sensitive to rotation and affine transformation. Capsule network is a data-efficient network design proposed to overcome such limitations by replacing pooling layers with dynamic routing and convolutional strides, which aims to preserve the part-whole relationships. Capsule network has shown a great performance in image recognition and natural language processing, but applications for medical image segmentation, particularly volumetric image segmentation, has been limited. In this work, we propose 3D-UCaps, a 3D voxel-based Capsule network for medical volumetric image segmentation. We build the concept of capsules into a CNN by designing a network with two pathways: the first pathway is encoded by 3D Capsule blocks, whereas the second pathway is decoded by 3D CNNs blocks. 3D-UCaps, therefore inherits the merits from both Capsule network to preserve the spatial relationship and CNNs to learn visual representation. We conducted experiments on various datasets to demonstrate the robustness of 3D-UCaps including iSeg-2017, LUNA16, Hippocampus, and Cardiac, where our method outperforms previous Capsule networks and 3D-Unets. Our code is available at https://github.com/VinAIResearch/3D-UCaps.
Keywords
- Capsule network
- CapsNet
- Medical image segmentation
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Armato, S.G., III., et al.: The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on CT scans. Med. Phys. 38(2), 915–931 (2011)
Bilic, P., et al.: The liver tumor segmentation benchmark (lits). arXiv preprint arXiv:1901.04056 (2019)
Bui, T.D., Shin, J., Moon, T.: Skip-connected 3d densenet for volumetric infant brain MRI segmentation. Biomed. Signal Process. Control 54, 101613 (2019)
Chen, H., Dou, Q., Yu, L., Qin, J., Heng, P.A.: Voxresnet: deep voxelwise residual networks for brain segmentation from 3d MR images. NeuroImage 170, 446–455 (2018)
Çiçek, Özgün., Abdulkadir, Ahmed, Lienkamp, Soeren S.., Brox, Thomas, Ronneberger, Olaf: 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. In: Ourselin, Sebastien, Joskowicz, Leo, Sabuncu, Mert R.., Unal, Gozde, Wells, William (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Gu, J., Tresp, V.: Improving the robustness of capsule networks to image affine transformations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7285–7293 (2020)
Gu, J., Tresp, V., Hu, H.: Capsule network is not more robust than convolutional network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14309–14317 (2021)
Hatipoglu, N., Bilgin, G.: Cell segmentation in histopathological images with deep learning algorithms by utilizing spatial relationships. Med. Biological Eng. Comput. 55(10), 1829–1848 (2017)
Hinton, G.E., Sabour, S., Frosst, N.: Matrix capsules with em routing. In: International conference on learning representations (2018)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)
Jégou, S., Drozdzal, M., Vazquez, D., Romero, A., Bengio, Y.: The one hundred layers tiramisu: fully convolutional densenets for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 11–19 (2017)
Jin, D., Xu, Z., Tang, Y., Harrison, A.P., Mollura, D.J.: Ct-realistic lung nodule simulation from 3d conditional generative adversarial networks for robust lung segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 732–740. Springer (2018)
LaLonde, R., Bagci, U.: Capsules for object segmentation. arXiv preprint arXiv:1804.04241 (2018)
LaLonde, R., Xu, Z., Irmakci, I., Jain, S., Bagci, U.: Capsules for biomedical image segmentation. Med. Image Anal. 68, 101889 (2021)
Le, N., Gummadi, R., Savvides, M.: Deep recurrent level set for segmenting brain tumors. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 646–653 (2018)
Le, N., Le, T., Yamazaki, K., Bui, T.D., Luu, K., Savides, M.: Offset curves loss for imbalanced problem in medical segmentation. In: 25th International Conference on Pattern Recognition (ICPR), pp. 6189–6195 (2021)
Le, N., Yamazaki, K., Gia, Q.K., Truong, T., Savvides, M.: A multi-task contextual atrous residual network for brain tumor detection & segmentation. In: 25th International Conference on Pattern Recognition (ICPR), pp. 5943–5950 (2021)
Li, X., Chen, H., Qi, X., Dou, Q., Fu, C.W., Heng, P.A.: H-denseunet: hybrid densely connected unet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37(12), 2663–2674 (2018)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Moshkov, N., Mathe, B., Kertesz-Farkas, A., Hollandi, R., Horvath, P.: Test-time augmentation for deep learning-based cell segmentation on microscopy images. Sci. Rep. 10(1), 1–7 (2020)
Nie, D., Wang, L., Adeli, E., Lao, C., Lin, W., Shen, D.: 3-d fully convolutional networks for multimodal isointense infant brain image segmentation. IEEE Trans. Cybernetics 49(3), 1123–1136 (2018)
Qamar, S., Jin, H., Zheng, R., Ahmad, P., Usama, M.: A variant form of 3d-unet for infant brain segmentation. Future Generation Comput. Syst. 108, 613–623 (2020)
Ramani, R.G., Shanthamalar, J.J.: Improved image processing techniques for optic disc segmentation in retinal fundus images. Biomed. Signal Process. Control 58, 101832 (2020)
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. arXiv preprint arXiv:1710.09829 (2017)
Simpson, A.L., et al.: A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063 (2019)
Souza, J.C., Diniz, J.O.B., Ferreira, J.L., da Silva, G.L.F., Silva, A.C., de Paiva, A.C.: An automatic method for lung segmentation and reconstruction in chest x-ray using deep neural networks. Comput. Methods Programs Biomed. 177, 285–296 (2019)
Survarachakan, S., Johansen, J.S., Aarseth, M., Pedersen, M.A., Lindseth, F.: Capsule nets for complex medical image segmentation tasks. In: Colour and Visual Computing Symposium (2020)
Veena, H., Muruganandham, A., Kumaran, T.S.: A review on the optic disc and optic cup segmentation and classification approaches over retinal fundus images for detection of glaucoma. SN Appl. Sci. 2(9), 1–15 (2020)
Wang, L., et al.: Benchmark on automatic six-month-old infant brain segmentation algorithms: the iseg-2017 challenge. IEEE Trans. Med. Imaging 38(9), 2219–2230 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Nguyen, T., Hua, BS., Le, N. (2021). 3D-UCaps: 3D Capsules Unet for Volumetric Image Segmentation. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12901. Springer, Cham. https://doi.org/10.1007/978-3-030-87193-2_52
Download citation
DOI: https://doi.org/10.1007/978-3-030-87193-2_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87192-5
Online ISBN: 978-3-030-87193-2
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://miccai.org/
