Abstract
Purpose
Fully Convolutional neural Networks (FCNs) are the most popular models for medical image segmentation. However, they do not explicitly integrate spatial organ positions, which can be crucial for proper labeling in challenging contexts.
Methods
In this work, we propose a method that combines a model representing prior probabilities of an organ position in 3D with visual FCN predictions by means of a generalized prior-driven prediction function. The prior is also used in a self-labeling process to handle low-data regimes, in order to improve the quality of the pseudo-label selection.
Results
Experiments carried out on CT scans from the public TCIA pancreas segmentation dataset reveal that the resulting STIPPLE model can significantly increase performances compared to the FCN baseline, especially with few training images. We also show that STIPPLE outperforms state-of-the-art semi-supervised segmentation methods by leveraging the spatial prior information.
Conclusions
STIPPLE provides a segmentation method effective with few labeled examples, which is crucial in the medical domain. It offers an intuitive way to incorporate absolute position information by mimicking expert annotators.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
Data are publicly available.
Notes
Here we choose to designate the coordinates with (w,h,z) so it is a different notation than the model’s output and input, x and y.
References
Bai W, Oktay O, Sinclair M, Suzuki H, Rajchl M, Tarroni G, Glocker B, King A, Matthews PM, Rueckert D (2017) Semi-supervised learning for network-based cardiac MR image segmentation. In: MICCAI, pp 253–260
Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille A (2018) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS. IEEE Trans Pattern Anal Mach Intell 40:834–848
Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3d u-net: learning dense volumetric segmentation from sparse annotation. In: MICCAI, pp 424–432
Dalca AV, Guttag J, Sabuncu MR (2018) Anatomical priors in convolutional networks for unsupervised biomedical segmentation. In: IEEE CVPR, pp 9290–9299
Feng X, Qing K, Tustison NJ, Meyer CH, Chen Q (2019) Deep convolutional neural network for segmentation of thoracic organs-at-risk using cropped 3d images. Med Phys 46(5):2169–2180
Hung WC, Tsai YH, Liou YT, Lin YY, Yang MH (2018) Adversarial learning for semi-supervised semantic segmentation. In: BMVC
Kakeya H, Okada T, Oshiro Y (2018) 3d u-japa-net: mixture of convolutional networks for abdominal multi-organ CT segmentation. In: MICCAI, pp 426–433
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: NeurIPS, pp 1097–1105
Liu R, Lehman J, Molino P, Such FP, Frank E, Sergeev A, Yosinski J (2018) An intriguing failing of convolutional neural networks and the coordconv solution. In: NeurIPS
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE CVPR, pp 3431–3440
Milletari F, Navab N, Ahmadi S (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 fourth international conference on 3D vision (3DV), pp 565–571
Nie D, Gao Y, Wang L, Shen D (2018) Asdnet: attention based semi-supervised deep networks for medical image segmentation. In: MICCAI, pp 370–378
Oktay O, Ferrante E, Kamnitsas K, Heinrich MP, Bai W, Caballero J, Cook SA, de Marvao A, Dawes T, O’Regan DP, Kainz B, Glocker B, Rueckert D (2018) Anatomically constrained neural networks (acnns): application to cardiac image enhancement and segmentation. IEEE Trans Med Imaging 37:384–395
Oktay O, Schlemper J, Le Folgoc L, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B, Glocker B, Rueckert D (2018) Attention u-net: learning where to look for the pancreas. In: MIDL
Petit O, Thome N, Charnoz A, Hostettler A, Soler L (2018) Handling missing annotations for semantic segmentation with deep convnets. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, pp 20–28
Roth H, Farag A, Turkbey EB, Lu L, Liu J, Summers RM (2016) Data from pancreas-ct. In: The cancer imaging archive (TCIA). https://doi.org/10.7937/K9/TCIA.2016.tNB1kqBU
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: MICCAI, pp 234–241
Roth HR, Lu L, Lay N, Harrison AP, Farag A, Sohn A, Summers RM (2018) Spatial aggregation of holistically-nested convolutional neural networks for automated pancreas localization and segmentation. Med Image Anal 45:94–107
Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: IEEE CVPR
Trullo R, Petitjean C, Dubray B, Ruan S (2019) Multiorgan segmentation using distance-aware adversarial networks. J Med Imaging 6(1):014001
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: NeurIPS, pp 5998–6008
Yu L, Wang S, Li X, Fu CW, Heng PA (2019) Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation. In: MICCAI, pp 605–613
Zheng H, Lin L, Hu H, Zhang Q, Chen Q, Iwamoto Y, Han X, Chen YW, Tong R, Wu J (2019) Semi-supervised segmentation of liver using adversarial learning with deep atlas prior. In: MICCAI, pp 148–156
Funding
This study was funded by Visible Patient with le CNAM as an academic partner.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
conflict of interest
The authors declare that they have no conflict of interest.
Ethics approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Not applicable.
Appendices
A Details on the network used in the study
See Table 5.
B Additional training details
In this work, we use a 2D U-Net as our main backbone FCN. It was trained with a batch size of 6. The learning rate was \(1e-4\) with an inverse time decay scheduler and a decay rate set to get a learning rate of \(1e-5\) at the end of training. We train the model for 25 epochs which corresponds to the observed convergence in all the experiments.
The data augmentation consists of small random translations (e.g. between \(-15\) and \(+15\)), small rotations (e.g., \(-6\) to \(+6\) degrees) and zooms (e.g., 0.9 to 1.1). This augmentation is applied to the image and label but also to the prior. Moreover, we have another augmentation on the prior to simulate imperfect positioning by using an additional translation.
The code was developed with tensorflow and the training performed on Nvidia RTX 2080Ti GPU cards.
Rights and permissions
About this article
Cite this article
Petit, O., Thome, N. & Soler, L. 3D spatial priors for semi-supervised organ segmentation with deep convolutional neural networks. Int J CARS 17, 129–139 (2022). https://doi.org/10.1007/s11548-021-02494-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-021-02494-y