Deep Multi-Modal Encoder-Decoder Networks for Shape Constrained Segmentation and Joint Representation Learning
Deep learning approaches have been very successful in segmenting cardiac structures from CT and MR volumes. Despite continuous progress, automated segmentation of these structures remains challenging due to highly complex regional characteristics (e.g. homogeneous gray-level transitions) and large anatomical shape variability. To cope with these challenges, the incorporation of shape priors into neural networks for robust segmentation is an important area of current research. We propose a novel approach that leverages shared information across imaging modalities and shape segmentations within a unified multi-modal encoder-decoder network. This jointly end-to-end trainable architecture is advantageous in improving robustness due to strong shape constraints and enables further applications due to smooth transitions in the learned shape space. Despite no skip connections are used and all shape information is encoded in a low-dimensional representation, our approach achieves high-accuracy segmentation and consistent shape interpolation results on the multi-modal whole heart segmentation dataset.
Unable to display preview. Download preview PDF.
- 2.Milletari F, et al.; IEEE. V-net: fully convolutional neural networks for volumetric medical image segmentation. Proc 3D Vis. 2016; p. 565–571.Google Scholar
- 3.Ronneberger O, et al.; Springer. U-net: convolutional networks for biomedical image segmentation. Proc MICCAI. 2015; p. 234–241.Google Scholar
- 4.Oktay O, et al. Anatomically constrained neural networks (ACNN): application to cardiac image enhancement and segmentation. CoRR. 2017;abs/1705.08302.Google Scholar
- 5.Ravishankar H, et al. Learning and incorporating shape models for semantic segmentation. Proc MICCAI. 2017; p. 203–211.Google Scholar
- 6.Jetley S, Sapienza M, Golodetz S, et al. Straight to shapes: real-time detection of encoded shapes. CoRR. 2016;abs/1611.07932.Google Scholar
- 7.Bouteldja N, Heinrich MP. Deep 3D encoder-decoder networks with applications to organ segmentation. Stud Conf. 2018; p. 229–232.Google Scholar