Abstract
Deep learning (DL) is the state-of-the-art methodology in various medical image segmentation tasks. However, it requires relatively large amounts of manually labeled training data, which may be infeasible to generate in some applications. In addition, DL methods have relatively poor generalizability to out-of-sample data. Multi-atlas segmentation (MAS), on the other hand, has promising performance using limited amounts of training data and good generalizability. A hybrid method that integrates the high accuracy of DL and good generalizability of MAS is highly desired and could play an important role in segmentation problems where manually labeled data is hard to generate. Most of the prior work focuses on improving single components of MAS using DL rather than directly optimizing the final segmentation accuracy via an end-to-end pipeline. Only one study explored this idea in binary segmentation of 2D images, but it remains unknown whether it generalizes well to multi-class 3D segmentation problems. In this study, we propose a 3D end-to-end hybrid pipeline, named deep label fusion (DLF), that takes advantage of the strengths of MAS and DL. Experimental results demonstrate that DLF yields significant improvements over conventional label fusion methods and U-Net, a direct DL approach, in the context of segmenting medial temporal lobe subregions using 3T T1-weighted and T2-weighted MRI. Further, when applied to an unseen similar dataset acquired in 7T, DLF maintains its superior performance, which demonstrates its good generalizability.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This work was supported by NIH (grant numbers R01-AG056014, R01-AG040271, P30-AG010124, R01-EB017255, R01-AG055005) and Google Cloud.
References
Iglesias, J.E., Sabuncu, M.R.: Multi-atlas segmentation of biomedical images: a survey. Med. Image Anal. 24, 205–219 (2015)
Heckemann, R.A., Hajnal, J.V., Aljabar, P., Rueckert, D., Hammers, A.: Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. Neuroimage 33, 115–126 (2006)
Coupé, P., Manjón, J.V., Fonov, V., Pruessner, J., Robles, M., Collins, D.L.: Patch-based segmentation using expert priors: application to hippocampus and ventricle segmentation. Neuroimage 54, 940–954 (2011)
Sanroma, G., Benkarim, O.M., Piella, G., Wu, G., Zhu, X., Shen, D., Ballester, M.Á.G.: Discriminative Dimensionality Reduction for Patch-Based Label Fusion. Presented at the (2015)
Wang, H., Suh, J.W., Das, S.R., Pluta, J., Craige, C., Yushkevich, P.A.: Multi-atlas segmentation with joint label fusion. IEEE Trans. Pattern Anal. Mach. Intell. 35, 611–623 (2012)
Sanroma, G., et al.: Learning non-linear patch embeddings with neural networks for label fusion. Med. Image Anal. 44, 143–155 (2018)
Ding, W., Li, L., Zhuang, X., Huang, L.: Cross-modality multi-atlas segmentation using deep neural networks. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12263, pp. 233–242. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59716-0_23
Ding, Z., Han, X., Niethammer, M.: VoteNet+ : An improved deep learning label fusion method for multi-atlas segmentation. Lecture Notes in Computer Science (including Subseries Lecture Notes in Artificial Intelligence Lecture Notes in Bioinformatics), vol. 11766 LNCS, pp. 202–210 (2019).
Xie, L., Wang, J., Dong, M., Wolk, D.A., Yushkevich, P.A.: Improving multi-atlas segmentation by convolutional neural network based patch error estimation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 347–355. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_39
Yang, H., Sun, J., Li, H., Wang, L., Xu, Z.: Neural multi-atlas label fusion: application to cardiac MR images. Med. Image Anal. 49, 60–75 (2018)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: 32nd International Conference on Machine Learning, ICML 2015. pp. 448–456. International Machine Learning Society (IMLS) (2015).
Nair, V., Hinton, G.E.: Rectified Linear Units Improve Restricted Boltzmann Machines (2010)
Yushkevich, P.A., et al.: Automated volumetry and regional thickness analysis of hippocampal subfields and medial temporal cortical structures in mild cognitive impairment. Hum. Brain Mapp. 36, 258–287 (2015)
Lin, M., Chen, Q., Yan, S.: Best practices for ConvNets applied to object recognition. arXiv Prepr. 1, 10 (2013)
Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Cardoso, M.J.: Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations. Lecture Notes in Computer Science (including Subseries Lecture Notes in Artificial Intelligence Lecture Notes in Bioinformatics), vol. 10553 LNCS, pp. 240–248 (2017)
Dou, Q., Chen, H., Jin, Y., Yu, L., Qin, J., Heng, P.-A.: 3D deeply supervised network for automatic liver segmentation from CT volumes. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 149–157. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_18
Wang, H., Suh, J.W., Das, S., Pluta, J., Altinay, M., Yushkevich, P.: Regression-Based Label Fusion for Multi-Atlas Segmentation. Conf. Comput. Vis. Pattern Recognit. Work. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognition. Work, pp. 1113–1120 (2011)
Wang, H., et al.: A learning-based wrapper method to correct systematic errors in automatic image segmentation: consistently improved performance in hippocampus, cortex and brain segmentation. Neuroimage 55, 968–985 (2011)
Crum, W.R., Camara, O., Hill, D.L.G.: Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Trans. Med. Imaging 25, 1451–1461 (2006)
Xie, L., et al.: Multi-template analysis of human perirhinal cortex in brain MRI: explicitly accounting for anatomical variability. Neuroimage 144, 183–202 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, L. et al. (2021). Deep Label Fusion: A 3D End-To-End Hybrid Multi-atlas Segmentation and Deep Learning Pipeline. In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds) Information Processing in Medical Imaging. IPMI 2021. Lecture Notes in Computer Science(), vol 12729. Springer, Cham. https://doi.org/10.1007/978-3-030-78191-0_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-78191-0_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78190-3
Online ISBN: 978-3-030-78191-0
eBook Packages: Computer ScienceComputer Science (R0)