Abstract
Precise segmentation of myocardial pathology is significant for the assessment of myocardial infarction (MI). Generally, manual segmentation of myocardial pathology is burdensome and time-consuming, and the burden of disease assessment is greatly increased when considering multi-modal images. To better detect the correlations across modalities and adequately leverage the complementary information between them, we present an end-to-end architecture for automatic cardiac multi-task segmentation in magnetic resonance images (MRI) with a U-shaped network (CMS-UNet), which simultaneous segmenting left ventricular (LV) blood pool, LV myocardium, right ventricular (RV) blood pool, myocardial edema, and myocardial scars. In this work, multi-modal data are employed as the input of the network, which merely utilizes one shared encoder for extracting the feature information of different modalities respectively. Therefore, our network can automatically explore the correlations between modalities and better learn the complicated and interdependent feature representation of each modality. In decoder, we aggregated the feature information extracted from different modalities and exploited a channel reconstruction upsampling (CRU) to restore the pixel-level prediction while addressing the problem of missing more detailed information, especially for edge in bilinear upsampling. In addition, we adopted a multi-scale convolution module (MSCM) at the top of the network to capture multi-scale features, which is extremely beneficial for achieving accurate segmentation results. We evaluated our approach on the Multi-sequence CMR based Myocardial Pathology Segmentation Challenge 2020 (MyoPS 2020) dataset, and obtained the Dice 0.581 for the myocardial scars and the average Dice 0.725 for the myocardial edema and myocardial scars.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhuang, X.: Multivariate mixture model for cardiac segmentation from multi-sequence MRI. In: Ourselin, S., Joskowicz, L., Sabuncu, Mert R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 581–588. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_67
Zhuang, X.: Multivariate mixture model for myocardial segmentation combining multi-source images. IEEE Trans. Pattern Anal. Mach. Intell. 41(12), 2933–2946 (2019)
Zabihollahy, F., White, J.A., Ukwatta, E.: Convolutional neural network-based approach for segmentation of left ventricle myocardial scar from 3D late gadolinium enhancement MR images. Med. Phys. 46(4), 1740–1751 (2019)
Li, L., Weng, X., Schnabel, J. A., Zhuang, X.: Joint left atrial segmentation and scar quantification based on a DNN with spatial encoding and shape attention. arXiv preprint arXiv:2006.13011 (2020)
Li, L., et al.: Atrial scar quantification via multi-scale CNN in the graph-cuts framework. Med. Image Anal. 60, 101595 (2020)
Clèrigues A., Valverde, S., Bernal, J., Freixenet, J., Oliver, A., Lladó, X.: SUNet: a deep learning architecture for acute stroke lesion segmentation and outcome prediction in multimodal mri. arXiv preprint arXiv:1810.13304 (2018)
Cui, S., Mao, L., Jiang, J., Liu, C., Xiong, S.: Automatic semantic segmentation of brain gliomas from MRI images using a deep cascaded neural network. J. Healthc. Eng. 2018(1), 1–14 (2018)
Dolz, J., Desrosiers, C., Ben Ayed, I.: IVD-Net: intervertebral disc localization and segmentation in mri with a multi-modal UNet. In: Zheng, G., Belavy, D., Cai, Y., Li, S. (eds.) CSI 2018. LNCS, vol. 11397, pp. 130–143. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-13736-6_11
Dolz, J., Gopinath, K., Yuan, J., Lombaert, H., Desrosiers, C., Ayed, I.B.: HyperDense-Net: a hyper-densely connected CNN for multi-modal image segmentation. IEEE Trans. Med. Imaging 38(5), 1116–1126 (2018)
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imaging 39(6), 1856–1867 (2019)
Takikawa, T., Acuna, D., Jampani, V., Fidler, S.: Gated-SCNN: gated shape CNNs for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5228–5237 (2019)
Fu, J., et al.: Dual attention network for scene segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3141–3149 (2019)
Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883 (2016)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, William M., Frangi, Alejandro F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Kamnitsas, K., et al.: Ensembles of multiple models and architectures for robust brain tumour segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 450–462. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75238-9_38
Tong, Q., Ning, M., Si, W., Liao, X., Qin, J.: 3D deeply-supervised U-Net based whole heart segmentation. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 224–232. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_24
Zhou, X.Y., Yang, G.Z.: Normalization in training U-Net for 2-D biomedical semantic segmentation. IEEE Robot. Autom. Lett. 4(2), 1792–1799 (2019)
Chen, C., et al.: Unsupervised multi-modal style transfer for cardiac MR segmentation. In: Pop, M., Sermesant, M., Camara, O., Zhuang, X., Li, S., Young, A., Mansi, T., Suinesiaputra, A. (eds.) STACOM 2019. LNCS, vol. 12009, pp. 209–219. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-39074-7_22
Oktay, O., et al.: Attention U-Net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018)
Li, F., Li, W., Shu, Y., Qin, S., Xiao, B., Zhan, Z.: Multiscale receptive field based on residual network for pancreas segmentation in CT images. Biomed. Signal Process. Control 57, 101828 (2020)
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., Xu, C.: GhostNet: more features from cheap operations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1580–1589 (2020)
Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017)
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Acknowledgements
This work was supported by the National Natural Science Foundation of China [Nos. 61972060 and U1713213], National Science & Technology Major Project [2016YFC1000307-3], Natural Science Foundation of Chongqing [cstc2019cxcyljrc-td0270, cstc2019jcyj-cxttX0002, cstc2019jcyj-zdxmX0011].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, W., Wang, L., Qin, S. (2020). CMS-UNet: Cardiac Multi-task Segmentation in MRI with a U-Shaped Network. In: Zhuang, X., Li, L. (eds) Myocardial Pathology Segmentation Combining Multi-Sequence Cardiac Magnetic Resonance Images. MyoPS 2020. Lecture Notes in Computer Science(), vol 12554. Springer, Cham. https://doi.org/10.1007/978-3-030-65651-5_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-65651-5_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65650-8
Online ISBN: 978-3-030-65651-5
eBook Packages: Computer ScienceComputer Science (R0)