Abstract
Accurate segmentation of the left ventricle myocardium is the key step of automatic assessment of cardiac function. However, the current methods mainly focus on the end-diastolic and the end-systolic frames in cine MR sequences and lack the attention to myocardial motion in the cardiac cycle. Additionally, due to the lack of fine segmentation tools, the simplified approach, excluding papillary muscles and trabeculae from myocardium, is applied in clinical practice. To solve these problems, we propose a motion-aware DNN model with edge focus loss and quality control in this paper. Specifically, the bidirectional ConvLSTM layer and a new motion attention layer are proposed to encode motion-aware feature maps, and an edge focus loss function is proposed to train the model to generate the fine segmentation results. Additionally, a quality control method is proposed to filter out the abnormal segmentations before subsequent analyses. Compared with state-of-the-art segmentation models on the public dataset and the in-house dataset, the proposed method has obtained high segmentation accuracy. On the 17-segment model, the proposed method has obtained the highest Pearson correlation coefficient at 14 of 17 segments, and the mean PCC of 85%. The experimental results highlight the segmentation accuracy of the proposed method as well as its availability to substitute for the manually annotated boundaries for the automatic assessment of cardiac function.
Similar content being viewed by others
Data Availability
The ACDC dataset is a public dataset (online website https://acdc.creatis.insa-lyon.fr/#challenges). The in-house dataset that has been used is confidential.
Notes
Although motion tracking commercial software is available for cardiac cine imaging in clinically, manual correction of delineated contours used for tracing is often required, resulting in increased processing times and a significant degree of inter and intra-observer variability. Thus, the proposed model can be used to substitute for manual contouring the endocardium and the epicardium, to implement automatic myocardial stain estimation.
References
Ouyang, D, He, B, Ghorbani, A, Yuan, N, Zou, J Y: Video-based ai for beat-to-beat assessment of cardiac function. Nature, 580(7802), 2020
Cerqueira MD, Weissman NJ, Dilsizian V, Jacobs AK, Kaul S, Laskey WK, Pennell DJ, Rumberger JA, Ryan T, Verani MS: American Heart Association Writing Group on Myocardial Segmentation and Registration for Cardiac Imaging. Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart. A statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association. Circulation, 105(4):539–542, 2002
Chen C, Qin C, Qiu H, Tarroni G, Duan J, Bai W, Rueckert D: Deep Learning for Cardiac Image Segmentation: A Review. Front Cardiovasc Med., 7:25, 2020
Li F Y, Li W, Gao X, Xiao B: A Novel Framework With Weighted Decision Map Based on Convolutional Neural Network for Cardiac MR Segmentation. IEEE J Biomed Health Inform., 26(5):2228-2239, 2022
Xue W, Li J, Hu Z, Kerfoot E, Clough J, Oksuz I, Xu H, Grau V, Guo F, Ng M, Li X, Li Q, Liu L, Ma J, Grinias E, Tziritas G, Yan W, Atehortua A, Garreau M, Jang Y, Debus A, Ferrante E, Yang G, Hua T, Li S: Left Ventricle Quantification Challenge: A Comprehensive Comparison and Evaluation of Segmentation and Regression for Mid-Ventricular Short-Axis Cardiac MR Data. IEEE J Biomed Health Inform., 25(9):3541-3553, 2021
Yang X, Zhang Y, Lo B, Wu D, Liao H, Zhang YT: DBAN: Adversarial Network with Multi-Scale Features for Cardiac MRI Segmentation. IEEE J Biomed Health Inform., 25(6):2018–28, 2021
Riffel J H, Schmucker K, Andre F, Ochs M, Hirschberg K, Schaub E, Fritz T, Mueller-Hennessen M, Giannitsis E, Katus H A, Friedrich M G: Cardiovascular magnetic resonance of cardiac morphology and function: impact of different strategies of contour drawing and indexing. Clin Res Cardiol., 108(4):411-429, 2019
Wang K N, Yang X, Miao J, Li L, Yao J, Zhou P, Xue W, Zhou G Q, Zhuang X, Ni D: AWSnet: An auto-weighted supervision attention network for myocardial scar and edema segmentation in multi-sequence cardiac magnetic resonance images. Med Image Anal., https://doi.org/10.1016/j.media.2022.102362, 2022
Vesal S, Gu M, Maier A, Ravikumar N: Spatio-Temporal Multi-Task Learning for Cardiac MRI Left Ventricle Quantification. IEEE J Biomed Health Inform., 25(7):2698-2709, 2021
Ostvik A, Salte IM, Smistad E, Nguyen TM, Melichova D, Brunvand H, Haugaa K, Edvardsen T, Grenne B, Lovstakken L: Myocardial Function Imaging in Echocardiography Using Deep Learning. IEEE Trans Med Imaging, 40(5):1340-1351, 2021
Uslu F, Varela M, Boniface G, Mahenthran T, Chubb H, Bharath AA: LA-Net: A Multi-Task Deep Network for the Segmentation of the Left Atrium. IEEE Trans Med Imaging, 41(2):456-464, 2022
Jafari M H, Woudenberg N van, Luong C, Abolmaesumi P, Tsang T: Deep bayesian image segmentation for a more robust ejection fraction estimation. In: Proceedings - International Symposium on Biomedical Imaging, 1264–1272, 2021
Dong S, Zhao J, Zhang M, Shi Z, Deng J, Shi Y, et al. DeU-Net: Deformable U-Net for 3D Cardiac MRI Video Segmentation. In: MICCAI 2020: Medical Image Computing and Computer Assisted Intervention, 98–107, 2020
Dong S, Pan Z, Fu Y, Yang Q, Gao Y, Yu T, Shi Y, Zhuo C: DeU-Net 2.0: Enhanced deformable U-Net for 3D cardiac cine MRI segmentation. Med Image Anal., https://doi.org/10.1016/j.media.2022.102389, 2022
Uslu F, Varela M: SA-net: A sequence aware network for the segmentation of the left atrium in cine MRI datasets. In: Proceedings - International Symposium on Biomedical Imaging, 766–775, 2021
Yang D, Huang Q, Mikael K, Al’aref S Al, Axel L, Metaxas D: Mri-Based Characterization of Left Ventricle Dyssynchrony with Correlation to Crt Outcomes. In: Proceedings - International Symposium on Biomedical Imaging, 822–827, 2020
Tan L K, Liew Y M, Lim E, McLaughlin R A: Convolutional neural network regression for short-axis left ventricle segmentation in cardiac cine MR sequences. Med Image Anal., 39:78–86, 2017
Robert R, Vanya V V, Wenjia B, Hideaki S, Paul M M, Chris P, Daniel R, Ben G: Automatic quality control of cardiac MRI segmentation in large-scale population imaging. MICCAI 2017 Cham, https://doi.org/10.1007/978-3-319-66182-7_82, 2017
Ruijsink B, Puyol-Antón E, Oksuz I, Sinclair M, Bai W, Schnabel J A, Razavi R, King A P: Fully Automated, Quality-Controlled Cardiac Analysis From CMR: Validation and Large-Scale Application to Characterize Cardiac Function. JACC Cardiovasc Imaging., 13(3):684-695, 2020
Li K, Yu L, Heng P A: Towards reliable cardiac image segmentation: Assessing image-level and pixel-level segmentation quality via self-reflective references. Med Image Anal., https://doi.org/10.1016/j.media.2022.102426, 2022
Zakeri A, Hokmabadi A, Ravikumar N, Frangi A F, Gooya A: A probabilistic deep motion model for unsupervised cardiac shape anomaly assessment. Med Image Anal., https://doi.org/10.1016/j.media.2021.102276, 2022
Wang W, Shen J, Lu X, Hoi S, Ling H: Paying Attention to Video Object Pattern Understanding. IEEE Trans Pattern Anal Mach Intell., 43(7):2413-2428, 2021
Painchaud N, Skandarani Y, Judge T, Bernard O, Lalande A, Jodoin P M: Cardiac Segmentation With Strong Anatomical Guarantees. IEEE Trans Med Imaging., 39(11):3703-3713, 2020
Suinesiaputra A, Cowan B, Al-Agamy A, Elattar M, Ayache N, Fahmy A, Khalifa A, Medrano-Gracia P, Jolly M, Kadish A, Lee D, Margeta J, Warfield S, Young A: A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images. Med Image Anal., 18(1):50-62, 2014
Jha D, Smedsrud P, Riegler M, Johansen D, de Lange T, Halvorsen P, et al: ResUNet++: An Advanced Architecture for Medical Image Segmentation. 2019 IEEE International Symposium on Multimedia (ISM), 2019
Alom M, Hasan M, Yakopcic C, Taha T, Asari V: Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation. https://doi.org/10.48550/arXiv.1802.06955, 2018
Ozan O, Jo S, Loic L, Matthew L, Mattias H, Kazunari M, Kensaku M, Steven M, Nils Y,Bernhard K, Ben G, Daniel R: Attention U-Net: Learning Where to Look for the Pancreas. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.1804.03999, 2018
Jieneng C, Yongyi L, Qihang Y, Xiangde L, Ehsan A, Yan W, Le L, Alan L, Yuyin Z: TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.2102.04306, 2021
Hu J, Shen L, Albanie S, Sun G, Wu E: Squeeze-and-Excitation Networks. IEEE Trans Pattern Anal Mach Intell., 42(8):2011-2023, 2020
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J: UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation. IEEE Trans Med Imaging., 39(6):1856-1867, 2020
Ronneberger O, Fischer P, Brox T: U-Net: Convolutional Networks for Biomedical Image Segmentation. in Proc MICCAI Munich, Germany: Springer., 18;234–241, 2015
Huang H, Lin L, Tong R, Hu H, Zhang Q, Iwamoto Y, et al; UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation. ICASSP 2020, DOI:https://doi.org/10.1109/ICASSP40776.2020.9053405, Apr 19, 2020
Diakogiannis F, Waldner F, Caccetta P, Wu C; ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing, 162:94–114, 2020
Takikawa T, Acuna D, Jampani V, Fidler S: Gated-SCNN: Gated Shape CNNs for Semantic Segmentation. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.1907.05740, Jul 12, 2019
Sun J, Darbehani F, Zaidi M, Wang B: SAUNet: Shape Attentive U-Net for Interpretable Medical Image Segmentation. DOI:https://doi.org/10.1007/978-3-030-59719-1_77, Jan 21, 2020
Rohé M, Sermesant M, Pennec X: Automatic Multi-Atlas Segmentation of Myocardium with SVF-Net. Statistical Atlases and Computational Models of the Heart, 170–177, 2018
Patravali J, Jain S, Chilamkurthy S: 2d-3d fully convolutional neural networks for cardiac mr segmentation. In: In: International Workshop on Statistical Atlases and Computional Models of the Heart, Springer, 130–139, 2017
Simantiris G, Tziritas G: Cardiac MRI segmentation with a dilated CNN incorporating domain-specific constraints. IEEE J Sel Top Signal Process, 14 (6): 1235–1243, 2020
Clement Z, Zhiming L, Alain L, Olivier H, Pierre-Marc J: GridNet with automatic shape prior registration for automatic MRI cardiac segmentation. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.1705.08943, 2017
Du T, Lubomir B, Rob F, Lorenzo T, Manohar P: Learning Spatiotemporal Features with 3D Convolutional Networks. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.1412.0767, 2014
Yunbo W, Lu J, Ming-Hsuan Y, Li-Jia L, Mingsheng L, Li F: Eidetic 3D LSTM: A Model for Video Prediction and Beyond. In: International Conference on Learning Representations, 2019
Funding
This work was supported by the National Natural Science Foundation of China (No.62172288 and No. 61672362).
Author information
Authors and Affiliations
Contributions
Guarantor of integrity of the entire study: Jie Lu, Nan Zhang Study design: Jie Lu, Nan Zhang Algorithm implementation: Yu Wang, Zheng Sun, Nan Zhang Literature research:Yu Wang, Zheng Sun Clinical studies: Zheng Sun, Zhi Liu, Jie Lu Experimental studies: Yu Wang, Zheng Sun, Nan Zhang Data acquisition: Zheng Sun, Zhi Liu Data analysis: Yu Wang, Zheng Sun, Nan Zhang Manuscript preparation, editing and review: Yu Wang, Zheng Sun, Zhi Liu, Jie Lu, Nan Zhang.
Corresponding authors
Ethics declarations
Ethical Approval
The Ethics Committee approved the study of Xuanwu Hospital of Capital Medical University (D2014044). By the Declaration of Helsinki, written informed consent was obtained from each subject before enrollment.
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, Y., Sun, Z., Liu, Z. et al. A Motion-Aware DNN Model with Edge Focus Loss and Quality Control for Short-Axis Left Ventricle Segmentation of Cine MR Sequences. J Digit Imaging. Inform. med. (2024). https://doi.org/10.1007/s10278-023-00942-6
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10278-023-00942-6