Skip to main content
SpringerLink
Log in

A Motion-Aware DNN Model with Edge Focus Loss and Quality Control for Short-Axis Left Ventricle Segmentation of Cine MR Sequences

  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

Accurate segmentation of the left ventricle myocardium is the key step of automatic assessment of cardiac function. However, the current methods mainly focus on the end-diastolic and the end-systolic frames in cine MR sequences and lack the attention to myocardial motion in the cardiac cycle. Additionally, due to the lack of fine segmentation tools, the simplified approach, excluding papillary muscles and trabeculae from myocardium, is applied in clinical practice. To solve these problems, we propose a motion-aware DNN model with edge focus loss and quality control in this paper. Specifically, the bidirectional ConvLSTM layer and a new motion attention layer are proposed to encode motion-aware feature maps, and an edge focus loss function is proposed to train the model to generate the fine segmentation results. Additionally, a quality control method is proposed to filter out the abnormal segmentations before subsequent analyses. Compared with state-of-the-art segmentation models on the public dataset and the in-house dataset, the proposed method has obtained high segmentation accuracy. On the 17-segment model, the proposed method has obtained the highest Pearson correlation coefficient at 14 of 17 segments, and the mean PCC of 85%. The experimental results highlight the segmentation accuracy of the proposed method as well as its availability to substitute for the manually annotated boundaries for the automatic assessment of cardiac function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The ACDC dataset is a public dataset (online website https://acdc.creatis.insa-lyon.fr/#challenges). The in-house dataset that has been used is confidential.

Notes

  1. Although motion tracking commercial software is available for cardiac cine imaging in clinically, manual correction of delineated contours used for tracing is often required, resulting in increased processing times and a significant degree of inter and intra-observer variability. Thus, the proposed model can be used to substitute for manual contouring the endocardium and the epicardium, to implement automatic myocardial stain estimation.

References

  1. Ouyang, D, He, B, Ghorbani, A, Yuan, N, Zou, J Y: Video-based ai for beat-to-beat assessment of cardiac function. Nature, 580(7802), 2020

  2. Cerqueira MD, Weissman NJ, Dilsizian V, Jacobs AK, Kaul S, Laskey WK, Pennell DJ, Rumberger JA, Ryan T, Verani MS: American Heart Association Writing Group on Myocardial Segmentation and Registration for Cardiac Imaging. Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart. A statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association. Circulation, 105(4):539–542, 2002

  3. Chen C, Qin C, Qiu H, Tarroni G, Duan J, Bai W, Rueckert D: Deep Learning for Cardiac Image Segmentation: A Review. Front Cardiovasc Med., 7:25, 2020

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Li F Y, Li W, Gao X, Xiao B: A Novel Framework With Weighted Decision Map Based on Convolutional Neural Network for Cardiac MR Segmentation. IEEE J Biomed Health Inform., 26(5):2228-2239, 2022

    Article  PubMed  Google Scholar 

  5. Xue W, Li J, Hu Z, Kerfoot E, Clough J, Oksuz I, Xu H, Grau V, Guo F, Ng M, Li X, Li Q, Liu L, Ma J, Grinias E, Tziritas G, Yan W, Atehortua A, Garreau M, Jang Y, Debus A, Ferrante E, Yang G, Hua T, Li S: Left Ventricle Quantification Challenge: A Comprehensive Comparison and Evaluation of Segmentation and Regression for Mid-Ventricular Short-Axis Cardiac MR Data. IEEE J Biomed Health Inform., 25(9):3541-3553, 2021

    Article  PubMed  PubMed Central  Google Scholar 

  6. Yang X, Zhang Y, Lo B, Wu D, Liao H, Zhang YT: DBAN: Adversarial Network with Multi-Scale Features for Cardiac MRI Segmentation. IEEE J Biomed Health Inform., 25(6):2018–28, 2021

    Article  PubMed  Google Scholar 

  7. Riffel J H, Schmucker K, Andre F, Ochs M, Hirschberg K, Schaub E, Fritz T, Mueller-Hennessen M, Giannitsis E, Katus H A, Friedrich M G: Cardiovascular magnetic resonance of cardiac morphology and function: impact of different strategies of contour drawing and indexing. Clin Res Cardiol., 108(4):411-429, 2019

    Article  PubMed  Google Scholar 

  8. Wang K N, Yang X, Miao J, Li L, Yao J, Zhou P, Xue W, Zhou G Q, Zhuang X, Ni D: AWSnet: An auto-weighted supervision attention network for myocardial scar and edema segmentation in multi-sequence cardiac magnetic resonance images. Med Image Anal., https://doi.org/10.1016/j.media.2022.102362, 2022

  9. Vesal S, Gu M, Maier A, Ravikumar N: Spatio-Temporal Multi-Task Learning for Cardiac MRI Left Ventricle Quantification. IEEE J Biomed Health Inform., 25(7):2698-2709, 2021

    Article  PubMed  Google Scholar 

  10. Ostvik A, Salte IM, Smistad E, Nguyen TM, Melichova D, Brunvand H, Haugaa K, Edvardsen T, Grenne B, Lovstakken L: Myocardial Function Imaging in Echocardiography Using Deep Learning. IEEE Trans Med Imaging, 40(5):1340-1351, 2021

    Article  PubMed  Google Scholar 

  11. Uslu F, Varela M, Boniface G, Mahenthran T, Chubb H, Bharath AA: LA-Net: A Multi-Task Deep Network for the Segmentation of the Left Atrium. IEEE Trans Med Imaging, 41(2):456-464, 2022

    Article  PubMed  Google Scholar 

  12. Jafari M H, Woudenberg N van, Luong C, Abolmaesumi P, Tsang T: Deep bayesian image segmentation for a more robust ejection fraction estimation. In: Proceedings - International Symposium on Biomedical Imaging, 1264–1272, 2021

  13. Dong S, Zhao J, Zhang M, Shi Z, Deng J, Shi Y, et al. DeU-Net: Deformable U-Net for 3D Cardiac MRI Video Segmentation. In: MICCAI 2020: Medical Image Computing and Computer Assisted Intervention, 98–107, 2020

  14. Dong S, Pan Z, Fu Y, Yang Q, Gao Y, Yu T, Shi Y, Zhuo C: DeU-Net 2.0: Enhanced deformable U-Net for 3D cardiac cine MRI segmentation. Med Image Anal., https://doi.org/10.1016/j.media.2022.102389, 2022

  15. Uslu F, Varela M: SA-net: A sequence aware network for the segmentation of the left atrium in cine MRI datasets. In: Proceedings - International Symposium on Biomedical Imaging, 766–775, 2021

  16. Yang D, Huang Q, Mikael K, Al’aref S Al, Axel L, Metaxas D: Mri-Based Characterization of Left Ventricle Dyssynchrony with Correlation to Crt Outcomes. In: Proceedings - International Symposium on Biomedical Imaging, 822–827, 2020

  17. Tan L K, Liew Y M, Lim E, McLaughlin R A: Convolutional neural network regression for short-axis left ventricle segmentation in cardiac cine MR sequences. Med Image Anal., 39:78–86, 2017

    Article  PubMed  Google Scholar 

  18. Robert R, Vanya V V, Wenjia B, Hideaki S, Paul M M, Chris P, Daniel R, Ben G: Automatic quality control of cardiac MRI segmentation in large-scale population imaging. MICCAI 2017 Cham, https://doi.org/10.1007/978-3-319-66182-7_82, 2017

  19. Ruijsink B, Puyol-Antón E, Oksuz I, Sinclair M, Bai W, Schnabel J A, Razavi R, King A P: Fully Automated, Quality-Controlled Cardiac Analysis From CMR: Validation and Large-Scale Application to Characterize Cardiac Function. JACC Cardiovasc Imaging., 13(3):684-695, 2020

    Article  PubMed  Google Scholar 

  20. Li K, Yu L, Heng P A: Towards reliable cardiac image segmentation: Assessing image-level and pixel-level segmentation quality via self-reflective references. Med Image Anal., https://doi.org/10.1016/j.media.2022.102426, 2022

  21. Zakeri A, Hokmabadi A, Ravikumar N, Frangi A F, Gooya A: A probabilistic deep motion model for unsupervised cardiac shape anomaly assessment. Med Image Anal., https://doi.org/10.1016/j.media.2021.102276, 2022

  22. Wang W, Shen J, Lu X, Hoi S, Ling H: Paying Attention to Video Object Pattern Understanding. IEEE Trans Pattern Anal Mach Intell., 43(7):2413-2428, 2021

    Article  PubMed  Google Scholar 

  23. Painchaud N, Skandarani Y, Judge T, Bernard O, Lalande A, Jodoin P M: Cardiac Segmentation With Strong Anatomical Guarantees. IEEE Trans Med Imaging., 39(11):3703-3713, 2020

    Article  PubMed  Google Scholar 

  24. Suinesiaputra A, Cowan B, Al-Agamy A, Elattar M, Ayache N, Fahmy A, Khalifa A, Medrano-Gracia P, Jolly M, Kadish A, Lee D, Margeta J, Warfield S, Young A: A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images. Med Image Anal., 18(1):50-62, 2014

    Article  PubMed  Google Scholar 

  25. Jha D, Smedsrud P, Riegler M, Johansen D, de Lange T, Halvorsen P, et al: ResUNet++: An Advanced Architecture for Medical Image Segmentation. 2019 IEEE International Symposium on Multimedia (ISM), 2019

  26. Alom M, Hasan M, Yakopcic C, Taha T, Asari V: Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation. https://doi.org/10.48550/arXiv.1802.06955, 2018

  27. Ozan O, Jo S, Loic L, Matthew L, Mattias H, Kazunari M, Kensaku M, Steven M, Nils Y,Bernhard K, Ben G, Daniel R: Attention U-Net: Learning Where to Look for the Pancreas. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.1804.03999, 2018

  28. Jieneng C, Yongyi L, Qihang Y, Xiangde L, Ehsan A, Yan W, Le L, Alan L, Yuyin Z: TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.2102.04306, 2021

  29. Hu J, Shen L, Albanie S, Sun G, Wu E: Squeeze-and-Excitation Networks. IEEE Trans Pattern Anal Mach Intell., 42(8):2011-2023, 2020

    Article  PubMed  Google Scholar 

  30. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J: UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation. IEEE Trans Med Imaging., 39(6):1856-1867, 2020

    Article  PubMed  Google Scholar 

  31. Ronneberger O, Fischer P, Brox T: U-Net: Convolutional Networks for Biomedical Image Segmentation. in Proc MICCAI Munich, Germany: Springer., 18;234–241, 2015

    Google Scholar 

  32. Huang H, Lin L, Tong R, Hu H, Zhang Q, Iwamoto Y, et al; UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation. ICASSP 2020, DOI:https://doi.org/10.1109/ICASSP40776.2020.9053405, Apr 19, 2020

    Article  Google Scholar 

  33. Diakogiannis F, Waldner F, Caccetta P, Wu C; ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing, 162:94–114, 2020

    Article  ADS  Google Scholar 

  34. Takikawa T, Acuna D, Jampani V, Fidler S: Gated-SCNN: Gated Shape CNNs for Semantic Segmentation. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.1907.05740, Jul 12, 2019

  35. Sun J, Darbehani F, Zaidi M, Wang B: SAUNet: Shape Attentive U-Net for Interpretable Medical Image Segmentation. DOI:https://doi.org/10.1007/978-3-030-59719-1_77, Jan 21, 2020

  36. Rohé M, Sermesant M, Pennec X: Automatic Multi-Atlas Segmentation of Myocardium with SVF-Net. Statistical Atlases and Computational Models of the Heart, 170–177, 2018

  37. Patravali J, Jain S, Chilamkurthy S: 2d-3d fully convolutional neural networks for cardiac mr segmentation. In: In: International Workshop on Statistical Atlases and Computional Models of the Heart, Springer, 130–139, 2017

  38. Simantiris G, Tziritas G: Cardiac MRI segmentation with a dilated CNN incorporating domain-specific constraints. IEEE J Sel Top Signal Process, 14 (6): 1235–1243, 2020

    Article  ADS  Google Scholar 

  39. Clement Z, Zhiming L, Alain L, Olivier H, Pierre-Marc J: GridNet with automatic shape prior registration for automatic MRI cardiac segmentation. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.1705.08943, 2017

  40. Du T, Lubomir B, Rob F, Lorenzo T, Manohar P: Learning Spatiotemporal Features with 3D Convolutional Networks. Computer Vision and Pattern Recognition, https://doi.org/10.48550/arXiv.1412.0767, 2014

  41. Yunbo W, Lu J, Ming-Hsuan Y, Li-Jia L, Mingsheng L, Li F: Eidetic 3D LSTM: A Model for Video Prediction and Beyond. In: International Conference on Learning Representations, 2019

Download references

Funding

This work was supported by the National Natural Science Foundation of China (No.62172288 and No. 61672362).

Author information

Author notes

  1. Yu Wang and Zheng Sun contribute equally to this work and share the first authorship.

Authors and Affiliations

  1. School of Biomedical Engineering, Fengtai District, Capital Medical University, 10 Xitoutiao, YouanmenwaiBeijing, 100069, China

    Yu Wang, Zheng Sun & Nan Zhang

  2. Beijing Key Laboratory of Fundamental Research On Biomechanics in Clinical Application, Fengtai District, Capital Medical University, 10 Xitoutiao, YouanmenwaiBeijing, 100069, China

    Yu Wang, Zheng Sun & Nan Zhang

  3. Department of Radiology and Nuclear Medicine, Xuanwu Hospital, Capital Medical University, 45 Changchun Street, Xicheng District, Beijing, 100053, China

    Zheng Sun & Jie Lu

  4. Department of Cardiology, Xuanwu Hospital, Capital Medical University, 45 Changchun Street, Xicheng District, Beijing, 100053, China

    Zhi Liu

Authors
  1. Yu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zheng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Guarantor of integrity of the entire study: Jie Lu, Nan Zhang Study design: Jie Lu, Nan Zhang Algorithm implementation: Yu Wang, Zheng Sun, Nan Zhang Literature research:Yu Wang, Zheng Sun Clinical studies: Zheng Sun, Zhi Liu, Jie Lu Experimental studies: Yu Wang, Zheng Sun, Nan Zhang Data acquisition: Zheng Sun, Zhi Liu Data analysis: Yu Wang, Zheng Sun, Nan Zhang Manuscript preparation, editing and review: Yu Wang, Zheng Sun, Zhi Liu, Jie Lu, Nan Zhang.

Corresponding authors

Correspondence to Jie Lu or Nan Zhang.

Ethics declarations

Ethical Approval

The Ethics Committee approved the study of Xuanwu Hospital of Capital Medical University (D2014044). By the Declaration of Helsinki, written informed consent was obtained from each subject before enrollment.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 17 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Sun, Z., Liu, Z. et al. A Motion-Aware DNN Model with Edge Focus Loss and Quality Control for Short-Axis Left Ventricle Segmentation of Cine MR Sequences. J Digit Imaging. Inform. med. (2024). https://doi.org/10.1007/s10278-023-00942-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10278-023-00942-6

Keywords

Search

Navigation