Abstract
The segmentation of the left ventricle in echocardiograms is crucial for diagnosing cardiovascular diseases. However, current deep learning methods typically focus on 2D segmentations and overlook the temporal information in ultrasound sequences. This choice might be caused by the scarcity of manual annotations, which are typically limited to end-diastole and end-systole frames. Therefore, we propose a method that trains temporally consistent segmentation models from sparsely labeled echocardiograms. We leverage image registration to generate pseudo-labels for unlabeled frames enabling the training of 3D models. Using a state-of-the-art convolutional neural network, 3D nnU-Net, we delineate the left ventricle (LV) cavity, LV myocardium, and left atrium. Evaluation on the CAMUS dataset demonstrates the quality and robustness of the generated pseudo-labels, serving as effective training data for subsequent segmentation. Additionally, we evaluate the segmentation model both intrinsically, measuring accuracy and temporal consistency, and extrinsically, estimating cardiac function markers like ejection fraction and left ventricular volumes. The results show accurate delineation of the cardiac structures that evolves smoothly over time, effectively demonstrating the model’s accuracy and temporal consistency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
To aid readability, it may be worth specifying that “segmentations” and “labels” are used interchangeably throughout the paper.
- 2.
The code is publicly available at https://github.com/matteo-tafuro/temporally-consistent-echosegmentation.
References
Armstrong, A.C., et al.: Quality control and reproducibility in M-mode, two-dimensional, and speckle tracking echocardiography acquisition and analysis: the CARDIA study, year 25 examination experience. Echocardiography 32(8), 1233–1240 (2014). https://doi.org/10.1111/echo.12832
Chen, C., et al.: Deep learning for cardiac image segmentation: a review. Front. Cardiovasc. Med. 7 (2020). https://doi.org/10.3389/fcvm.2020.00025
Chen, S., Ma, K., Zheng, Y.: Tan: Temporal affine network for real-time left ventricle anatomical structure analysis based on 2D ultrasound videos. ArXiv (2019). https://doi.org/10.48550/ARXIV.1904.00631
Chen, Y., Zhang, X., Haggerty, C.M., Stough, J.V.: Assessing the generalizability of temporally coherent echocardiography video segmentation. In: Išgum, I., Landman, B.A. (eds.) Medical Imaging 2021: Image Processing. vol. 11596, p. 115961O. International Society for Optics and Photonics, SPIE (2021). https://doi.org/10.1117/12.2580874
Dai, W., Li, X., Ding, X., Cheng, K.T.: Cyclical self-supervision for semi-supervised ejection fraction prediction from echocardiogram videos. IEEE Trans. Med. Imaging 42(5), 1446–1461 (2023). https://doi.org/10.1109/TMI.2022.3229136
de Vos, B.D., Berendsen, F.F., Viergever, M.A., Sokooti, H., Staring, M., Išgum, I.: A deep learning framework for unsupervised affine and deformable image registration. Med. Image Anal. 52, 128–143 (2019). https://doi.org/10.1016/j.media.2018.11.010
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2020). https://doi.org/10.1038/s41592-020-01008-z
Leclerc, S., et al.: Deep learning for segmentation using an open large-scale dataset in 2D echocardiography. IEEE Trans. Med. Imaging 38(9), 2198–2210 (2019). https://doi.org/10.1109/tmi.2019.2900516
Li, M., Wang, C., Zhang, H., Yang, G.: MV-RAN: multiview recurrent aggregation network for echocardiographic sequences segmentation and full cardiac cycle analysis. Comput. Biol. Med. 120, 103728 (2020). https://doi.org/10.1016/j.compbiomed.2020.103728
Moal, O., et al.: Explicit and automatic ejection fraction assessment on 2D cardiac ultrasound with a deep learning-based approach. Comput. Biol. Med. 146, 105637 (2022). https://doi.org/10.1016/j.compbiomed.2022.105637
Ouyang, D., et al.: Video-based AI for beat-to-beat assessment of cardiac function. Nature 580(7802), 252–256 (2020). https://doi.org/10.1038/s41586-020-2145-8
Painchaud, N., Duchateau, N., Bernard, O., Jodoin, P.M.: Echocardiography segmentation with enforced temporal consistency. IEEE Trans. Med. Imaging 41(10), 2867–2878 (2022). https://doi.org/10.1109/TMI.2022.3173669
Rueckert, D.: Nonrigid registration using free-form deformations: application to breast MRI images. IEEE Trans. Med. Imaging 18(8), 712–721 (1999). https://doi.org/10.1109/42.796284
Schuuring, M.J., Išgum, I., Cosyns, B., Chamuleau, S.A.J., Bouma, B.J.: Routine echocardiography and artificial intelligence solutions. Front. Cardiovasc. Med. 8, 648877 (2021)
Sfakianakis, C., Simantiris, G., Tziritas, G.: GUDU: geometrically-constrained ultrasound data augmentation in U-net for echocardiography semantic segmentation. Biomed. Signal Process. Control 82, 104557 (2023). https://doi.org/10.1016/j.bspc.2022.104557
Thomas, L., Marwick, T.H., Popescu, B.A., Donal, E., Badano, L.P.: Left atrial structure and function, and left ventricular diastolic dysfunction: JACC state-of-the-art review. J. Am. Coll. Cardiol. 73(15), 1961–1977 (2019). https://doi.org/10.1016/j.jacc.2019.01.059
Wang, C., et al.: Pseudo-labeled auto-curriculum learning for semi-supervised keypoint localization (2022)
Wei, H., et al.: Temporal-consistent segmentation of echocardiography with co-learning from appearance and shape. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 623–632. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_60
Wei, H., Ma, J., Zhou, Y., Xue, W., Ni, D.: Co-learning of appearance and shape for precise ejection fraction estimation from echocardiographic sequences. Med. Image Anal. 84, 102686 (2023). https://doi.org/10.1016/j.media.2022.102686
Xia, Y., et al.: 3D semi-supervised learning with uncertainty-aware multi-view co-training (2020)
Xue, W., Cao, H., Ma, J., Bai, T., Wang, T., Ni, D.: Improved segmentation of echocardiography with orientation-congruency of optical flow and motion-enhanced segmentation. IEEE J. Biomed. Health Inform. 26(12), 6105–6115 (2022). https://doi.org/10.1109/JBHI.2022.3221429
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tafuro, M., Jansen, G., Išgum, I. (2023). Temporally Consistent Segmentations from Sparsely Labeled Echocardiograms Using Image Registration for Pseudo-labels Generation. In: Kainz, B., Noble, A., Schnabel, J., Khanal, B., Müller, J.P., Day, T. (eds) Simplifying Medical Ultrasound. ASMUS 2023. Lecture Notes in Computer Science, vol 14337. Springer, Cham. https://doi.org/10.1007/978-3-031-44521-7_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-44521-7_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44520-0
Online ISBN: 978-3-031-44521-7
eBook Packages: Computer ScienceComputer Science (R0)
