Abstract
Accurate and consistent predictions of echocardiography parameters are important for cardiovascular diagnosis and treatment. In particular, segmentations of the left ventricle can be used to derive ventricular volume, ejection fraction (EF) and other relevant measurements. In this paper we propose a new automated method called EchoGraphs for predicting ejection fraction and segmenting the left ventricle by detecting anatomical keypoints. Models for direct coordinate regression based on Graph Convolutional Networks (GCNs) are used to detect the keypoints. GCNs can learn to represent the cardiac shape based on local appearance of each keypoint, as well as global spatial and temporal structures of all keypoints combined. We evaluate our EchoGraphs model on the EchoNet benchmark dataset. Compared to semantic segmentation, GCNs show accurate segmentation and improvements in robustness and inference run-time. EF is computed simultaneously to segmentations and our method also obtains state-of-the-art ejection fraction estimation. Source code is available online: https://github.com/guybenyosef/EchoGraphs.
S. Thomas and A. Gilbert contributed equally
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmedt-Aristizabal, D., Armin, M.A., Denman, S., Fookes, C., Petersson, L.: Graph-based deep learning for medical diagnosis and analysis: past, present and future. Sensors 21(14), 4758 (2021). https://doi.org/10.3390/s21144758
Baltabaeva, A., et al.: Regional left ventricular deformation and geometry analysis provides insights in myocardial remodelling in mild to moderate hypertension. Eur. J. Echocardiogr. 9(4), 501–508 (2008). https://doi.org/10.1016/j.euje.2007.08.004
Bouritsas, G., Bokhnyak, S., Ploumpis, S., Bronstein, M., Zafeiriou, S.: Neural 3D morphable models: spiral convolutional networks for 3D shape representation learning and generation. In: The IEEE International Conference on Computer Vision (ICCV) (2019)
Gilbert, A., Marciniak, M., Rodero, C., Lamata, P., Samset, E., Mcleod, K.: Generating synthetic labeled data from existing anatomical models: an example with echocardiography segmentation. IEEE Trans. Med. Imaging 40(10), 2783–2794 (2021). https://doi.org/10.1109/TMI.2021.3051806
Gong, S., Chen, L., Bronstein, M., Zafeiriou, S.: SpiralNet++: a fast and highly efficient mesh convolution operator. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (CVPR) (2019)
Gopinath, K., Desrosiers, C., Lombaert, H.: Graph domain adaptation for alignment-invariant brain surface segmentation. In: Sudre, C.H., et al. (eds.) UNSURE/GRAIL -2020. LNCS, vol. 12443, pp. 152–163. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60365-6_15
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021). https://doi.org/10.1038/s41592-020-01008-z
Jafari, M.H., et al.: A unified framework integrating recurrent fully-convolutional networks and optical flow for segmentation of the left ventricle in echocardiography data. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 29–37. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_4
Kazemi Esfeh, M.M., Luong, C., Behnami, D., Tsang, T., Abolmaesumi, P.: A deep Bayesian video analysis framework: towards a more robust estimation of ejection fraction. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 582–590. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_56
Leclerc, S., et al.: Deep Learning Segmentation in 2D echocardiography using the CAMUS dataset : automatic assessment of the anatomical shape validity. In: International Conference Medical Imaging with Deep Learning - Extended Abstract Track (2019)
Marciniak, M., et al.: Septal curvature as a robust and reproducible marker for basal septal hypertrophy. J. Hypertens. 39(7), 1421 (2021). https://doi.org/10.1097/HJH.0000000000002813
Oktay, O., et al.: Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation. IEEE Trans. Med. Imaging 37(2), 384–395 (2017)
Østvik, A., Smistad, E., Espeland, T., Berg, E.A.R., Lovstakken, L.: Automatic myocardial strain imaging in echocardiography using deep learning. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 309–316. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_35
Ouyang, D., et al.: Interpretable AI for beat-to-beat cardiac function assessment. Nature 580(7802), 252–256 (2020). https://doi.org/10.1038/s41586-020-2145-8
Payer, C., Štern, D., Bischof, H., Urschler, M.: Integrating spatial configuration into heatmap regression based CNNs for landmark localization. Med. Image Anal. 54, 207–219 (2019). https://doi.org/10.1016/j.media.2019.03.007
Reynaud, H., Vlontzos, A., Hou, B., Beqiri, A., Leeson, P., Kainz, Bernhard: Ultrasound video transformers for cardiac ejection fraction estimation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12906, pp. 495–505. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87231-1_48
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018). https://doi.org/10.1109/CVPR.2018.00474
Savarese, G., Stolfo, D., Sinagra, G., Lund, L.H.: Heart failure with mid-range or mildly reduced ejection fraction. Nature Rev. Cardiol. 19, 100–116 (2022). https://doi.org/10.1038/s41569-021-00605-5
Smistad, E., et al.: Real-time automatic ejection fraction and foreshortening detection using deep learning. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 67(12), 2595–2604 (2020). https://doi.org/10.1109/TUFFC.2020.2981037
Tian, Z., et al.: Graph-convolutional-network-based interactive prostate segmentation in MR images. Med. Phys. 47(9), 4164–4176 (2020)
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., Paluri, M.: A closer look at spatiotemporal convolutions for action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6450–6459 (2018)
Wang, J., Yan, S., Xiong, Y., Lin, D.: Motion guided 3D pose estimation from videos. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12358, pp. 764–780. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58601-0_45
Yan, S., Xiong, Y., D, L.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Proceedings of the AAAI Conference Artificial Intelligence (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (mp4 3763 KB)
Supplementary material 2 (mp4 6442 KB)
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Thomas, S., Gilbert, A., Ben-Yosef, G. (2022). Light-weight Spatio-Temporal Graphs for Segmentation and Ejection Fraction Prediction in Cardiac Ultrasound. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13434. Springer, Cham. https://doi.org/10.1007/978-3-031-16440-8_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-16440-8_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16439-2
Online ISBN: 978-3-031-16440-8
eBook Packages: Computer ScienceComputer Science (R0)
