Skip to main content
SpringerLink
Log in
Book cover

Annual Conference on Medical Image Understanding and Analysis

MIUA 2020: Medical Image Understanding and Analysis pp 267–279Cite as

Prediction of Thrombectomy Functional Outcomes Using Multimodal Data

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1248))

Abstract

Recent randomised clinical trials have shown that patients with ischaemic stroke due to occlusion of a large intracranial blood vessel benefit from endovascular thrombectomy. However, predicting outcome of treatment in an individual patient remains a challenge. We propose a novel deep learning approach to directly exploit multimodal data (clinical metadata information, imaging data, and imaging biomarkers extracted from images) to estimate the success of endovascular treatment. We incorporate an attention mechanism in our architecture to model global feature inter-dependencies, both channel-wise and spatially. We perform comparative experiments using unimodal and multimodal data, to predict functional outcome (modified Rankin Scale score, mRS) and achieve 0.75 AUC for dichotomised mRS scores and 0.35 classification accuracy for individual mRS scores.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://www.isles-challenge.org.

  2. 2.

    https://www.mrclean-trial.org/home.html.

References

  1. Abraham, N., Khan, N.M.: A novel focal Tversky loss function with improved attention U-Net for lesion segmentation. In: ISBI, pp. 683–687. IEEE (2019)

    Google Scholar 

  2. Albers, G.W., et al.: Thrombectomy for stroke at 6 to 16 hours with selection by perfusion imaging. NEJM 378(8), 708–718 (2018)

    Article  Google Scholar 

  3. Asadi, H., et al.: Machine learning for outcome prediction of acute ischemic stroke post intra-arterial therapy. PLoS ONE 9(2), e88225 (2014)

    Article  MathSciNet  Google Scholar 

  4. Bacchi, S., et al.: Deep learning in the prediction of ischaemic stroke thrombolysis functional outcomes: a pilot study. Acad. Radiol. 27(2), e19–e23 (2019)

    Article  Google Scholar 

  5. Bentley, P., et al.: Prediction of stroke thrombolysis outcome using CT brain machine learning. NeuroImage Clin. 4, 635–640 (2014)

    Article  Google Scholar 

  6. Berkhemer, O.A., et al.: A randomized trial of intraarterial treatment for acute ischemic stroke. NEJM 372(1), 11–20 (2015)

    Article  Google Scholar 

  7. Boers, A., et al.: Automated cerebral infarct volume measurement in follow-up noncontrast CT scans of patients with acute ischemic stroke. AJN 34(8), 1522–1527 (2013)

    Google Scholar 

  8. Böhme, L., Madesta, F., Sentker, T., Werner, R.: Combining good old random forest and DeepLabv3+ for ISLES 2018 CT-based stroke segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 335–342. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_34

    Chapter  Google Scholar 

  9. Chawla, M., et al.: A method for automatic detection and classification of stroke from brain CT images. In: IEEEMBS, pp. 3581–3584. IEEE (2009)

    Google Scholar 

  10. Chen, S., et al.: Med3D: transfer learning for 3D medical image analysis. arXiv preprint arXiv:1904.00625 (2019)

  11. Choi, Y., et al.: Ensemble of deep convolutional neural networks for prognosis of ischemic stroke. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Winzeck, S., Handels, H. (eds.) BrainLes 2016. LNCS, pp. 231–243. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-55524-9_22

    Chapter  Google Scholar 

  12. Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49

    Chapter  Google Scholar 

  13. Fahed, R., et al.: Dwi-aspects (diffusion-weighted imaging-alberta stroke program early computed tomography scores) and dwi-flair (diffusion-weighted imaging-fluid attenuated inversion recovery) mismatch in thrombectomy candidates: An intrarater and interrater agreement study. Stroke 49(1), 223–227 (2018)

    Article  Google Scholar 

  14. Forkert, N.D., et al.: Multiclass support vector machine-based lesion mapping predicts functional outcome in ischemic stroke patients. PLoS ONE 10(6), e0129569 (2015)

    Article  Google Scholar 

  15. Fransen, P.S., et al.: MR CLEAN, a multicenter randomized clinical trial of endovascular treatment for acute ischemic stroke in the Netherlands: study protocol for a randomized controlled trial. Trials 15(1), 343 (2014)

    Article  Google Scholar 

  16. Goyal, M., et al.: Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. The Lancet 387(10029), 1723–1731 (2016)

    Article  Google Scholar 

  17. Gupta, N., Mittal, A.: Brain ischemic stroke segmentation: a survey. J. Multi Disciplinary Eng. Technol. 8(1), 1 (2014)

    Google Scholar 

  18. He, K., et al.: Deep residual learning for image recognition. In: CVPR (2016)

    Google Scholar 

  19. Heo, J., et al.: Machine learning-based model can predict stroke outcome. Stroke 49(Suppl 1), A194–A194 (2018)

    Google Scholar 

  20. Hilbert, A., et al.: Data-efficient deep learning of radiological image data for outcome prediction after endovascular treatment of patients with acute ischemic stroke. Comput. Biol. Med., 103516 (2019)

    Google Scholar 

  21. Hu, J., et al.: Squeeze-and-excitation networks. In: CVPR (2018)

    Google Scholar 

  22. Isensee, F., et al.: Brain tumor segmentation and radiomics survival prediction: contribution to BRATS 2017 challenge. In: MICCAIBW, pp. 287–297 (2017)

    Google Scholar 

  23. Jansen, I.G., et al.: Endovascular treatment for acute ischaemic stroke in routine clinical practice: prospective, observational cohort study (MR CLEAN Registry). BMJ 360, k949 (2018)

    Article  Google Scholar 

  24. Kamnitsas, K., et al.: Ensembles of multiple models and architectures for robust brain tumour segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 450–462. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75238-9_38

    Chapter  Google Scholar 

  25. Lin, T.Y., et al.: Focal loss for dense object detection. In: CVPR (2017)

    Google Scholar 

  26. Lisowska, A., et al.: Context-aware convolutional neural networks for stroke sign detection in non-contrast CT scans. In: Valdés Hernández, M., González-Castro, V. (eds.) MIUA 2017. CCIS, vol. 723, pp. 494–505. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60964-5_43

    Chapter  Google Scholar 

  27. Liu, S., et al.: On the design of convolutional neural networks for automatic detection of Alzheimer’s disease. In: NeurIPS ML4H (2019)

    Google Scholar 

  28. Maier, O., et al.: Ischemic stroke lesion segmentation in multi-spectral MR images with support vector machine classifiers. In: Medical Imaging 2014: Computer-Aided Diagnosis, vol. 9035, p. 903504. ISOP (2014)

    Google Scholar 

  29. Maier, O., et al.: ISLES 2015-A public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI. MIA 35, 250–269 (2017)

    Google Scholar 

  30. Maier, O., Handels, H.: Predicting Stroke Lesion and Clinical Outcome with Random Forests. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Winzeck, S., Handels, H. (eds.) BrainLes 2016. LNCS, pp. 219–230. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-55524-9_21

    Chapter  Google Scholar 

  31. Matesin, M., et al.: A rule-based approach to stroke lesion analysis from CT brain images. In: ISPA, pp. 219–223. IEEE (2001)

    Google Scholar 

  32. McKinley, R., et al.: Fully automated stroke tissue estimation using random forest classifiers (FASTER). JCBFM 37(8), 2728–2741 (2017)

    Google Scholar 

  33. Nishi, H., et al.: Predicting clinical outcomes of large vessel occlusion before mechanical thrombectomy using machine learning. Stroke 50(9), 2379–2388 (2019)

    Article  Google Scholar 

  34. Parisot, S., et al.: Disease prediction using graph convolutional networks: application to autism spectrum disorder and Alzheimer’s disease. MIA 48, 117–130 (2018)

    Google Scholar 

  35. Pinto, A., et al.: Enhancing clinical MRI perfusion maps with data-driven maps of complementary nature for lesion outcome prediction. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 107–115. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_13

    Chapter  Google Scholar 

  36. Rekik, I., et al.: Medical image analysis methods in MR/CT-imaged acute-subacute ischemic stroke lesion: Segmentation, prediction and insights into dynamic evolution simulation models. A critical appraisal. NeuroImage Clin. 1(1), 164–178 (2012)

    Article  Google Scholar 

  37. Renowden, S.: Imaging in stroke and vascular disease–part 1: ischaemic stroke. Pract. Neurol. 14(2), 77–87 (2014)

    Article  Google Scholar 

  38. Roy, A.G., Navab, N., Wachinger, C.: Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 421–429. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_48

    Chapter  Google Scholar 

  39. Stroke Association: State of the Nation: stroke statistics (2018). https://www.stroke.org.uk/resources/state-nation-stroke-statistics. Accessed Nov 2019

  40. Van Os, H.J., et al.: Predicting outcome of endovascular treatment for acute ischemic stroke: potential value of machine learning algorithms. Front. Neurol. 9, 784 (2018)

    Article  Google Scholar 

  41. Venema, E., et al.: Selection of patients for intra-arterial treatment for acute ischaemic stroke: development and validation of a clinical decision tool in two randomised trials. BMJ 357, j1710 (2017)

    Article  Google Scholar 

  42. Weimar, C., Ziegler, A., König, I.R., Diener, H.-C.: Predicting functional outcome and survival after acute ischemic stroke. J. Neurol. 249(7), 888–895 (2002). https://doi.org/10.1007/s00415-002-0755-8

    Article  Google Scholar 

  43. WHO: The top 10 causes of death (2018). https://www.who.int/en/news-room/fact-sheets/detail/the-top-10-causes-of-death. Accessed Nov 2019

  44. Winzeck, S., et al.: ISLES 2016 and 2017-benchmarking ischemic stroke lesionoutcome prediction based on multispectral MRI. Front. Neurol. 9, 679 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the MR CLEAN Registry team: Prof. Aad van der Lugt, Prof. Diederik W.J. Dippel, Prof. Charles B.L.M. Majoie, Prof. Wim H. van Zwam and Prof. Robert J. van Oostenbrugge for providing the data. Zeynel Samak gratefully acknowledges funding from Ministry of Education (1416/YLSY), the Republic of Turkey. The Titan V used for this research was donated by the NVIDIA Corporation.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Bristol, Bristol, UK

    Zeynel A. Samak & Majid Mirmehdi

  2. Translational Health Sciences, University of Bristol, Bristol, UK

    Philip Clatworthy

  3. Stroke Neurology, Southmead Hospital, North Bristol NHS Trust, Bristol, UK

    Philip Clatworthy

Authors
  1. Zeynel A. Samak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Philip Clatworthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Majid Mirmehdi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zeynel A. Samak .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Bartłomiej W. Papież

  2. University of Oxford, Oxford, UK

    Ana I. L. Namburete

  3. University of Oxford, Oxford, UK

    Mohammad Yaqub

  4. University of Oxford, Oxford, UK

    J. Alison Noble

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Samak, Z.A., Clatworthy, P., Mirmehdi, M. (2020). Prediction of Thrombectomy Functional Outcomes Using Multimodal Data. In: Papież, B., Namburete, A., Yaqub, M., Noble, J. (eds) Medical Image Understanding and Analysis. MIUA 2020. Communications in Computer and Information Science, vol 1248. Springer, Cham. https://doi.org/10.1007/978-3-030-52791-4_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-52791-4_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-52790-7

  • Online ISBN: 978-3-030-52791-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation