Skip to main content
SpringerLink
Log in

Use of quantitative angiographic methods with a data-driven model to evaluate reperfusion status (mTICI) during thrombectomy

  • Diagnostic Neuroradiology
  • Published:
Neuroradiology Aims and scope Submit manuscript

Abstract

Purpose

Intra-procedural assessment of reperfusion during mechanical thrombectomy (MT) for emergent large vessel occlusion (LVO) stroke is traditionally based on subjective evaluation of digital subtraction angiography (DSA). However, semi-quantitative diagnostic tools which encode hemodynamic properties in DSAs, such as angiographic parametric imaging (API), exist and may be used for evaluation of reperfusion during MT. The objective of this study was to use data-driven approaches, such as convolutional neural networks (CNNs) with API maps, to automatically assess reperfusion in the neuro-vasculature during MT procedures based on the modified thrombolysis in cerebral infarction (mTICI) scale.

Methods

DSAs from patients undergoing MTs of anterior circulation LVOs were collected, temporally cropped to isolate late arterial and capillary phases, and quantified using API peak height (PH) maps. PH maps were normalized to reduce injection variability. A CNN was developed, trained, and tested to classify PH maps into 2 outcomes (mTICI 0,1,2a/mTICI 2b,2c,3) or 3 outcomes (mTICI 0,1,2a/mTICI 2b/mTICI 2c,3), respectively. Ensembled networks were used to combine information from multiple views (anteroposterior and lateral).

Results

The study included 383 DSAs. For the 2-outcome classification, average accuracy was 81.0% (95% CI, 79.0–82.9%), and the area under the receiver operating characteristic curve (AUROC) was 0.86 (0.84–0.88). For the 3-outcome classification, average accuracy was 64.0% (62.0–66.0), and AUROC values were 0.85 (0.83–0.87), 0.74 (0.71–0.77), and 0.78 (0.76–0.81) for the mTICI 0,1,2a, mTICI 2b, and mTICI 2c,3 classes, respectively.

Conclusion

This study demonstrated the feasibility of using hemodynamic information in API maps with data-driven models to autonomously assess intra-procedural reperfusion during MT.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

AIS:

Acute ischemic stroke

AP:

Anteroposterior

API:

Angiographic parametric imaging

AUROC:

Area under the ROC curve

CAM:

Class activation map

CNN:

Convolutional neural network

CT:

Computed tomography

DSA:

Digital subtraction angiography

ICA:

Internal carotid artery

LVO:

Large vessel occlusion

MCA:

Middle cerebral artery

MCCV:

Monte Carlo cross-validation

MCC:

Matthews correlation coefficient

MT:

Mechanical thrombectomy

mTICI:

Modified TICI

NIHSS:

NIH stroke score

TDC:

Time density curve

TICI:

Thrombolysis in cerebral infarction

PH:

Peak height

ROC:

Receiver operating characteristic

References

  1. Rennert RC et al (2019) Epidemiology, natural history, and clinical presentation of large vessel ischemic stroke. Neurosurgery 85(suppl_1):S4–S8

    Article  Google Scholar 

  2. Waqas M, Mokin M, Primiani CT, Gong AD, Rai HH, Chin F, Rai AT, Levy EI, Siddiqui AH (2020) Large vessel occlusion in acute ischemic stroke patients: a dual-center estimate based on a broad definition of occlusion site. J Stroke Cerebrovasc Dis 29(2):104504

    Article  Google Scholar 

  3. Waqas M, Rai AT, Vakharia K, Chin F, Siddiqui AH (2020) Effect of definition and methods on estimates of prevalence of large vessel occlusion in acute ischemic stroke: a systematic review and meta-analysis. J Neurointerv Surg 12(3):260–265

    Article  Google Scholar 

  4. Smith WS, Sung G, Saver J, Budzik R, Duckwiler G, Liebeskind DS, Lutsep HL, Rymer MM, Higashida RT, Starkman S, Gobin YP, Multi MERCI Investigators, Frei D, Grobelny T, Hellinger F, Huddle D, Kidwell C, Koroshetz W, Marks M, Nesbit G, Silverman IE (2008) Mechanical thrombectomy for acute ischemic stroke: final results of the multi MERCI trial. Stroke 39(4):1205–1212

    Article  Google Scholar 

  5. Berlis A, Lutsep H, Barnwell S, Norbash A, Wechsler L, Jungreis CA, Woolfenden A, Redekop G, Hartmann M, Schumacher M (2004) Mechanical thrombolysis in acute ischemic stroke with endovascular photoacoustic recanalization. Stroke 35(5):1112–1116

    Article  Google Scholar 

  6. Smith WS, Sung G, Starkman S, Saver JL, Kidwell CS, Gobin YP, Lutsep HL, Nesbit GM, Grobelny T, Rymer MM, Silverman IE, Higashida RT, Budzik RF, Marks MP (2005) Safety and efficacy of mechanical embolectomy in acute ischemic stroke: results of the MERCI trial. Stroke 36(7):1432–1438

    Article  Google Scholar 

  7. Bourcier R, Goyal M, Liebeskind DS, Muir KW, Desal H, Siddiqui AH, Dippel DWJ, Majoie CB, van Zwam WH, Jovin TG, Levy EI, Mitchell PJ, Berkhemer OA, Davis SM, Derraz I, Donnan GA, Demchuk AM, van Oostenbrugge RJ, Kelly M, Roos YB, Jahan R, van der Lugt A, Sprengers M, Velasco S, Lycklama à Nijeholt GJ, Ben Hassen W, Burns P, Brown S, Chabert E, Krings T, Choe H, Weimar C, Campbell BCV, Ford GA, Ribo M, White P, Cloud GC, San Roman L, Davalos A, Naggara O, Hill MD, Bracard S, for the HERMES Trialists Collaboration (2019) Association of time from stroke onset to groin puncture with quality of reperfusion after mechanical thrombectomy: a meta-analysis of individual patient data from 7 randomized clinical trials. JAMA Neurology 76(4):405–411

    Article  Google Scholar 

  8. Evans MR et al (2017) Revolution in acute ischaemic stroke care: a practical guide to mechanical thrombectomy. Pract Neurol 17(4):252–265

    Article  Google Scholar 

  9. Rava RA et al (2020) Assessment of computed tomography perfusion software in predicting spatial location and volume of infarct in acute ischemic stroke patients: a comparison of Sphere, Vitrea, and RAPID. J Neurointerv Surg

  10. Higashida RT, Furlan AJ (2003) Trial design and reporting standards for intra-arterial cerebral thrombolysis for acute ischemic stroke. Stroke 34(8):e109–e137

    Article  Google Scholar 

  11. Almekhlafi MA, Mishra S, Desai JA, Nambiar V, Volny O, Goel A, Eesa M, Demchuk AM, Menon BK, Goyal M (2014) Not all “successful” angiographic reperfusion patients are an equal validation of a modified TICI scoring system. Interv Neuroradiol 20(1):21–27

    Article  Google Scholar 

  12. Kallmes DF (2012) TICI: if you are not confused, then you are not paying attention. Am J Neuroradiol 33(5):975–976

    Article  CAS  Google Scholar 

  13. Sattar A et al (2014) E-011 TICI quantified: automated cerebral revascularization grading in acute ischemic stroke. J NeuroInterv Surg 6(Suppl 1):A43–A43

    Article  Google Scholar 

  14. De Leacy RA et al (2019) Wide-neck bifurcation aneurysms of the middle cerebral artery and basilar apex treated by endovascular techniques: a multicentre, core lab adjudicated study evaluating safety and durability of occlusion (BRANCH). Journal of NeuroInterventional Surgery 11(1):31–36

    Article  Google Scholar 

  15. Rezek I, Mousan G, Wang Z, Murad MH, Kallmes DF (2013) Effect of core laboratory and multiple-reader interpretation of angiographic images on follow-up outcomes of coiled cerebral aneurysms: a systematic review and meta-analysis. Am J Neuroradiol 34(7):1380–1384

    Article  CAS  Google Scholar 

  16. Zhang G, Treurniet KM, Jansen IGH, Emmer BJ, van den Berg R, Marquering HA, Uyttenboogaart M, Jenniskens SFM, Roos YBWEM, van Doormaal PJ, van Es ACGM, van der Lugt A, Vos JA, Lycklama à Nijeholt GJ, van Zwam WH, Shi H, Yoo AJ, Dippel DWJ, Majoie CBLM, for the MR CLEAN Registry Investigators (2018) Operator versus core lab adjudication of reperfusion after endovascular treatment of acute ischemic stroke. Stroke 49(10):2376–2382

    Article  Google Scholar 

  17. Bhurwani, M.M.S., et al. Feasibility study of deep neural networks to classify intracranial aneurysms using angiographic parametric imaging. In Medical Imaging 2019: Computer-Aided Diagnosis. 2019. International Society for Optics and Photonics

  18. Thompson HK Jr et al (1964) Indicator transit time considered as a gamma variate. Circ Res 14(6):502–515

    Article  Google Scholar 

  19. Albers GW, Wald MJ, Mlynash M, Endres J, Bammer R, Straka M, Maier A, Hinson HE, Sheth KN, Taylor Kimberly W, Molyneaux BJ (2019) Automated calculation of Alberta stroke program early CT score: validation in patients with large hemispheric infarct. Stroke 50(11):3277–3279

    Article  Google Scholar 

  20. Bhurwani, M.M.S., et al., Feasibility study for use of angiographic parametric imaging and deep neural networks for intracranial aneurysm occlusion prediction. Journal of NeuroInterventional Surgery, 2019

  21. Mokin M, Khalessi AA, Mocco J, Lanzino G, Dumont TM, Hanel RA, Lopes DK, Fessler RD, Ringer AJ, Bendok BR, Veznedaroglu E, Siddiqui AH, Hopkins LN, Levy EI (2014) Endovascular treatment of acute ischemic stroke: the end or just the beginning? Neurosurg Focus 36(1):E5

    Article  Google Scholar 

  22. Zaidat OO, Yoo AJ, Khatri P, Tomsick TA, von Kummer R, Saver JL, Marks MP, Prabhakaran S, Kallmes DF, Fitzsimmons BF, Mocco J, Wardlaw JM, Barnwell SL, Jovin TG, Linfante I, Siddiqui AH, Alexander MJ, Hirsch JA, Wintermark M, Albers G, Woo HH, Heck DV, Lev M, Aviv R, Hacke W, Warach S, Broderick J, Derdeyn CP, Furlan A, Nogueira RG, Yavagal DR, Goyal M, Demchuk AM, Bendszus M, Liebeskind DS, Cerebral Angiographic Revascularization Grading (CARG) Collaborators, STIR Revascularization working group, STIR Thrombolysis in Cerebral Infarction (TICI) Task Force (2013) Recommendations on angiographic revascularization grading standards for acute ischemic stroke: a consensus statement. Stroke 44(9):2650–2663

    Article  Google Scholar 

  23. Chollet, F., Keras. 2015

  24. Bluemke DA et al (2019) Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers-from the Radiology Editorial Board. Radiology 294:192515

    Google Scholar 

  25. Xu Q-S, Liang Y-Z (2001) Monte Carlo cross validation. Chemom Intell Lab Syst 56(1):1–11

    Article  CAS  Google Scholar 

  26. Storn R, Price K (1997) Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359

    Article  Google Scholar 

  27. Virtanen P et al (2020) SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat Methods 17(3):261–272

    Article  CAS  Google Scholar 

  28. Zhou, B., et al. Learning deep features for discriminative localization. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016

  29. Matthews BW (1975) Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim Biophys Acta (BBA)-Protein Struct 405(2):442–451

    Article  CAS  Google Scholar 

  30. Chicco D, Jurman G (2020) The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21(1):6

    Article  Google Scholar 

  31. Goyal M, Menon BK, van Zwam WH, Dippel DWJ, Mitchell PJ, Demchuk AM, Dávalos A, Majoie CBLM, van der Lugt A, de Miquel MA, Donnan GA, Roos YBWEM, Bonafe A, Jahan R, Diener HC, van den Berg LA, Levy EI, Berkhemer OA, Pereira VM, Rempel J, Millán M, Davis SM, Roy D, Thornton J, Román LS, Ribó M, Beumer D, Stouch B, Brown S, Campbell BCV, van Oostenbrugge RJ, Saver JL, Hill MD, Jovin TG (2016) Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. Lancet 387(10029):1723–1731

    Article  Google Scholar 

  32. Dargazanli C, Fahed R, Blanc R, Gory B, Labreuche J, Duhamel A, Marnat G, Saleme S, Costalat V, Bracard S, Desal H, Mazighi M, Consoli A, Piotin M, Lapergue B, Redjem H, Ciccio G, Smajda S, Desilles JP, Rodesch G, Coskun O, di Maria F, Bourdain F, Decroix JP, Wang A, Tchikviladze M, Evrard S, Eker O, Turjman F, Labeyrie PE, Riva R, Mounayer C, Saleme S, Bonafé A, Gascou G, Tonnelet R, Derelle AL, Anxionnat R, Bourcier R, Daumas-Duport B, Berge J, Barreau X, Djemmane L (2018) Modified thrombolysis in cerebral infarction 2C/thrombolysis in cerebral infarction 3 reperfusion should be the aim of mechanical thrombectomy: insights from the ASTER trial (Contact Aspiration Versus Stent Retriever for Successful Revascularization). Stroke 49(5):1189–1196

    Article  Google Scholar 

  33. Cramer H (1946) Mathematical methods of statistics. Princeton University Press, Princeton

    Google Scholar 

  34. Powers, D.M., Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. 2011

  35. Podgorsak, A., et al. Initial evaluation of the use of a convolutional neural network to determine coronary artery disease severity using computed tomography angiography. in Medical physics. 2019. WILEY 111 RIVER ST, HOBOKEN 07030–5774, NJ USA

Download references

Contributors

MMSB, KVS, MM, AHS, and CNI conceived and designed the research.

JMD, EIL, KVS, and AHS performed all the clinical procedures.

MMSB, MW, RAR, and CNI collected the data.

MMSB analyzed the data.

MMSB performed the statistical analysis.

CNI handled funding and supervision.

MMSB drafted the manuscript.

All authors made critical revisions of the manuscript and reviewed the final version.

Code availability

Additional data may be made available by contacting the corresponding author.

Data availability

Additional data may be made available by contacting the corresponding author.

Funding

This work is supported by the James H. Cummings Foundation and the University at Buffalo Clinical and Translational Science Institute (CTSI).

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, University at Buffalo, Buffalo, NY, 14228, USA

    Mohammad Mahdi Shiraz Bhurwani, Ryan A. Rava, Alexander R. Podgorsak & Ciprian N. Ionita

  2. Canon Stroke and Vascular Research Center, Buffalo, NY, 14203, USA

    Mohammad Mahdi Shiraz Bhurwani, Kenneth V. Snyder, Muhammad Waqas, Ryan A. Rava, Alexander R. Podgorsak, Felix Chin, Jason M. Davies, Elad I. Levy, Adnan H. Siddiqui & Ciprian N. Ionita

  3. Department of Neurosurgery, University at Buffalo, Buffalo, NY, 14203, USA

    Kenneth V. Snyder, Muhammad Waqas, Felix Chin, Jason M. Davies, Elad I. Levy, Adnan H. Siddiqui & Ciprian N. Ionita

  4. Department of Neurosurgery and Brain Repair, University of South Florida, Tampa, FL, 33606, USA

    Maxim Mokin

Authors
  1. Mohammad Mahdi Shiraz Bhurwani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kenneth V. Snyder
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Waqas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maxim Mokin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ryan A. Rava
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alexander R. Podgorsak
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Felix Chin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jason M. Davies
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Elad I. Levy
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Adnan H. Siddiqui
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Ciprian N. Ionita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ciprian N. Ionita.

Ethics declarations

Competing interests

Maxim Mokin—Grants: Principal investigator NIH R21NS109575. Consultant: Medtronic, Cerenovus. Stock options: Serenity Medical, Synchron, Endostream, VICIS.

Ciprian Ionita—Equipment grant from Canon Medical Systems, support from the Cummings Foundation, NIH R21 grant.

Jason Davies—Research grant: National Center for Advancing Translational Sciences of the National Institutes of Health under award number KL2TR001413 to the University at Buffalo. Speakers’ bureau: Penumbra; Honoraria: Neurotrauma Science, LLC. Shareholder/ownership interests: RIST Neurovascular.

Kenneth Snyder—Consulting and teaching for Canon Medical Systems Corporation, Penumbra Inc., Medtronic, and Jacobs Institute. Co-Founder: Neurovascular Diagnostics, Inc.

Elad Levy—Shareholder/ownership interests: NeXtGen Biologics, RAPID Medical, Claret Medical, Cognition Medical, Imperative Care (formerly the Stroke Project), Rebound Therapeutics, StimMed, Three Rivers Medical. National Principal Investigator/Steering Committees: Medtronic (merged with Covidien Neurovascular) SWIFT Prime and SWIFT Direct Trials. Honoraria: Medtronic (training and lectures). Consultant: Claret Medical, GLG Consulting, Guidepoint Global, Imperative Care, Medtronic, Rebound, StimMed. Advisory Board: Stryker (AIS Clinical Advisory Board), NeXtGen Biologics, MEDX, Cognition Medical, Endostream Medical. Site Principal Investigator: CONFIDENCE study (MicroVention), STRATIS Study—Sub I (Medtronic).

Adnan Siddiqui—Research grant: NIH/NINDS 1R01NS091075 as a co-investigator for “Virtual Intervention of Intracranial Aneurysms.” Financial interest/investor/stock options/ownership: Amnis Therapeutics, Apama Medical, Blink TBI Inc., Buffalo Technology Partners Inc., Cardinal Consultants, Cerebrotech Medical Systems, Inc., Cognition Medical, Endostream Medical Ltd., Imperative Care, International Medical Distribution Partners, Neurovascular Diagnostics Inc., Q’Apel Medical Inc., Rebound Therapeutics Corp., Rist Neurovascular Inc., Serenity Medical Inc., Silk Road Medical, StimMed, Synchron, Three Rivers Medical Inc., Viseon Spine Inc. Consultant/advisory board: Amnis Therapeutics, Boston Scientific, Canon Medical Systems USA Inc., Cerebrotech Medical Systems Inc., Cerenovus, Corindus Inc., Endostream Medical Ltd., Guidepoint Global Consulting, Imperative Care, Integra LifeSciences Corp., Medtronic, MicroVention, Northwest University–DSMB Chair for HEAT Trial, Penumbra, Q’Apel Medical Inc., Rapid Medical, Rebound Therapeutics Corp., Serenity Medical Inc., Silk Road Medical, StimMed, Stryker, Three Rivers Medical, Inc., VasSol, W.L. Gore & Associates. Principal investigator/steering comment of the following trials: Cerenovus LARGE and ARISE II; Medtronic SWIFT PRIME and SWIFT DIRECT; MicroVention FRED & CONFIDENCE; MUSC POSITIVE; and Penumbra 3D Separator, COMPASS, and INVEST.

Ethics approval

Institutional review board approval was obtained, and informed consent was waived for this Health Insurance Portability and Accountability Act–compliant retrospective study (IRB ID: STUDY00001529).

Informed consent

This is a retrospective study; hence, no informed consent is needed.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shiraz Bhurwani, M., Snyder, K.V., Waqas, M. et al. Use of quantitative angiographic methods with a data-driven model to evaluate reperfusion status (mTICI) during thrombectomy. Neuroradiology 63, 1429–1439 (2021). https://doi.org/10.1007/s00234-020-02598-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00234-020-02598-3

Keywords

Search

Navigation