Avoid common mistakes on your manuscript.
We read with interest the multicenter manuscript of Psychogios et al. in which they reported on the comparison of infarct core and tissue at risk maps generated by four different vendors as well as visual Cerebral Blood Volume—Alberta Stroke Program Early CT Score (CBV-ASPECTS) and visually assessed collateral scores [1]. They related the maps of 182 patients undergoing mechanical thrombectomy (MT) and receiving a TICI 2b, 2c or III reperfusion to the clinical outcome assessed with the modified Rankin score (mRS) and the functional disability defined as mRS > 2. They calculated mean differences between RAPID (iSchemaView Inc, Menlo Parc, CA, USA) and other software packages and illustrated them with Bland-Altman plots. They concluded that the infarct core defined by the RAPID software correlates best with the clinical outcome whilst VEOcore (VEObrain GmbH, Freiburg, Germany) and syngo.via (Siemens Healthineers AG, Erlangen, Germany) overestimate the infarct core and Olea (OLEA medical Inc., La Ciotat, France) underestimates it [1].
The message is clear but can we trust it? In the manuscript the authors clearly state that out of 215 cases 33 cases have been excluded from the final analysis due to “… technical failure of at least 1 perfusion software”; however, if we take a look at the Bland-Altman plot of RAPID-VEOcore (only available in the Supplemental Material) there is a striking outlier in the infarct core volume difference of around −2131 mL (which is distinctly larger than an entire brain). This outlier leads to a massive bias in the statistics: it can be estimated that without the outlier the true mean difference between RAPID and VEOcore is in a very good agreement range of −1.5 mL instead of the −13.4 mL reported in the manuscript.
Thus, a single nonplausible outlier caused a significant difference and led to an erroneous conclusion that the VEOcore software is inappropriate for treatment decisions beyond the 6h window [1].
Besides this inconsistency in the manuscript—which should be addressed in an erratum—we would further like to point out another general limitation of the work, which in our view leads to an increased distraction from the true value of CTP in acute stroke management in our community:
Postprocessing of CTP has two major goals: 1) to display the hypoperfused tissue and 2) to estimate how much of this tissue is already infarcted (infarct core). The only way to prove whether the infarct core is correctly identified on CTP is to compare it with a posttreatment MRI in recanalized patients and to calculate, for example, dice indices. To relate volume measurements to the clinical outcome is a too simplistic approach. We do not deny the weaknesses of CTP, however, the most accepted infarct core definition is tissue with a relative CBF < 30% compared to the contralateral hemisphere [2]. This threshold derived from the data of 103 patients in which DWI-MRI was acquired shortly after CTP [3] is implemented in the RAPID and VEOcore, but not in the original syngo.via software, for example [4]. In the meantime, apparently, the syngo.via software has also been adopted the < 30% rCBF value [5, 6]. Methodologically, issues such as different vulnerability of gray and white matter, order and timing of CTA and CTP, bolus interference of CTP and CTA, differences in CBF in gray and white matter (especially in patients with small vessel disease) must be considered when interpreting infarct core maps [7, 8]. CTP as well as the temporal thresholds applied in DAWN and DEFUSE 3 have empirically been chosen and are now required to indicate mechanical thrombectomy beyond the 6h window. Nevertheless, we use CTP in more situations. In medium and distal vessel occlusions it displays the hypoperfused tissue better than CTA (Fig. 1) and especially outside working hours it allows a rapid communication within a stroke network via mobile devices.
In conclusion, we appreciate a critical debate on the use of CTP using different postprocessing tools but such comparisons should be performed thoroughly and based on valid data.
References
Psychogios MN, Sporns PB, Ospel J, Katsanos AH, Kabiri R, Flottmann FA, Menon BK, Horn M, Liebeskind DS, Honda T, Ribo M, Ruiz MR, Kabbasch C, Lichtenstein T, Maurer CJ, Berlis A, Hellstern V, Henkes H, Möhlenbruch MA, Seker F, Ernst MS, Liman J, Tsivgoulis G, Brehm A. Automated Perfusion Calculations vs. Visual Scoring of Collaterals and CBV-ASPECTS : Has the Machine Surpassed the Eye? Clin Neuroradiol. 2020. https://doi.org/10.1007/s00062-020-00974-3.
Rava RA, Snyder KV, Mokin M, Waqas M, Zhang X, Podgorsak AR, Allman AB, Senko J, Shiraz Bhurwani MM, Hoi Y, Davies JM, Levy EI, Siddiqui AH, Ionita CN. Assessment of computed tomography perfusion software in predicting spatial location and volume of infarct in acute ischemic stroke patients: a comparison of Sphere, Vitrea, and RAPID. J Neurointerv Surg. 2021;13:130–5.
Cereda CW, Christensen S, Campbell BCV, Mishra NK, Mlynash M, Levi C, Straka M, Wintermark M, Bammer R, Albers GW, Parsons MW, Lansberg MG. A benchmarking tool to evaluate computer tomography perfusion infarct core predictions against a DWI standard. J Cereb Blood Flow Metab. 2016;36:1780–9.
Kellner E, Reisert M, Kiselev VG, Maurer CJ, Beume LA, Urbach H, Egger K. Automated Infarct Core Volumetry Within the Hypoperfused Tissue: Technical Implementation and Evaluation. J Comput Assist Tomogr. 2017;41:515–20.
Bathla G, Limaye K, Policeni B, Klotz E, Juergens M, Derdeyn C. Achieving comparable perfusion results across vendors. The next step in standardizing stroke care: a technical report. J Neurointerv Surg. 2019;11:1257–60.
Bathla G, Ortega-Gutierrez S, Klotz E, Juergens M, Zevallos CB, Ansari S, Ward CE, Policeni B, Samaniego E, Derdeyn C. Comparing the outcomes of two independent computed tomography perfusion softwares and their impact on therapeutic decisions in acute ischemic stroke. J Neurointerv Surg. 2020;12:1028–32.
Goyal M, Ospel JM, Menon B, Almekhlafi M, Jayaraman M, Fiehler J, Psychogios M, Chapot R, van der Lugt A, Liu J, Yang P, Agid R, Hacke W, Walker M, Fischer U, Asdaghi N, McTaggart R, Srivastava P, Nogueira RG, Moret J, Saver JL, Hill MD, Dippel D, Fisher M. Challenging the Ischemic Core Concept in Acute Ischemic Stroke Imaging. Stroke. 2020;51:3147–55. Erratum in: Stroke. 2021;52:e49.
Rudilosso S, Laredo C, Vivancos C, Urra X, Llull L, Renú A, Obach V, Zhao Y, Moreno JL, Lopez-Rueda A, Amaro S, Chamorro Á. Leukoaraiosis May Confound the Interpretation of CT Perfusion in Patients Treated with Mechanical Thrombectomy for Acute Ischemic Stroke. AJNR Am J Neuroradiol. 2019;40:1323–9.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
E. Kellner and H. Urbach are some of the scientific inventors of the VEOcore software. E. Kellner is shareholder of and receives payments from VEObrain GmbH, Freiburg, Germany.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kellner, E., Urbach, H. Machine Outputs Must Be Checked. Clin Neuroradiol 31, 507–508 (2021). https://doi.org/10.1007/s00062-021-01012-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00062-021-01012-6