MRI software for diffusion-perfusion mismatch analysis may impact on patients’ selection and clinical outcome

Objective Impact of different MR perfusion software on selection and outcome of patients with acute ischemic stroke (AIS) and large vessel occlusion (LVO) treated by endovascular thrombectomy (EVT) is unclear. We aimed at comparing two commercial MRI software, semi-automated with unadjusted (method A) and adjusted mask (method B), and fully automated (method C) in this setting. Methods MRI from 144 consecutive AIS patients with anterior circulation LVO was retrospectively analysed. All diffusion- and perfusion-weighted images (DWI-PWI) were post-processed with the three methods using standard thresholds. Concordance for core and hypoperfusion volumes was assessed with Lin’s test. Clinical outcome was compared between groups in patients who underwent successful EVT in the early and late time window. Results Mean core volume was higher and mean hypoperfusion volume was lower in method C than in methods A and B. In the early time window, methods A and B found fewer patients with a mismatch ratio ≤ 1.2 than method C (1/67 [1.5%] vs. 12/67 [17.9%], p = 0.0013). In the late time window, methods A and B found fewer patients with a mismatch ratio < 1.8 than method C (3/46 [6.5%] and 2/46 [4.3%] vs. 18/46 [39.1%], p ≤ 0.0002). More patients with functional independence at 3 months would not have been treated using method C versus methods A and B in the early (p = 0.0063) and late (p ≤ 0.011) time window. Conclusions MRI software for DWI-PWI analysis may influence patients’ selection before EVT and clinical outcome. Key Points • Method C detects fewer patients with favourable mismatch profile. • Method C might underselect more patients with functional independence at 3 months. • Software used before thrombectomy may influence patients’ outcome.


Introduction
Endovascular therapy for patients with acute ischemic stroke (AIS) is nowadays supported by class A evidence [1][2][3][4][5][6]. Beyond 6 h from last proof of good health (LPGH), AIS patient with large vessel occlusion (LVO) might still benefit from endovascular therapy when they met either a clinicalcore mismatch or core-hypoperfusion mismatch (i.e. mismatch ratio ≥ 1.8) [7,8], using severely ischemic tissue with uncertain viability (SIT-uv) imaging (called 'core' in this manuscript) as a base [9]. As a consequence, perfusion CT or MR imaging is now the standard of care for patients presenting with acute stroke [10]. Many stroke centres introduced automated perfusion processing software in their clinical practice in order to save time and to reduce inter-observer variability [11]. However, it has been proven that some tools require improvement in order to be able to reliably differentiate between patients with a credible ischemia-related region of hypoperfusion and those without [12]. Current literature on AIS perfusion imaging is mostly based on CT imaging. A comprehensive analysis on CT perfusion imaging [13] has shown that there is marked variability in penumbra and infarct prediction among various deconvolution techniques. Xiong et al [14] compared two automated CT perfusion software (RAPID® and Olea Sphere®) for evaluation of AIS patients, showing that core volumes calculated with RAPID® were larger than with Olea Sphere®. The only report about MR perfusion software actually available, recently published by Deutschmann et al [15], compared RAPID® and Olea Sphere® software. They reported a small systematic difference between software, in the sense that RAPID® outlines slightly larger ADC and smaller hypoperfused tissue volumes. In this context, it seems straightforward to suppose that different perfusion software perform differently in AIS patients' classification. To the best of our knowledge, there is no study that compared RAPID® and Carestream® MR perfusion software and none that evaluated the potential impact on patients' outcome.
In the present study, we aimed at evaluating the concordance between these two software for core and hypoperfusion volumes' estimation in AIS patients with anterior circulation LVO. The second endpoint was to assess the potential impact of volumes' discrepancies on patients' classification prior therapy and on patients' functional independence at 3 months.

Patient cohort
All consecutive patients with acute ischemic stroke and occlusion of ICA or MCA (M1 or M2 segment) who underwent an acute MR imaging between May 2018 and June 2020 at our institution were retrospectively extracted from the Acute STroke Registry and Analysis of Lausanne (ASTRAL) (N = 178). ASTRAL collects all acute ischemic strokes admitted to the stroke unit and intensive care unit of the Lausanne University Hospital presenting within 24 h of stroke onset or LPGH [16].
In this cohort, the inclusion criterion was availability of both DWI and PWI raw data (N = 144/178, 81%); MRI with movement artefacts was also included to compare performance of the two software in 'real life' setting. There were no additional exclusion criteria.
Clinical and epidemiological data, including occlusion site, sex, age, cardiovascular risk factors, aetiology classified according to the Trial of Org 10172 in Acute Stroke Treatment (TOAST) criteria [17], National Institutes of Health Stroke Scale (NIHSS) and treatment type, including intravenous tissue plasminogen activator (IV tPA) and/or EVT, were prospectively collected and extracted from ASTRAL. Eligibility for endovascular treatment followed current guidelines. In the early time window, the presence of acute stroke symptoms and anterior circulation LVO was a sufficient criterion to undergo an EVT. In the late time window, we used liberal clinical/imaging mismatch criteria, based on the NIHSS and DWI-ASPECTS, as previously reported [18]. Treatment decision was thus made independently of any software result in order to avoid potential software-related selection bias. Primary revascularisation following EVT was assessed with the modified Treatment in Cerebral Ischemia (mTICI) score. A successful revascularisation was defined as mTICI superior or equal to 2b. Outcome, expressed as NIHSS at 7 days and modified Rankin score (mRS) at 7 days and at 3 months, and symptomatic intracranial haemorrhage according to the criteria of the European Cooperative Acute Stroke Study (sICHECASS) were also extracted from ASTRAL. A favourable outcome was defined as functional independence (i.e. mRS = 0 to 2) [19]. The ethics committee for research on humans of the Canton of Vaud approved collection, analysis and publication of data from ASTRAL, without requesting patients' consent.

Imaging acquisition and post-processing
All MR examinations were performed on a MAGNETOM VIDA 3T system (Siemens Healthcare) in the Emergency Radiology Unit of our institution, according to the standard protocol used in our department for AIS.
DWI and PWI were processed with two commercial software in three different methods. In all cases, the same thresholds were applied. The core was defined as an apparent diffusion coefficient (ADC) value inferior to 620 × 10 −6 mm 2 /s [21] and the hypoperfusion as a T max superior to 6 s [22,23]. The mismatch ratio was calculated as hypoperfusion/core.

Method A
Carestream® Picture Archiving and Communication System (PACS) (Version 12.2.2.1025, Philips Healthcare Information Solutions) incorporates stroke perfusion software analysis. Once DWI-PWI raw data arrive on the PACS, the radiologist activates the reconstruction using the 'Perfusion Stroke' module and then chooses the affected hemisphere. No further manipulations of the brain mask are applied for this analysis. This version provides perfusion parameter maps (cerebral blood flow, cerebral blood volume, mean transit time and T max ) and ADC map. Volumes of core, and hypoperfusion, are calculated in millilitres using thresholds set as previously described, and saved in a table on the PACS. The mismatch ratio (hypoperfusion/core) is manually calculated.

Method B
Same steps described in method A are followed with additional manual adjustment of the brain mask, which allows suppression of extracerebral mask errors (i.e. suppression of area outside the brain volume) when necessary and is part of standard options of the 'Perfusion Stroke' tool. Core and hypoperfusion volumes are then estimated the same way as described for method A. For the purpose of this study, two neuroradiologists (S.P. favourable mismatch profile) with method B (midline); and 93 mL, 100 mL and 1.1 (i.e. unfavourable mismatch profile) with method C (bottom line). White arrows indicate brain mask error found on method A that was removed on method B. The patient underwent an early EVT and had moderate disability at 3 months (mRS = 3) and V.D. with 10 years' experience in neuroradiology) performed the manual brain mask adjustment of the cohort in consensus reading. This procedure has been performed for all cases and took two additional minutes per case. An exemplary case is shown in Fig. 1.

Method C
RAPID® (iSchemaView) fully automatically analysed raw data received from MR system and sends the results to our PACS, reporting core and hypoperfusion volumes in millilitres as well as mismatch ratio on a single summary image.
For every method, the core and hypoperfusion volumes were recorded. The DWI-PWI mismatch ratio was recorded in a dichotomous way. In the early time window, a mismatch ratio > 1.2 defined a favourable profile, while a mismatch ratio ≤ 1.2 defined an unfavourable profile, according to the FRAME analysis [3,24]. In the late time window, a mismatch ratio ≥ 1.8 defined a favourable profile, while a ratio < 1.8 defined an unfavourable profile, according to DEFUSE 3 study criteria. An exemplary case is shown in Fig. 2.

Statistical analysis
All statistics were performed with the Stata software (version 15.0, StataCorp). Continuous variables are presented as mean ± standard deviation and categorical variables as number or percentage. For further analysis, we divided our study population according to the time window from LPGH to imaging (early ≤ 6 h or late > 6 h). Continuous variables and categorical variables were compared between groups using the Wilcoxon signedrank test or Fisher exact test. Inter-method concordance for core and hypoperfusion volume estimation was assessed by Lin's test with estimation of the Pearson correlation coefficient (rho), concordance correlation coefficient (rho_c), bias correction factor (C_b) and calculation of mean difference with 95% Bland-Altman limits-of-agreement (95% LOA). Agreement for mismatch-based patients' classification was assessed with Gwet's AC1 (95%CI). Correlation coefficient and Gwet's AC1 were interpretated according to the modified Landis and Koch scale as follows: poor when < 0.00, slight between 0.00 and 0.20, fair between 0.21 and 0.40, moderate between 0.41 and 0.60, good between 0.61 and 0.80 and excellent above 0.81. Finally, for each method, clinical outcome (NIHSS at 7 days, mRS at 7 days and 3 months) was compared between classification groups in patients who underwent EVT in the early and late windows. A p value < 0.05 was considered statistically significant. Adjustment of the significance level was done according to the Bonferroni method for every multiple comparisons presented in Tables 4 and 5.

Study population and technical success of perfusion imaging
Demographic data and clinical information for the 144 enrolled patients are provided in Table 1 DWI-PWI analysis was successfully performed with methods A and B in all 144 patients and in 142/144 patients (99%) with method C. In two cases, method C failed to perform the post-processing: in the first case, it did not calculate both core and hypoperfusion volumes, and in the second case, it did not calculate hypoperfusion volume. We excluded these two patients from further analysis.

Inter-method concordance analysis
Estimated absolute mean values of core and hypoperfusion volumes as well as patients' classification based on dichotomised mismatch ratio are shown in Table 2.
Statistical agreement analysis is shown in Table 3. Regarding core volume, methods A and B had an excellent concordance. Compared to method A or B, the core volume was larger using method C despite an overall good concordance between methods. Regarding hypoperfusion volume, methods A and B had a good concordance. Compared to methods A and B, the hypoperfusion volume was lower using method C, with a fair to moderate concordance.

Discussion
Three main results from the present study are as follows: (1) despite using identical core/penumbra thresholds, core volumes are larger and hypoperfusion volumes are smaller when estimated by method C; (2) this results in a higher proportion of patients with unfavourable DWI-PWI mismatch profile using method C, both in the early and late time windows; (3) in patients with functional independence at 3 months after successful late EVT, method C would have underselected more patients than methods A and B. Overall the analysis of outcome of patients that underwent successful EVT suggests that some DWI-PWI software may be too restrictive. MRI is considered the gold standard in the detection of AIS. Nevertheless, unlike CT, MRI is not widely available in acute stroke settings. Accordingly, current literature on AIS perfusion imaging based on MR imaging modality is still scarce.
In this study, we compared two commercially available MR perfusion software using three methods. We found larger core volume and lower hypoperfusion volume using method C compared to methods A and B. This observation was made for AIS patients with anterior circulation LVO admitted either in the early or in late time window who underwent EVT. This is in line with the study by Deutschmann et al [15] who EVT endovascular thrombectomy, MR mismatch ratio, mRS modified Rankin scale, NIHSS National Institutes of Health Stroke Scale, TICI Thrombolysis in Cerebral Infarction. p values in the right column referred to comparison between favourable and unfavourable mismatch profiles. p value < 0.05, corr p value < 0.005. † p < 0.05 for NIHSS at 7 days compared to NIHSS at admission. ‡ p < 0.005 for NIHSS at 7 days compared to NIHSS at admission compared RAPID® and Olea Sphere® software and reported that RAPID® outlines slightly larger ADC and smaller hypoperfused tissue volumes. They considered that small differences of the outlined regions in single slices can sum up to considerable differences in 3D volumes. Our results are also in line with those reported by Xiong et al [14,25]. However, the potential impact on patients' selection and outcome either in the early or in the late time window remained unclear. According to the American Heart Association [26] guidelines, no specific advanced imaging selection criteria are required for EVT in patients who present within 6 h of LPGH and have anterior circulation LVO. Nevertheless, there are proofs that perfusion imaging profiles predict the clinical response to EVT even in this early time window [3,24] when applying the criterion of ischemic core < 70 mL and a DWI-PWI mismatch ratio > 1.2 [3]. In accordance, we found that up to 49.2% of patients having a favourable mismatch profile had functional independence at 3 months after successful EVT. However, our results also indicate that method C is more restrictive for patients' selection, while 19.3% of patients with a good outcome would not have undergone EVT when basing selection only on imaging criterion. As all multicentre studies used method C for patients' selection, our results may suggest that more AIS patients could in fact benefit from EVT in the early time window than previously reported. This may also imply that selection for EVT should not be only based on imaging criterion in the early time window.
In the late time window, EVT is recommended for AIS patients who present within 6 to 16 h of LPGH, have anterior circulation LVO and meet the DEFUSE 3 (infarct core < 70  [28] reported that 30% of patients undergoing off-label EVT according to the DAWN and DEFUSE 3 criteria when assessed with method C achieved functional independence at 3 months. This might overall suggest that more AIS patients could benefit from late EVT than reported in multicentre trials using optimised perfusion software because they were underselected. We have to acknowledge several limitations in our study. First, this is a retrospective study, even if it relies on a prospectively collected and consecutive cohort of patients, which can reduce some selection bias. Second, there is a risk of bias in a monocentric study; its advantage is the uniformity of clinical data collection following an identical protocol at 3T on a single MR scanner. Third, our imaging protocol used high-resolution 3-mm slice thickness diffusion and perfusion imaging, which could limit interpolation-related overestimation due to thicker slices, although the manufacturer of method C does recommend 5-mm slice thickness acquisition. Moreover, comparing methods A and C, all manual interferences on volume segmentation were avoided, both methods being independent from user manipulation-related bias and experience. On the contrary, method B included mask quality control and manual adjustment. However, this needed only minimal expertise and was poorly time-consuming (2 min per case). Furthermore, we found that revocation of patients with a good outcome might occur at a lower rate in the late time window using method B (5.5% versus 16.6% and 38.9% for methods A and C), which suggests that losing 2 min of time could at the end help saving functional independence in some patients. Taking into account recent development of artificial intelligence-based software, use of machine learning or more advanced deep learning technologies in perfusion tools may help in improving volume delineation, reduce inter-reader and inter-software variability and harmonise patients' triage [29]. This was out of the scope of our work and needs to be evaluated in large prospective multicentre studies.

Conclusion
MRI software used for DWI-PWI analysis may influence patients' selection before EVT and impact clinical outcome either at the early and late time window. More restrictive software could deprive some AIS patients of the functional benefit of a successful endovascular thrombectomy. Quality control and manual refining should be considered when using fully automated software not to preclude functional independence of some patients.
Funding Open Access funding provided by Université de Lausanne.

Declarations
Guarantor The scientific guarantor of this publication is Vincent Dunet.

Conflict of interest
The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.
Statistics and biometry One of the authors (Vincent Dunet) has significant statistical expertise. No complex statistical methods were necessary for this paper.
Informed consent Written informed consent was waived by the Institutional Review Board.
Ethical approval Institutional Review Board approval was obtained. The ethics committee for research on humans of the Canton of Vaud approved collection, analysis and publication of data from ASTRAL, without requesting patients' consent.

Methodology
• retrospective • observational • performed at one institution Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.