One of the milestones of evidence-based medicine is the critical appraisal [1]. Critical appraisal shifts clinical practice from eminence-based medicine (medicine based on authority) to evidence-based medicine (medicine based on the proof and the judgment of the proof). Clinical practice guideline recommendations should be based on the best available evidence. Therapeutic recommendations should be developed upon randomized controlled trials (RCTs) with clinically relevant (defined as “hard”) primary end points, low risk of bias, rigorous internal validity and good external validity [2]. The Italian guidelines on thrombolysis indications in ischemic stroke [3] have been revised after the IST-3 trial [4]. Does IST-3 satisfy the above-mentioned RCT characteristics providing the best evidence on which a clinical practice guideline should be modified?

When the IST-3 trial was published many considered it the end of the struggle between “pros” and “cons” about thrombolysis [5–7]. Indeed IST-3 is the largest trial on this topic. The Lancet editorial accompanying the IST-3 trial claims “The role of stroke and emergency physicians is now not to identify patients who will be given rt-PA, but to identify the few who will not” [5]. An authoritative IAEM editorial [8] commenting on the IST-3 trial concludes “We have no longer to ask the age, but just the onset time!” However, some weaknesses should be considered [9] and we suggest that there are still some reasons to introduce some notes of caution. If a large trial has some biases, it should not be considered as the “best” evidence and used to change the recommendations of clinical practice guidelines.

The aim of this paper is not a comprehensive critical appraisal of the IST-3 trial, already published [10]. We would like just to highlight some major criticisms indicating that maybe the IST-3 trial results are not adequate to change clinical practice.

  • First of all, the primary end point of the trial is not statistically significant: changing guideline recommendations based on a non-significant result is quite unique.

  • Results on which guideline recommendations have been changed came from secondary and subgroup analyses. In this regard, a sentence in the manuscript of the IST-3 trial draws our attention: the analysis of the results suggests “greater benefit in those older than 80 years of age; contrary to expectations”. Subgroups analyses are not considered methodologically adequate to derive conclusions [2]. The more subgroups are present the greater the probability that a false positive result emerges [11]. The GISSI study provocatively carried out some subgroup analyses on the benefit of aspirin in myocardial infarction according to the zodiac sign, so that someone may conclude that aspirin could be useful just in some of the Zodiac signs [12]. If a subgroup analysis leads to an unexpected result (from an epidemiological or pathophysiological point of view) it should be even more cautiously evaluated [2].

  • The duration of the trial (12 years) is likely to create some biases, since changes in practice standards regarding thrombolytics in stroke and stroke management have occurred during this period. All these sources of variation could have altered the enrollment procedure and the type of patient enrolled thus undermining the internal and external validity of the trial.

  • The method used to evaluate disability changed during the trial and this is quite unusual, even though the decision was made before knowing the data. Authors decided to apply an ordinal shift analysis to evaluate how much thrombolytic treatment could reduce disability, but the results, expressed as odds, are barely applicable in clinical practice [10].

  • A Cochrane systematic review was published by Lancet [13] together with IST-3. Although a systematic review should be the best available evidence, in this case the results are inconclusive about the appropriate time and age for stroke thrombolysis because the data are prominently influenced by the IST-3 trial that has a remarkable weight in the meta-analysis.

Patients evaluated in everyday clinical practice often cannot be included in most of clinical trials, due to some factors such as age, comorbidities or social conditions [14]. Sometimes these discrepancies threaten the generalizability of RCTs findings. The pragmatic IST-3 trial nature leads to a very good generalizability thanks to the inclusion of many patients evaluated in everyday clinical practice, but the main problem of the trial is in its internal validity.

So what about patients over 80 years who can receive stroke thrombolysis? The IST-3 trial recruited many patients over 80 years showing that in these patients the outcomes are not worse than in younger patients. This result combined with stroke pathophysiology suggests that age cannot be a criterion strictly applicable for thrombolysis inclusion. The same is probably true for onset hour as well.

In conclusion, we think that physicians should make clinical decisions balancing the benefits and harms of each individual patient, considering that the greater the age of the patient, and the more time that has elapsed from the stroke onset to thrombolysis, the higher are the risks. Last, among the thrombolysis debate, it seems that patient preferences and expectations are currently neglected. In the complex evaluation of the benefits and harms, there is a need to take into account the patient preferences between a short-term high risk of death and hemorrhage and a possible long-term increase in functional independence. This is a difficult ethical question implying an even more difficult ethical answer especially in the emergency care setting.