In the previous issue of Critical Care, Kozek-Langenecker and colleagues [1] report the findings, as described in their title, of a 'systematic review' in which the clinical effectiveness of fresh frozen plasma (FFP) is compared with that of fibrinogen concentrate. In their conclusions, the authors indicate a 'consistent message of benefit of fibrinogen concentrate over both FFP and crystalloids/colloids on a number of outcome measures, including reduction of blood loss and allogeneic transfusions'. The key question for a reader is whether the results in this review provide confidence in these statements.

We argue for extreme caution in the interpretation of the review. The main blood component as an alternative specific source of (concentrated) fibrinogen is cryoprecipitate, not FFP. It is likely that, in many studies, the FFP was given for other reasons such as to raise levels of procoagulant factors in non-bleeding patients. The criteria for inclusion of studies are very broad: the studies included were defined as taking place in the field of perioperative and 'massive' trauma, the latter of which was undefined. Many of the perioperative studies included cardiovascular surgery, and others were studies in infants. Is it appropriate to pull together studies com-paring different formulations of plasma (for example, pathogen inactivation) with studies of fibrinogen concentrate in cystectomy? Crucially, the studies cover a whole range of designs, from randomized controlled trials (RCTs) to retrospective non-comparator studies and case reports. Many studies were uncontrolled, making comparative assessments impossible.

One key component of a systematic review process is methodological quality, which forms part of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [2] used to assess the under-taking and reporting of systematic reviews, but it is unclear how this has been undertaken in this review. Comparable standards - the Consolidated Standards for Reporting Trials (CONSORT) [3] and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) [4] statements - can be applied for randomized trials and observational studies. The vast majority of the eligible studies in this review would be expected to be at risk of bias, other issues (such as those of publication bias) notwithstanding.

To support the aims and conclusions in this review, a reader might initially expect that much of the data will be drawn from clinical studies directly comparing these two interventions. But, in fact, out of the 91 eligible studies, only three compared fibrinogen concentrate against FFP, and, in this case, plasma in combination with other products. As the authors indicate, RCTs do provide a more robust assessment, but no detail is presented in the quality assessment of the 15 RCTs of plasma or, crucially, the three RCTs of fibrinogen concentrate. The three trials of fibrinogen concentrate enroll 17, 10, and 10 patients into the intervention arm, and this must raise questions about confidence intervals for the trial outcomes. The numbers of small studies not only may exaggerate the amount of evidence available but also, in the setting of RCTs, could compromise the ability of randomization to achieve equivalence of baseline characteristics. Studies comparing fibrinogen with FFP must also presume that evidence of effectiveness is known for one of the interventions, as the standard, but this is far from clear for FFP.

There are a number of other issues to consider. Fibrinogen is not licensed to be given in acquired fibrinogen deficiency in most countries. Are clinical trials being planned or undertaken to support the case for licensing? There is no mention of cost-effectiveness or of the true risk-benefit of giving potentially prothrombotic concentrates to patients (post-bleeding patients are at higher risk of hospital-acquired venous thromboembolism, and this is a major safety issue that has never been addressed).

Our view is that these data show that FFP and fibrinogen have never been adequately assessed in the management of bleeding, and we would suggest that, until robust trial data emerge, the use of fibrinogen concentrate be confined to clinical trials. We are fearful that the danger of this review is that the message supports a move toward greater use of fibrinogen concentrate without proper evaluation first.

The research into managing bleeding is littered with failure. Ian Chalmers, the guru of clinical trials, uses aprotinin as a scandalous example of ethical and clinical trial failure [5]. Eighty RCTs were performed to show that aprotinin reduced bleeding, but how was it ethical to randomly assign patients in the later trials when this antifibrinolytic had already been shown to work? And yet the even bigger failing was using the wrong endpoint - reducing bleeding or transfusion requirements - rather than reduction of premature death. After many years, the BART trial (Blood Conservation Using Antifibrinolytics in a Randomized Trial) addressed these concerns and showed that aprotinin, despite reducing bleeding, led to a higher mortality than other antifibrinolytics [6]. The future for studying blood components lies in pragmatic RCTs, with endpoints of mortality, which assess the rate of hospital-acquired thrombosis as well as reduction in bleeding. It has been said in the past that it is too difficult to do such studies in sick, bleeding patients, but CRASH-2 (Clinical Randomisation of an Antifibrinolytic in Significant Haemorrhage), which randomly assigned 20,000 patients, shows that this is a fallacy [7].

Why are we, as a community of health professionals working to improve patient outcome in those with major bleeding, getting it so wrong and failing to undertake high-quality studies? This appears to be a common theme in the acute management of trauma hemorrhage, even though a recent systematic review identified over 30 RCTs [8]. Is there a general lack of understanding, among health professionals in this field, of how to weigh up and respond to deficiencies in the scientific evidence? It would be ethically unacceptable and a moral tragedy for the greater use of different sources of fibrinogen, what-ever the cause, to slip into routine clinical practice with-out an adequate evidence base.