Randomized controlled trials (RCTs) are experiments designed to confirm or disprove a given hypothesis. As such they should be considered in the stream of acquiring knowledge, a process which finds its first explicit expression by Aristotle (384–322 b.c.) who codified the syllogism as the ideal method to sort and display the knowledge gained through experience. The syllogism is a way of reasoning that, starting from general premises, derives conclusions applicable to particular cases through a stringent logic process (deduction): “Each animal is mortal, each man is animal, each man is mortal” (Prior Analytics I 1,24b 18 ss). The problem of the truth in the syllogism is not in its logic deductive process but in the premises: who assures that these are universally true? In the sixteenth to seventeenth centuries Copernicus, Keplero, Tycho Brahe, Galileo, and Newton, to different degrees and in sharp contrast with “dogmas”, worked along a similar pathway: the inductive processes (sense-experiences) provide general theory which has to be proved (necessary demonstrations) by experiments and described mathematically. Accordingly, the theory (the premises) is deducted from the experience, and has to be proved experimentally (the RCTs).

In extreme synthesis, it seems that in different ages and through different times, the process of deduction is undiscussed and unquestioned, while what really matters is the origin and validity of the premises: intuition, mind, and God until the Middle Ages, and abstraction from experiments beginning from the seventeenth century. More recently in Vienna, at the beginning of the twentieth century, Popper provided a new standard to establish whether a theory is scientific or not and concluded that it is impossible to prove a given theory or premise. The only possibility is to disprove them. If 100,000 swans are white we cannot conclude with certainty that all swans are white, it is just probable; it is enough for the appearance of one black swan to disprove the theory and a new theory can be advanced, new hypotheses deducted, and new experiments performed through a continuous trial and error process (Popper K, The Logic of Scientific Discovery, 1959).

Therefore our reasoning in acquiring new knowledge in medicine may be summarized according to the following sequence: observation, inductive processes, proposal of theories and premises, deductive hypothesis, and experiments to prove or disprove the hypothesis, which is implicit in the premises. RCTs are one of the possible experiments. Testing an hypothesis deducted from wrong or vague premises leads necessarily to inconclusive results. Hence the fundamental importance of the premises when using RCTs. Let us consider, as examples, some of the major trials performed in intensive care units (ICU), by analyzing their premises, deducted hypotheses, and outcomes (Table 1).

Table 1 RCTs in ICU, examples and considerations
Full size table

As shown, the successful trials are based on robust and well-defined hypotheses, sometimes refined with time. Good examples are the prone position trials and the low tidal volume ventilation. Many other trials, in our opinion, failed either because they were based on wrong premises (the first RCT on extracorporeal support) or were based on premises that were too vague and generic. Possible examples are the PEEP trials and the trials on early goal-directed therapy. Practical problems of the trials such as “population heterogeneity” or time of enrollment often hypothesized as an explanation of failing trials are implicit in the premises. If too generic, the heterogeneity of the population is an unavoidable consequence. Other series of trials, not analyzed in the table, concentrated on anti-inflammatory therapy in severe sepsis and septic shock, either by using corticosteroids or by blocking a large series of inflammatory mediators. These trials, in general, suffered from weak premises and hypotheses: sepsis is a generalized inflammatory reaction and blocking inflammation would improve survival. This approach does not consider a number of co-factors which should be included in the premises to make them credible and effective. In other trials the premises are inconsistent and lacking any physiological or clinical meaning. We found it difficult to identify credible premises in the trials of simvastatine in ARDS or fixed pressure levels in septic shock.

Considering the number of expensive and time-consuming trials performed in intensive care which failed to show differences between treated and control patients, it is tempting to say that in the ICU setting the trial is the wrong tool to advance our knowledge. However, a closer look at the structure and meaning of this methodology clearly suggests that the tool is not wrong per se, but the way we apply it. As the trials became a sort of dogma in ICU research, the trialists spent a lot of effort in improving their internal validity, by increasing the number of rules and caveats to avoid possible biases in the conduct of the trials. This contrasts sharply with the lower attention in clearly identifying and making explicit the premises of the trials, which are the core of the problem.

RCTs are successful, in general, only when the premises are well defined. This obviously makes the results valid only for the restricted patient population enrolled in the study and the generalization of the results (the external validity) is problematic. Using thousands of patients, studies with weak premises are usually inconclusive as the presence of subgroups in which the intervention could be of benefit may be obscured by subgroups in which the same intervention may be harmful. Therefore the effort to conduct the trial in a large population unavoidably carries the high risk of vague premises and useless results.

In conclusion we believe that trials may be an extremely effective method to advance our knowledge when “positive”. This usually occurs when the premises are carefully considered and defined, the hypothesis logical, and the target population (unfortunately “restricted”) adequate to test the hypothesis. In contrast the negative trials do not indicate that a given intervention per se is useless or wrong, but simply disproves the premises. This usually occurs when they are too generic or wrong.