Abstract
Most phase III clinical trials today are explanatory. Because explanatory, or efficacy, trials test hypotheses under “ideal” conditions, they are not well suited to providing guidance on decisions made in most clinical care contexts. Pragmatic trials, which test hypotheses under “usual” conditions, are often better suited to this task. Yet, pragmatic, or effectiveness, trials are infrequently carried out. This mismatch between the design of clinical trials and the needs of health care professionals is frustrating for everyone involved, and explains some of the challenges inherent in attempts to enhance knowledge translation and encourage evidence-based practice. The situation is more than simply frustrating, however; it is potentially unethical. Clinical trials must be socially valuable in order to (1) warrant the risks they impose on human research subjects and (2) fairly and efficiently assess new clinical interventions. Most bioethicists would agree that trials that have no social value, for instance, because their results do not have the potential to advance clinical care, should not be performed. What is less widely appreciated is that given limited research resources, trials that are more socially valuable should be preferred to trials that are less socially valuable when all else is equal. With respect to clinical trial design, I argue that while explanatory trials often have some social value, many have less social value than their pragmatic counterparts. On the basis of this general ethical assessment, I provide a preliminary defense of the position that clinical researchers should aim to conduct pragmatic trials, that is, that researchers face a burden of justification related to any idealizing elements added to trial designs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
I do not assess the relative social value of different methodological choices in early phase (I or II) or late phase (IV) trials in this paper. While I would argue that the social value requirement applies “all the way down,” much work is still needed to connect methodological choices to goals at different stages of clinical research.
Classic problems about how to define “the public” and how responsibilities transfer to the international context are relevant here but are beyond the scope of this paper.
On the basis of the description provided by Schwartz and Lellouch, there appears to be a third option: a partly historically controlled trial that builds in the 30-day “blank” period before the official start of the trial. One can, however, charitably adjust one’s reading of the hypothetical case to rule out this third option, for instance, by specifying that the “blank” period has to consist of a thorough washout in which subjects are not given the other medications they would typically be given for their condition.
As I clarify later in the paper, the trials are better described as more-explanatory and more-pragmatic. Schwartz and Lellouch identify only one factor on which trials can be assessed (comparison treatment).
I will proceed with the terminology chosen by Schwartz and Lellouch, however imperfect it may be, since, surely, explanatory trials may produce results relevant to practice and pragmatic trials may add to our explanations.
In this paper, then, references to pragmatic trials would more precisely be written as “more-pragmatic trials” and likewise for explanatory trials, though I will stick to the simpler description for ease of discussion.
Context is defined by the researchers in this debate as “the circumstances where and when an intervention is implemented that modify (or might modify) the effectiveness of the intervention” [5]. The same trial design in different contexts may be more or less explanatory or pragmatic.
Though it would amount to an abuse of the purpose of this distinction, this means that every trial could, with some effort, be reclassified as pragmatic, because any particular population of research subjects is a more-or-less accurate reflection of that particular patient population. Testing on a homogenous population of healthy, compliant, young men in a high-tech urban setting will reflect the effectiveness of the particular treatment in that context. Many different populations and types of care are “usual” for many different contexts. However, recommendations for researchers to defend the purpose of their trial and the contexts in which their trial would count as pragmatic would still provide important information to the end users of clinical trial evidence. Attempts to design more idealized studies (in order to increase the likelihood of a positive result), while labeling them as pragmatic, could at least be more easily identified for what they really are (cf. [7] as cited in [8]). Karincolas and colleagues point out that abuse is also possible by designers of pragmatic trials, who might include a subset of highly compliant, highly responsive subjects within a larger pragmatic trial in the hopes of influencing the results [6].
The only review of the literature performed on this subject indicates that there have been fewer than 100 pragmatic randomized controlled trials (RCTs) among the approximately quarter of a million RCTs listed by the US National Library of Medicine ([7] as cited in [8]). This review is imperfect for a variety of reasons. I offer further support for the claim that most RCTs are explanatory below.
In what follows I assume that where there is a widespread practical problem (in this case, nearly universal preference for explanatory trials), there probably is, if not a deep theoretical problem, at least significant theoretical confusion. In this case, where bioethicists have a broad commitment to social value, the confusion seems to be over what exactly the requirement of social value entails, whether it is a threshold concept, how trade-offs with scientific validity are to be negotiated, whether methodological choices are fair game for ethical scrutiny, and so on.
In a fascinating twist in the debate, recognition of this variation is now used in some cases to justify three-armed trials (where there are two controls: one placebo, one active).
Emanuel et al. indicate that the first requirement of ethical medical research is “social or scientific value,” though they also refer to “social, scientific, or clinical value” and “asking socially valuable questions.” Since social value, scientific value, and clinical value seem to be pulling in different directions in the debate over explanatory and pragmatic trials, it would be useful to clarify what this first requirement should be aiming to protect. I take it that it is aiming to protect social value, where that extends beyond benefits to the trial participants and beyond considerations of scientific validity (which is the second related, but not identical, requirement on the list). Further conceptual work on this point is desperately needed, though beyond the scope of this paper.
Emanuel et al. state that the order of the set of requirements reflects the order in which researchers encounter them in the process of designing and executing a research project. It is in this sense that social value is the first consideration. The ordering does, if indirectly, indicate a sense of priority or importance, since the earliest decisions in the design of research trials tend to shape much of what follows. (This is one of the points made earlier by Schwartz and Lellouch).
The social value requirement is thought by some to be paternalistic (if justifiably so). A widespread fear of paternalism may contribute to a general resistance to taking the social value requirement seriously. For a helpful discussion of these issues, see [16].
Because context matters to the assessment of the ethics of trials, these general comments are not meant to be decisive declarations about the ethics of any particular trial.
I am using ‘direct’ and ‘indirect’ loosely here. There is no clear direct/indirect distinction but rather, again, a spectrum from the more directly applicable pragmatic trials to the less directly applicable explanatory trials.
Emanuel et al. offer a more nuanced position, which will be considered later in this paper.
There is a certain irony in suggesting that explanatory trials are justified because they alone answer questions about biological mechanisms and advance scientific understanding. Clinical epidemiology—a relatively new discipline that emerged in the 1960s and 1970s—has always emphasized the importance of RCTs precisely because they ask the question “does this really work?” which can then inform practice. But explanatory RCTs aim to be “rigorous” and in so doing move further away from asking this practical question. Further, they fail to ask another important question, namely: “why does this work?” (the mechanism question, which is what was traditionally asked by bench researchers). They frequently fail, then, to be helpful in either of the ways that bench research or pragmatic trials are helpful. What often results is a sort of pseudo-rigor: the appearance of a law-like generalization, with very little actual generalizability.
The relationship to validity becomes quite complicated, as Freedman acknowledges: “Yet all of these useless studies may be valid. Indeed a useless study is more likely to be valid than a useful study for the reasons noted above.” After all, “a trial’s validity may always be sustained by modifying its hypothesis appropriately. Even the sloppiest study’s validity may be sustained, but at the cost of generality and interest of the hypothesis under study” [15, p. 9].
This raises questions about the extent to which we, as a society, believe human subjects should be experimented on purely for greater understanding. The differences between human subjects research and other types of research are important here. Even if the boundaries of protection are extended, as some critics would suggest, to cover certain other non-human animals, it is probably safe to assume they will not extend all the way to bacteria and one-celled organisms. Basic, or bench, science research is then not held to the same standard of immediate practical applicability of research. The particular concerns raised here relate to the use of human subjects, where we believe different justifications for research need to be supplied.
In spite of Freedman’s seeming support for the strong position offered above, he does complicate his account later in the same paper, where he acknowledges the importance of a debate over the value of research likely to lead to bad consequences in the future (using germ-cell genetic engineering as an example) by noting that “the distinction between the immediate and the long-range costs and benefits of a study is somewhat arbitrary.” His emphasis throughout the paper is on the context-relativity of ethics appraisal, but this statement does suggest that the judgment call involved in assessing social value requires attention to difficult questions about the relative value of research with either immediate or long-term effects on practice.
An important question for the empirically minded is whether pragmatic trials actually do provide better evidence to clinicians, that is, evidence that has a beneficial impact on patient outcomes in clinical practice. As far as I have been able to determine, we are still waiting on the results of research investigating the comparative practical impact of the results of pragmatic and explanatory trials. I believe there is at least one study on this question underway at Sunnybrook Health Sciences Centre in Toronto, headed by Dr. Merrick Zwarenstein (one of the authors in the debate mentioned earlier). It is important, though, not to take this as a reason to avoid pragmatic trials, since the challenge of producing evidence for explanatory trials has also been a longstanding concern within evidence-based medicine. Explanatory and pragmatic trials are on equal footing when it comes to empirical evidence of their impact on clinical practice. Since uncertainty is a familiar starting point, it need not compromise efforts to improve practice in ways guided by reason. Zwarenstein et al. have offered modifications to the widely used CONSORT statement in order to improve the identification and assessment of pragmatic RCTs. See [24].
I do not want to downplay the importance of the roles played by these other parties: each has responsibilities here. Institutional regulations often shape what is possible in research, and these systemic policies will need to change. But I believe it is too frequently the case that researchers themselves are neglected in ethical analyses. I think there is a tremendous ethical burden on all clinical researchers, and that this carries through decisions made at every stage in trial design. This ethical-epistemic work needs to be made transparent and be more widely recognized.
References
Schwartz, D., and J. Lellouch. 2009. Explanatory and pragmatic attitudes in therapeutic trials. Journal of Clinical Epidemiology 62: 499–505.
Thorpe, K.E., M. Zwarenstein, A. Oxman, et al. 2009. A pragmatic-explanatory continuum indicator summary (PRECIS): A tool to help designers. Journal of Clinical Epidemiology 62(2009): 464–475.
Riddle, D.L., R.E. Johnson, M.P. Jensen, et al. 2010. The Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) instrument was useful for refining a randomized trial design: Experiences from an investigative team. Journal of Clinical Epidemiology 63: 1271–1275.
Tosh, G., K. Soares-Weiser, and C.A. Adams. 2011. Pragmatic vs. explanatory trials: The pragmascope tool to help measure differences in protocols of mental health randomized controlled trials. Dialogues in Clinical Neuroscience 13: 209–215.
Oxman, A.D., C. Lombard, S. Treweek, et al. 2009. A pragmatic resolution. Journal of Clinical Epidemiology 62: 495–498.
Karincolas, P.J., V.M. Montori, P.J. Devereaux, H. Schünemann, and G.H. Guyatt. 2009. The practicalists’ response. Journal of Clinical Epidemiology 62: 489–494.
Vallve, C. 2003. A critical review of the pragmatic clinical trial [in Spanish]. Medicina clínica 27: 384–388.
Zwarenstein, M., and S. Treweek. 2009. What kind of randomized trials do we need? Journal of Clinical Epidemiology 62: 461–463.
Van Spall, J.G.C., A. Toren, A. Kiss, and R.A. Fowler. 2007. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: A systematic sampling review. Journal of the American Medical Association 297: 1233–1240.
Masoudi, F.A., E.P. Havranek, P. Wolfe, et al. 2003. Most hospitalized older persons do not meet the enrollment criteria for clinical trials in heart failure. American Heart Journal 146: 250–257.
Anderson, J. 2006. The ethics and science of placebo-controlled trials: Assay sensitivity and the Duhem–Quine thesis. Journal of Medicine and Philosophy 31: 65–81.
Anderson, J. 2009. Who’s in control of the choice of control? American Journal of Bioethics 9(9): 60–62.
Howick, J. 2009. Questioning the methodologic superiority of ‘placebo’ over ‘active’ controlled trials. American Journal of Bioethics 9(9): 34–48.
Emanuel, E.J., D. Wendler, and C. Grady. 2000. What makes clinical research ethical? Journal of the American Medical Association 283: 2701–2711.
Freedman, B. 1987. Scientific value and validity as ethical requirements for research: A proposed explication. IRB: Ethics and Human Research 9(6): 7–10.
Miller, F.G., and A. Wertheimer. 2007. Facing up to paternalism in research ethics. Hastings Center Report 37(3): 24–34.
Anderson, E. 1995. Knowledge, human interests, and objectivity in feminist epistemology. Philosophical Topics 23(2): 27–58.
Evans, E. 2010. In defense of valid design as a policy rule. American Journal of Bioethics 10(6): 18–19.
Dresser, R. 1992. Wanted: Single, white male for medical research. Hastings Center Report 22(1): 24–29.
Rothwell, P.M. 2006. Factors that can affect the external validity of randomised controlled trials. PLoS Clinical Trials 1(1): e9.
Rothwell, P.M. 2005. External validity of randomized controlled trials: “To whom do the results of this trial apply?”. Lancet 365: 82–93.
Hotopf, M. 2002. The pragmatic randomised controlled trial. Advances in Psychiatric Treatment 8: 326–333.
Holden, C. 2008. Women abound in NIH trials. Science 322(5899): 219.
Zwarenstein, M., S. Treweek, J.J. Gagnier, et al. 2008. Improving the reporting of pragmatic trials: An extension of the CONSORT statement. BMJ 337: a2390.
Acknowledgments
My research is supported by a Catalyst Grant from the Canadian Institutes of Health Research (CIHR). Special thanks to Ross Upshur for alerting me to the special issue of the Journal of Clinical Epidemiology on pragmatic trials, two anonymous reviewers for helpful comments on an earlier version of the paper, and colleagues at Dalhousie University and the New Scholars in Bioethics for ongoing scholarly support.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Borgerson, K. Are explanatory trials ethical? Shifting the burden of justification in clinical trial design. Theor Med Bioeth 34, 293–308 (2013). https://doi.org/10.1007/s11017-013-9262-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11017-013-9262-4