Background

In contrast to traditional randomized clinical trials (RCTs) that are designed to evaluate the efficacy and safety of a drug or intervention under ideal conditions, pragmatic trials aim to compare the effectiveness, benefits, and harms of interventions under real-world conditions [1]. Pragmatic trials embedded within usual clinical care delivery, or “learning health systems,” are designed to maximize the external validity of the knowledge they generate [2,3,4]. By facilitating rapid enrollment of large numbers of patients, pragmatic trials can markedly reduce recruitment challenges and high costs that characterize many traditional RCTs [5,6,7,8,9,10,11,12,13,14,15]. However, the benefits of trials embedded in clinical care must be considered in the context of the loss of patient and physician autonomy that they may impose especially if entire hospitals or clinics are randomized as a cluster to a specific treatment approach. Moreover, such trials may be conducted with waivers of informed consent. Several ethicists have noted that standard informed consent procedures may not be applicable in pragmatic trials comparing interventions that fall within the “standard of care” [16,17,18,19] because such trials do not increase risk to patients beyond the risk of usual care. However, such trials alter the process by which an individual’s care is selected, thereby limiting patients’ and physicians’ decisional autonomy [20].

As promising as pragmatic trials may be for improving the quality and costs of care [2], little empirical work has addressed these ethical concerns [17, 21,22,23]. In a recent qualitative study, we examined how patients and physicians value physician decisional autonomy and traditional informed consent in the context of pragmatic trials [24]. In that study, patients receiving maintenance dialysis generally accepted restrictions on physician decisional autonomy and, when necessary to achieve the goals of the study, endorsed simple notification rather than formal consent. Physicians generally expressed similar views, though some expressed greater reservations about forgoing opt-in consent [24]. Informed by these qualitative results, we designed the current study to quantify how strongly patients and physicians value physicians’ treatment autonomy relative to other desirable study features such as minimization of participant burden and opportunity for informed consent. We conducted a survey-based discrete choice study among patients with end-stage renal disease (ESRD) receiving maintenance hemodialysis and with physicians to determine the relative influence of these study attributes on willingness to participate in a pragmatic dialysis trial.

Methods

Setting and participants

Patients receiving maintenance hemodialysis for ESRD were recruited from 11 outpatient dialysis facilities in the greater Philadelphia region between June 2015 and March 2016. Exclusion criteria included age <18 years, initiation of dialysis within the previous 90 days, inability to speak English, and inadequate visual or cognitive function as determined by each facility’s charge nurse. Potential participants were approached during a dialysis session by a trained research coordinator. After providing informed consent, participants completed the survey using a tablet computer. Patient participants received compensation of US$10 after survey completion.

Physician members of the American Society of Nephrology (ASN) were invited to participate via email between July 2015 and February 2016. Eligible physicians were practicing nephrologists treating adult patients in outpatient dialysis units in the USA. Potentially eligible physicians were randomly selected from the ASN membership list and sent email invitations in two separate waves starting in July and December 2015, respectively, followed by two reminder emails over a 4-week period. Physician participants received a US$50 gift card for completing the study. The study was approved by DaVita Clinical Research (#18-2015) and the Institutional Review Board of the University of Pennsylvania (#819593).

Survey instrument and study design

The survey consisted of a presentation of scenarios, each describing a hypothetical cluster-randomized clinical trial comparing two approaches to blood pressure management for patients receiving maintenance hemodialysis. The basic trial design was held constant across the scenarios but trial attributes were varied. The survey content was informed by our previous qualitative study in which semi-structured interviews were used to understand views of patients and physicians about protocolized treatment approaches for clinical trials [24]. We chose two of the three attributes evaluated in this study based on themes from these interviews: (1) the effect of protocolized care on physician treatment autonomy (treatment autonomy); and (2) information exchange between patients and physicians as it relates to the mechanism of consent (consent approach). Although the third attribute, participant burden, did not emerge as a major theme in the qualitative study, most patients and physicians interviewed did express concern about the burden of participation, making it appropriate for further evaluation. For the treatment autonomy attribute, we varied the physician’s role in selecting a blood pressure target for the patient and choosing which anti-hypertensive medications to use. For the participant burden attribute we varied the testing required for the trial from routine non-invasive blood pressure measurement to monthly blood draws and an additional research visit separate from dialysis treatment. For the consent approach attribute we varied the trial’s enrollment mechanism (opt-in consent, opt-out consent, notification-only, no-notification) (Additional file 1).

A full factorial design using these three attributes produced 16 scenarios (2 autonomy levels × 2 burden levels × 4 consent levels): however, we restricted the number of scenarios to 8 per participant by randomly assigning each to receive one of six groups of 8 scenarios to minimize participant fatigue [25]. In each group, two of the four consent approaches were tested across the two levels of autonomy and two levels of burden. The 6 groups entailed all possible pairwise comparisons of the four consent attributes. We pilot-tested the computerized presentations with 5 patients and 14 physicians to assess acceptability of the format and language, and modified them accordingly. The final survey was written at an 8th-grade literacy level and supplemented with cartoon illustrations to promote readability.

The scenarios for physicians contained more medical terminology than those directed to patients (Additional file 1: Appendix A) but had comparable content. Physicians were informed that oversight of this large, adequately powered hypothetical trial would be provided by Institutional Review Boards and an independent Data and Safety Monitoring Board, and that patients would be provided with the following statement: Patients will not be included in the study if their doctor thinks they should not participate.

The sequence for the eight scenarios in each group was determined by electronic randomization to mitigate potential ordering effects. After reviewing each scenario, participants used a 4-point scale to indicate willingness to have their dialysis facility participate in the clinical trial (definitely participate, probably participate, probably not participate, definitely not participate). An a priori decision was made to dichotomize willingness to participate into either a “Yes” (definitely or probably participate) or “No” (probably not or definitely not participate) for the primary analysis.

Because of limited prior research exposure among this patient population [24], patients were asked to view a 3-minute animated video entitled “Approaches to Research in Medical Practice” from the Research on Medical Practices (ROMP) Ethics Study (rompethics.iths.org) prior to presentation of the clinical trial scenarios. Patients then completed a 5-item knowledge test about randomization and standard-of-care research, with the correct answers revealed afterwards (Additional file 1: Appendix B).

After completion of the scenarios, patients completed the Research Attitude Questionnaire [26], a 7-item measure that assesses patients’ support for and value of biomedical research (theoretical scores range from 7 to 35, with higher scores indicating more favorable views). Patients also completed the 9-item Revised Health Care System Distrust Scale [27] to capture their trust in the values and competence of the healthcare system (theoretical scores range 9 to 45, with higher scores indicating more distrust). Finally, we collected demographic information about patients (age, gender, race, ethnicity, employment status, income, comorbidities, hospitalizations in the past year, and years receiving dialysis therapy) and physicians (age, gender, race, ethnicity, years in practice, practice setting, and past experience with clinical trials).

Statistical analysis

The primary outcome, willingness to have the dialysis facility participate in the trial, was analyzed using hierarchical random-effects logistic regression with an exchangeable working correlation structure and robust error variance estimators to account for the clustering of the responses by each participant [28, 29]. Prior to model building, we examined all continuous variables using locally weighted scatterplot smoothing to determine the appropriateness of entering such variables as linear terms. No variable required transformation. The associations of participant characteristics with willingness to participate were assessed using t tests and Chi-square tests for continuous and categorical variables, respectively. We decided a priori to include all three trial attributes (exposures) in the baseline models for both patients and physicians. Other variables were added to the model if they had a p value <0.20 for the relationship between the variable and the primary outcome in a model that also included the three exposure variables. A priori we decided to also run an adjusted model without including either the healthcare system distrust or research attitude score. These data are not presented here as the results did not differ from the primary model. For physicians, none of the demographic variables met the inclusion criterion, thus the final model includes only the three trial attributes. We additionally explored potential interactions among all three trial attributes by adding two-way interaction terms one at a time to the final models for both patient and physician participants.

We based our sample size estimate on the number of trial attributes tested, the number of levels of each of these attributes, and the desire to detect an odds ratio (OR) of 0.67 for a main effect of an attribute or an OR of 0.5 for an interaction (between any two trial attributes) on the willingness to participate. We targeted 200 patients and 200 physicians for enrollment, yielding > 80% power to declare significance at p = 0.05. This calculation assumed that the “design effect,” [30] reflecting the number of responses per participant and the correlation of responses within participants, would be no greater than 6. The design effect is calculated as:

$$ 1+\uprho \left(\mathrm{k}-1\right), $$

where ρ is the intraclass correlation and k is the number of scenarios per participant [31].

For all analyses, two-sided p values <0.05 were considered statistically significant. Statistical analyses were conducted in Stata (v14.1, StataCorp, College Station, TX, USA) and R Studio (v1.0.136, RStudio Inc., Boston, MA, USA) using the R language (v1.0.136, R Foundation for Statistical Computing, Vienna, Austria) and the R package ggplot2 (v2.2.1, Springer-Verlag, New York, NY, USA).

Results

Patients

A total of 807 patients receiving hemodialysis therapy in 11 dialysis facilities were screened. Of these, 157 (19.4%) did not meet eligibility criteria, and 185 (22.9%) were not approached because they were absent or sleeping during attempts to enroll during two or more dialysis sessions. Among the 465 eligible patients approached, 210 (45.2%) consented to participate and 200 (43.0%) completed the entire survey.

Characteristics of the patient participants are provided in Table 1. Most were 50 years or older, male, and African American, and slightly more than half had been receiving dialysis for 3 or more years. Compared to the overall dialysis population in the geographic region, our population had a similar median age and gender profile, but a different racial distribution reflecting the predominantly urban locations of the enrolling facilities [32]. Overall, 85.0% of patient participants correctly responded to at least four of the five pre-test questions of research knowledge (Additional file 1: Appendix B). The median score of 23 (interquartile range 18.5–27) on the overall healthcare system distrust scale was consistent with scores among other outpatient populations in the same region [33].

Table 1 Patient characteristics
Full size table

Willingness to participate in at least one of the eight hypothetical clinical trials was high (78%). Most patients were willing to have their facility participate in the trial regardless of the level of treatment autonomy for the physician (low 79% vs. high 77%; p = 0.13) or research burden for the patient (low 80% vs. high 76%; p = 0.06). In contrast, the consent approach was influential (Table 2). More patients were willing to participate in trials that had opt-in (84%), opt-out (82%), or notification-only (81%) compared to a no-notification (66%) consent approach (p < 0.001 for each two-way comparison). Figure 1 displays the unadjusted results from the 4-point response scale for willingness to participate by study attribute. Associations between patient characteristics and willingness to participate are shown in Table 3. Increasing patient age was associated with increased willingness (OR 6.05 for > 70 years vs. < 40 years; 95% CI 1.64, 22.31; p = 0.016), and greater distrust in the healthcare system was associated with decreased willingness (OR 0.87; 95% CI 0.83, 0.92; p < 0.001). Only age, research attitude score, and health care system distrust score met the p value criterion for inclusion in the final adjusted model, along with the three trial attributes.

Table 2 Bivariate associations between trial attributes and patients’ willingness to have clinic participate
Full size table
Fig. 1
figure 1

Patients’ and physicians’ willingness to participate in a pragmatic clinical trial by attribute level. Bars represent the proportion of patients (a-c) and physicians (d-f) willing to participate in a hypothetical pragmatic trial based on the level of treatment autonomy (green), research burden (yellow), and consent approach (blue)

Full size image
Table 3 Bivariate associations between patient characteristics and willingness to participate
Full size table

When adjusted for levels of treatment autonomy and research burden, patients were less willing to participate in a clinical trial using a no-notification approach (OR 0.20; 95% CI 0.11, 0.36) compared to an opt-in approach (p < 0.001) (Table 4). Regardless of clinical trial attributes, age was associated with willingness to participate. For example, patients age 70 years or older were more likely to participate than those under 40 years of age (OR 6.05; 95% CI 1.64, 22.31; p = 0.02). Patients with greater distrust in the healthcare system were less likely to participate (OR 0.89 per 1-point increase in distrust; 95% CI 0.83, 0.95; p < 0.001). There was no statistically significant effect between any two attributes on willingness to participate (p values for all interaction terms >0.10).

Table 4 Influence of attributes on willingness to participatea in final adjusted model
Full size table

Physicians

Of the 1635 randomly selected physician members of the ASN who were emailed an invitation to participate, 747 (72%) opened the email and 272 (16.6%) initiated the online survey. Forty-two physicians (15.4%) were ineligible because they did not treat adults in a US outpatient dialysis unit and 27 (9.9%) did not complete the survey. Thus, 203 physicians completed the survey giving an overall response rate of 12.7%. The majority of participating physicians were male, white, 5 or more years beyond fellowship, and self-identified as having experience as a clinical trial investigator (Additional file 1: Appendix C). Demographics of responders were compared with those of a random sample of 300 non-responders. There were no significant differences in either the gender (p = 0.68) or mean age (p = 0.34) of responding and non-responding physicians.

Similar to the patient participants, most physician participants were willing to have their dialysis facility participate in at least one of the eight hypothetical clinical trials (61%). Physicians’ willingness to participate was not affected by either treatment autonomy (61% vs. 61% for low and high, respectively; p = 0.96) or research burden (60% vs. 61% for low and high, respectively; p = 0.79). Physicians were generally less willing than patients to have dialysis facilities participate in the described trials across all scenarios (Fig. 1), and the effect of consent approach on willingness to participate appeared to be stronger for physicians than for patients (Table 4). However, statistical comparisons between patients and physicians were not performed, as these were not specified in our analytic plan. As with patients, interactions among the three trial attributes were not evident among physicians (p > 0.1 for all interaction terms). Additionally, none of the physician characteristics was associated with willingness to participate (Additional file 1: Appendix D).

Discussion

In this study of patients with dialysis-dependent end-stage renal disease and physicians, we did not detect an effect of curtailed treatment autonomy or research burden on willingness to participate in a hypothetical pragmatic trial, but did find an effect of the approach to consent. Both patients and physicians indicated levels of comfort that were similar for opt-out, participant notification, and traditional opt-in consent, but were less willing to participate when participants would receive no notification about the trial. However, even with no notification, two thirds of patients indicated willingness to have their facilities participate.

Patients and physicians are accustomed to interactions in which decisions about treatment approaches are either shared between the patient and the physician or directed by the physician. In traditional RCTs, therapeutic approaches are determined by chance rather than by choice [34], and such restrictions may dissuade patients or physicians from participating in the study. However, in cluster-randomized pragmatic trials patients may not have an option to avoid trials that restrict treatment autonomy because entire hospitals or clinics may be assigned to provide the same treatment approach for all eligible patients. Even when randomization is at the individual patient level, for some pragmatic trials the requirement for informed consent is waived and thus access to alternative treatments is eliminated. Interestingly, for patients and physicians in the current study, limitations on treatment autonomy did not reduce willingness to have their dialysis facility participate in the trial. Moreover, patients were as likely to participate in a trial that relied on opt-out consent or a notification of enrollment as in a trial that used opt-in informed consent, and two thirds of patients were willing to participate even in the absence of any notification. This finding is particularly notable in light of increased distrust in medical care and biomedical research among African Americans [35], who constituted the majority of participants in this ESRD cohort. Indeed, we found that race was not associated with willingness to participate in the hypothetical pragmatic trials.

Strikingly, the hypothetical pragmatic trials presented in our surveys were more acceptable to patients than physicians. This may be explained, at least in part, by the desire of some physicians to rely on their own experience to determine the best treatment for an individual patient [24]. However, notably, one third of physicians expressed willingness to have their dialysis facility participate in a trial conducted without any patient notification. Another interesting finding is that, like patients, physicians’ willingness to participate was not influenced by degree of burden to patients resulting from the trial. The absence of an effect of this attribute might be due to the small incremental burdens for this group of patients of extra testing and blood collection relative to thrice-weekly hemodialysis treatments. It is also possible that effects smaller than we were powered to detect might be evident in a larger sample.

Overall, our findings suggest that a pragmatic clinical trial comparing standard-of-care interventions for blood pressure management is generally acceptable to patients receiving maintenance dialysis and to their treating physicians. Among those we evaluated, the only influential attribute for patients and physicians was the consent approach, a finding that is relevant to ongoing discussions about the use of alternatives to the “traditional” opt-in consent in pragmatic clinical trials [36]. In particular, so long as studies were transparent and included notification to patients, they were generally acceptable to both patients and physicians. Understanding the views of multiple stakeholders is important for informing these discussions. A strength of this study is that we assessed the views of both patents and physicians, and found generally similar results for these two groups.

The findings of our study need to be considered in light of several limitations. The most important is that responses of the patients and physicians to hypothetical RCTs may not reflect the decisions they would make if they were considering participation in actual trials. However, structured vignettes have been shown to be a valid methodology in other clinical settings for producing results that accurately reflect those from more expensive and time-intensive non-hypothetical experimental procedures [37], and hypothetical preferences are the single best predictor of future enrollment decisions [38]. A second limitation is that the valuations of these trial attributes and willingness to participate may not reflect those of patients receiving hemodialysis in other geographic areas or with other diseases, or physicians from other specialties. Likewise, results may vary among trials testing different types of interventions. We chose to focus on hemodialysis patients and physicians because of the potential for dialysis care delivery as a setting for pragmatic trials [39]. The findings from this study should inform future studies in different clinical contexts. An additional limitation is the low survey response rate by physicians that occurred despite pilot testing the study instrument and offering a US$50 financial incentive conditional on completing the survey. Physicians are historically difficult to engage in survey research [40], and we acknowledge the possibility of non-response bias in the physician component of this study. Somewhat reassuringly, the age and sex of our respondents mirrored those of a random sample of physician members of the ASN. Finally, the hypothetical trials used in this study tested only three trial attributes, and it is possible that additional attributes (e.g., risk to patients) may influence participation decisions. However, we informed our attribute selection from interviews with patients and nephrologists [24], and chose to constrain the number of attributes to avoid overburdening participants.

Conclusion

This study provides evidence that limitations placed on treatment autonomy in clinical trials may not be an important contributor to patients’ or physicians’ willingness to participate, and that patients may place similar value on a variety of consent approaches as long as there is some form of notification. These findings contribute important stakeholder perspectives to the ongoing debate about the applicability of traditional research regulations to studies involving standard-of-care interventions [4, 16, 17, 21, 22, 36, 41,42,43]. Future work should aim to elucidate factors driving the expressed attribute valuations and expand this evidence base to other clinical contexts.