Introduction

Over the past decade, patients have collaborated with researchers, funders, academia, and sponsors to inform research priorities, funding decisions, health services research, the selection of outcomes, clinical trial protocol designs, and recruitment and retention [1, 2]. Because patient group engagement has significant potential to improve the clinical trial enterprise, the Clinical Trials Transformation Initiative (CTTI)—a public–private partnership co-founded by Duke University and the US Food and Drug Administration (FDA) whose members include representatives from across the clinical trials ecosystem—developed the Patient Groups and Clinical Trials project in 2013 to foster this collaboration. Initially the project sought to better understand stakeholders’ perceptions regarding the importance and value of engaging patient groups and the various clinical trial services that patient groups provide. This led to the development of recommendations, best practices, and a list of specific activities for engaging patient groups throughout the clinical trial process [3]. To further advance mutually beneficial patient group engagement, the project developed a financial model to better articulate the impact that patient engagement may have on key business drivers and to demonstrate that return on investment should support broader adoption [4].

This work, along with emerging best practice resources on patient engagement in clinical trials, such as those from the Patient-Focused Medicines Development (PFMD) [5] and FasterCures [6], is helping to accelerate patient group engagement. While there is still much to understand, engaging patient groups in clinical trials is also gaining broader acceptance: for example, the Patient-Centered Outcomes Research Institute (PCORI) requires patient engagement in any of their funded clinical trials [7]; the National Academy of Medicine (NAM) recommends including patients as partners in research [8], and the US FDA has acknowledged the importance of patient involvement through a range of initiatives and guidance documents [9,10,11,12].

However, despite resources to help stakeholders understand the breadth of potential patient engagement activities and promising practices [13], there is no widely used framework or method to facilitate identifying fit-for-purpose activities that are mutually beneficial for the sponsor and for the patient group or the patient community they represent. To support this need, CTTI has developed a framework and a prioritization tool to aid both sponsors and patient groups in determining, from their perspective, (1) the benefit that patient group engagement can bring to their organizations and the clinical trial process, (2) the investment that such engagement would require, and (3) those engagements that are of highest priority to each organization. The tool supports users—both patient groups and sponsors—in identifying relevant engagement opportunities for a specific study, subjectively assessing the benefits and investments of each (low, moderate, high), and visualizing and discussing the output together as partners.

This manuscript describes the process CTTI used to gather evidence to develop the tool, describes the tool itself, and describes how the tool can be used by sponsors and patient groups to guide decisions on priority patient group engagement activities.

Methods

Evidence Gathering

Working from the CTTI Patient Group Organizational Expertise and Assets evaluation tool, we developed a list of 31 patient group engagement activities in medical product development [14]. We conducted 28 qualitative, semi-structured interviews (SSIs) with representatives involved in engaging patients in medical product development from research sponsor organizations (n = 14) and patient groups (n = 14), from January 26, 2017, to April 18, 2017 (Table 1). Representatives were purposively selected [15] based on their knowledge of the types of patient group engagement activities their organization has participated in and whether their organization is actively engaged in medical product development. In addition, we purposively recruited representatives from organizations of varying sizes (e.g., based on annual budget) and organizations involved in medical product development across the clinical trial continuum (i.e., pre-discovery through post-approval). Representatives were asked to review each of the 31 CTTI patient group engagement activities [14] and consider the relative benefit of each activity. They were instructed to categorize each engagement activity as either providing a high, moderate, low, or no benefit to their constituents or company using an interactive online pile sorting platform created for this study. They were then asked to describe their rationale for their ratings. Following the “benefits” questions, representatives individually reviewed the same 31 engagement activities again, considering whether the activity would require a relatively high, moderate, low, or no investment to perform. After classifying all of the activities, participants were asked to explain their rationale for determining the investment category for the activities. We also asked the participants if any of the 31 patient group engagement activities were unclear and if so, how to refine the description of the activity.

Table 1 Demographics.
Full size table

All interviews were audio recorded and transcribed verbatim. We used applied thematic analysis to analyze the data [16]. NVivo 11 software was used to organize and code the transcripts [17]. Three analysts initially coded each of the transcripts using an apriori coding structure based on questions in the interview guide. Inter-coder reliability was assessed on 10% of the transcripts. Any discrepancies in how these codes were applied were resolved through group discussion and edits were made to the codebook to aid in future application of the codes. Next, all coded text related to the initial coding structure was reviewed for information that revealed representatives’ beliefs about the benefits of and investments required for engaging patients in medical product development and also to refine the 31 activities. This information was coded and thematically organized by two trained qualitative analysts using a process of constantly comparing new information to information previously identified and coded. The data organized within the emergent thematic groups were verified by a third analyst. Finally, coding frequencies and matrices were reviewed to identify themes that were common across patient group and sponsor representatives, as well as those that were differentially expressed by certain groups, or possibly idiosyncratic. Themes were described in analytical memos, which were used to present the results below.

Results

Benefits of Patient Group Engagement

Patient group and sponsor representatives described similar potential benefits of patient group engagement (Table 2). Both groups suggested that patient group engagement can enhance the quality and efficiency of clinical trials by improving patient recruitment and retention, by reducing costs, and by making trials more able to meet expectations of regulators and payers. Other benefits suggested by the representatives include reducing the burden of participation by optimizing trial design and conduct, and amplifying the patient voice in medical product development, thereby improving the product’s ability to more directly address patient needs. In addition, sponsor representatives indicated that patient group engagement in clinical research motivates research staff, patient groups, and ultimately trial participants (if patient groups remain engaged throughout the trial period), which helps ensure that the trial is conducted well. Respondents also noted that patient group involvement in clinical research can strengthen grassroots advocacy of clinical trials and enhance the reputation of the sponsor, trial, and product in the public sphere.

Table 2 Quotes Regarding Benefits to Patient Engagement in Medical Product Development.
Full size table

Considerations Made When Evaluating Benefits

Representatives reported that the level of benefit offered by each of the 31 CTTI patient group engagement activities was determined by subjectively assessing one or more of the following factors:

  • The extent of the effect of the activity on the patient population or organization. For example, some patient group engagement activities could affect a large segment of the patient population or could affect several future trials.

  • The necessity of patient group involvement to conduct the activity.

  • The necessity of the activity to advance medical product development.

  • The ease of accomplishing the activity in the short term.

  • Reputational benefits gained by conducting the activity. For example, some patient group engagement activities might be perceived by patients and other stakeholders in the community as “the right thing to do.”

Investments in Patient Group Engagement

All representatives indicated that the investments required for successful patient group engagement include dedicated staff time and expertise, as well as financial resources, all of which can be impacted by the scope and longevity of the specific engagement activity (Table 3). Investments also could include the creation of new infrastructure, processes, and organizational policies to facilitate the activity. Representatives noted that some engagement activities may require additional time, effort, or burden placed directly on patients, which may be a cost that some groups are unable or unwilling to afford. Finally, a patient group representative reported that organizations may need to consider whether or not engaging in a particular activity, or associating themselves with a particular research partner, will cost them their reputation or ethical principles.

Table 3 Quotes Regarding Investments to Patient Engagement in Medical Product Development.
Full size table

Considerations Made When Evaluating Investments

Representatives indicated that the level of investment for each engagement activity was determined by subjectively assessing one or more of the following factors:

  • The amount of financial resources needed to conduct the activity. For example, some patient group engagement activities might be longer-term and require continual financial investment.

  • The level of staff time and expertise required across the lifespan of the activity.

  • The amount of organizational commitment needed, given existing infrastructure. For example, some patient group engagement activities might demand a great deal of commitment from the organization to establish necessary infrastructure and processes.

  • The amount of direct patient involvement and potential patient burden. For example, some patient group engagement activities might necessitate interacting directly with patient populations and require a great deal of patients' time and effort.

  • Reputational risks posed by engaging in the activity. For example, some patient group engagement activities might pose a potentially serious risk to the reputation of the sponsor or patient group if not done well or if the partnership is perceived to violate the ethos of the organization.

Modifications to the 31 Patient Group Engagement Activities

Representatives also suggested ways to refine the original list of 31 patient group engagement activities, such as clarifying any unclear descriptions of engagement methods, combining methods that were similar, and identifying any other engagement methods they felt were missing from the original list. Suggested modifications were compiled and used to condense the list of patient group engagement activities to 24 unique activities across the medical product development lifecycle (Fig. 1; see supplemental material for further description of each engagement activity).

Figure 1
figure 1

Refined List of Patient Group Engagement Activities.

Full size image

Discussion

Although best practices and research for assessing patient group engagement are still evolving, this type of collaboration is recognized as having the potential to significantly improve the clinical trial enterprise [2, 18]. Ensuring that collaboration is focused on areas where the greatest benefit can be achieved for everyone involved, given limited resources, is an important step in the development of strong partnerships to improve the relevance of information gathered from clinical trials.

CTTI Prioritization Tool for Sponsors and Patient Groups

We used the findings in these interviews to develop a web-based tool to help clinical research sponsors and patient groups, both individually and jointly, identify high-priority patient group engagement activities that will be most relevant to their clinical research interests and needs.

The resulting "prioritization tool" supports users in identifying engagement activities that are most relevant to their situation (e.g., a particular clinical trial, or a collaboration across a development program), and provides a framework for transparent and intentional decision-making. The tool is available on the CTTI website: https://prioritizationtool.ctti-clinicaltrials.org/.

The tool seeks to assist users in identifying:

  1. 1.

    Relevant engagement activities that would be of most value (high benefit and low investment) to pursue on their own or in partnership

  2. 2.

    Engagement activities that would be beneficial for their constituents but that may be too costly to invest in (high benefit and high investment activities)

  3. 3.

    Engagement activities that provide little direct benefit or cost to their constituents (low benefit and low investment activities) but could potentially be valuable to other strategic research partners

  4. 4.

    Engagement activities that are unlikely to be worth pursuing (low benefit, high investment activities).

By identifying these specific engagement activities, the user will be able to better choose which activities they would seek to gain in a new research partnership, as well as what they may have to offer potential partners. Then the partners can allocate resources to those projects that are of the most value jointly to both organizations.

Application of the Prioritization Tool

The tool walks users through completing the following 3-step decision-making process:

Step one (Fig. 2) involves patient groups and research sponsors—either working together or independently—identifying relevant engagement activities. Users of the tool are provided examples of each of the 24 patient engagement activities identified by CTTI (Fig. 1) and can also choose to add their own fit-for-purpose activities.

Figure 2
figure 2

Step 1—Identify Fit-for-Purpose Patient Engagement Activities.

Full size image

Step two (Fig. 3) involves evaluating the relative benefits and investments associated with each activity that was identified as relevant in step one. For each engagement activity, users are instructed to assess the expected level of benefit the activities will provide to their organization or constituents and the expected level of investment it would take their organization to accomplish the activity. To help evaluate the potential level of benefit offered and investment required by a particular engagement activity, the tool suggests that users consider the factors described above. These assessments are intentionally subjective, as detailed financial or strategic modeling is often unrealistic for projects at this stage. At this time users are encouraged to add more details about how they plan to implement each engagement activity and the rationale behind their benefit and investment ratings. This information is stored for future reference and may be used when sharing the results of the prioritization tool with colleagues and potential partners.

Figure 3
figure 3

Step 2—Evaluate Level of Benefit and Investment of Each Patient Engagement Activity.

Full size image

After the evaluations are made, the results are visualized in a priority matrix, where each activity is mapped onto a 3 × 3 grid consisting of rows pertaining to the level of benefit the activity is expected to achieve and columns for the level of investment required to perform the activity (Fig. 4). If desired, the user can still adjust its rating (and thus the position of the engagement activity in the matrix) either by going back to the earlier ratings or by placing the particular activity in a different cell in the matrix.

Figure 4
figure 4

Visualize Patient Engagment Activities Within a Priority Matrix.

Full size image

Step three involves identifying mutually beneficial activities. Research sponsors and patient groups interested in working together can compare and discuss priorities to arrive at activities that are of high value for each.

This project and the application of the tool have some limitations. First, we used purposive sampling to select participants who could provide expert experiential knowledge into the various ways patient groups are engaged in medical product development. Their opinions may be different from other patient group and sponsor representatives. Second, this tool does not provide guidance on all factors that influence how or why industry sponsors and patient groups may collaborate in medical product development. There may be a multitude of other activities or benefits or investments that could be considered that are not included in the tool or list of activities. To account for this in the tool, we have provided ways for users to enter their own list of engagement activities. In addition, the benefits and investments listed in the tool are only provided as aspects to consider as the user evaluates the value of the activities. Users are free to evaluate the relative “benefit” or “investment” based on their own understanding of these terms. Third, while revising the list of activities, we were guided by the participants’ feedback on the wording and thoroughness of the list but made our own subjective judgements as to what to revise for the final list. The final list of 24 activities was not re-evaluated using a consensus-seeking process. Future research could explore the breadth and clarity of the final list of engagement activities. Finally, we have not assessed the acceptability or feasibility of the final tool, and have no knowledge of users’ experience with the tool. Future research can be conducted to evaluate users’ willingness to implement the tool, as well as their experience with, and the perceived helpfulness of, the tool when engaging patient groups in medical product development.

Conclusion

In summary, CTTI has previously developed a foundational set of recommendations for patient group engagement [19, 20]. The recommendations address perceived barriers, including common legal and regulatory concerns, and encourage sponsors, investigators, and other stakeholders to engage with patient groups early and often for better and more efficient clinical trials and to develop meaningful partnerships and demonstrate mutual benefits [19]. The new tool helps implement these recommendations: it allows for up-front and continued collaboration by having both sponsors and patient groups define the level of expected benefit and investment when making decisions on which activities to prioritize. Important next steps may include demonstrating the usefulness of using this tool in fostering meaningful collaborations. Future work could focus on providing example cases where representatives from industry sponsors and patient groups use this tool collectively as a pair to identify and prioritize value-based engagement activities. These case studies may provide useful real-world examples of how the tool can be implemented as well as reveal the impact of intentional industry and patient group partnerships. It is our belief that by examining the comparative value of engagement activities and deciding which activities provide the most benefit for the least investment, meaningful partnerships may be developed that will naturally foster discussions regarding expectations, goals, and specific roles in the design, conduct, and dissemination of research. Future research can evaluate if meaning partnerships do in fact result from using this tool. Ultimately, the impact of meaningful engagement will and should be measured by the resulting usefulness of the information provided by the clinical trial [1].