FormalPara Key Points for Decision Makers

The methods were guided by the Strategy for Patient-Oriented Research (SPOR) Capacity Development Framework and the SPOR Patient Engagement Framework.

The innovative decision aids showed good usability and are the first to provide knowledge and support to patients and investigators to build capacity for research partnerships using a digital platform.

Patients partners were actively engaged across all phases of the project.

1 Introduction

Patient-oriented research (POR) is focused on engaging patients, their caregivers, and families as partners in the research process [1]. Efforts to date indicate that POR has a positive impact on clinical research in the areas of (1) setting research priorities; (2) developing proposals; (3) informing the cultural appropriateness of the research; (4) recruiting and retaining participants; (5) identifying outcomes important to patients; and (6) disseminating results [2,3,4,5]. Chakradhar [3] describes a scenario of a mother who reviewed a research proposal for a pediatric trial that required children to take six tablets/day and have a bone marrow biopsy every 3 months. Although the mother found the proposal scientifically interesting, she suggested that parents would never enroll their children in such a trial given the challenges of daily medicine intake coupled with a painful procedure every 3 months. Training and resources and a change in attitudes/shift in thinking from paternalism to partnerships are essential steps to including patients as partners in health research, policy, planning and practice [6].

Building relationships, improving research quality and impact, and developing best practices underpin values that impact patient engagement in health-related quality-of-life research [7]. The International Association for Public Participation (IAP2) defines five levels of engagement along a spectrum that outlines the patient’s/public’s impact on a decision (e.g. inform, consult, involve, collaborate and empower) [8]. The strongest predictors of patient/public engagement in research is researchers’ attitudes [9] and patient/public partnerships that move beyond consultative-only processes [10]. A 2017 overview of systematic reviews of patient involvement in clinical trials suggest patient engagement has been limited to providing feedback, moderating sessions, and recruitment, rather than more active processes in trial design, policy, analysis and dissemination [11]. Guidelines for establishing research partnerships with patients suggest (1) helpful organizational policies; (2) supportive researcher attitudes to patient partners grounded in shared goals and strong communication practices; (3) principles of trust, respect and co-learning; (4) POR training for all team members; (5) tools/resources for successful patient engagement; and (6) value for patient partnerships across various stages of the research cycle are essential components to building capacity for POR [12].

A report commissioned by Clinical Trials Ontario (CTO) in 2015 provided a foundation of information about patient engagement in clinical trials, including how to engage patients/public in clinical trials to improve recruitment and retention. Two of the report’s recommendations were to (1) facilitate dialog among patient organizations, health charities, sponsors, academic organizations, and industry on how best to engage potential research participants and encourage sponsors (industry or investigators) to work with patients/patient organizations when developing clinical trial protocols; and (2) offer practical tools and best practices for patient engagement [13]. Almost 50% of all Canadian clinical trials are conducted in Ontario. CTO is a provincial non-profit organization focused on collaborating with the clinical trials community to improve the environment for conducting high-quality clinical trials by leveraging the resources in Ontario. CTO has developed programs to streamline clinical trial processes (e.g. research ethics review) and to engage patients/public as partners in clinical trials. CTO seeks to ensure its programming is patient- and public-informed and engages and impacts not only patients and the public but also researchers, clinicians and decision makers [14].

There remains limited sex/gender reporting in clinical trial research [15, 16] despite the 1997 Guidance Document on the Inclusion of Women in Clinical Trials [17] and the 2008 Clinical Trials Regulatory Review: Targeted Measures for a Strengthened Framework [18], which supported the inclusion of women as participants in all phases of clinical trials. Sex and gender terms continue to be used interchangeably and incorrectly applied in research, suggesting there is a lack of appreciation that these are distinct concepts [16, 19]. In Canada, there has been poor uptake of sex and gender into clinical trial research; 6% (n =6) of trials published between January 2013 and July 2014 conducted a subgroup analysis across sex, 4% (n =4) reported sex-disaggregated data, and no publication defined sex/gender or conducted a sex/gender-based analysis [20]. Moreover, many data collection instruments fail to incorporate indicators associated with gender (e.g. income, caregiving responsibilities, household chores) [19]. Applying sex and gender terms in research requires a biological and/or sociocultural focus; it involves asking different questions and taking different approaches to collecting and analyzing trial results [21, 22]. The overall goal of this project was to disseminate sex/gender knowledge and build capacity for patient engagement in clinical trials. Specific objectives were to (1) create capacity and opportunities for patient engagement and sex/gender knowledge/uptake in clinical trials and sponsor- or investigator-led activities (e.g. clinical trial design and conduct); and (2) enhance new/early investigator sex/gender knowledge and skills related to POR.

2 Methods

2.1 Design

The Strategy for Patient-Oriented Research (SPOR) Capacity Development Framework [23] and the SPOR Patient Engagement Framework [24] were used to guide the project to build capacity and engagement for sex/gender knowledge/uptake and POR in clinical trials (Fig. 1).

Fig. 1
figure 1

Visual value model for building capacity for patient engagement and POR in clinical trials. POR patient-oriented research, PEP-CT patient engagement partnerships in clinical trials

Full size image

Our investigative team of patient partners, clinicians, trainees, early/new investigators, and decision makers are women and men who have formed strong partnerships to guide the successful execution of the project’s proposed activities. We integrated POR principles and practices by engaging patient partners across all phases of our project. Patient partners have (1) identified research priorities/search terms; (2) collected/analyzed data; (3) designed the patient partner decision aid; and (4) disseminated results through presentation.

2.2 Procedures

2.2.1 Phase 1

The goal in Phase 1 was to complete a scoping review of the literature using methods described by the Evidence for Policy and Practice Information (EPPI) and the Coordinating Centre at the Institute of Education [25,26,27,28,29], and to retrieve, screen and classify the evidence to answer the overarching review question ‘What technologies/tools/resources are used to build capacity for patient engagement and POR in clinical trials?’ This overarching review question was purposefully kept broad to ensure the characteristics of the evidence base were identified and described. Grey literature was included to determine the extent to which patient engagement and POR in clinical trials was utilized in workshops and conferences, applications/websites and repositories, and other resource materials. Patient partners identified and confirmed the search terms, and were engaged in defining search terms because there is evidence that demonstrates this strategy can increase the number of citations retrieved in a review by 34% [30, 31]. Conventional review strategies were used, i.e. sensitive searching and systematic screening. The electronic databases were searched from 1996 (inception of INVOLVE) [32] using keywords and Medical Subject Heading (MeSH) terms in accordance with the IAP2 [33] and the search criteria in the bibliographic databases (see Electronic Supplementary Material). Publications needed to be available in English and searches were conducted using selected databases, i.e. CINAHL, MEDLINE, EMBASE, PsychInfo, the Cochrane Library, ProQuest Sociological Abstracts, and grey literature sources (including Google, Google Scholar, and Web of Science). For ongoing and recently completed clinical trials, we searched the Clinical Trials Registry, International Register of Controlled Trials and the MetaRegister of Controlled Trials. Publication citations were exported from electronic search interfaces to Endnote. Titles of citations were scanned, abstracts reviewed, and all potentially relevant articles retrieved and imported to the EPPI database. Database data abstraction was performed by two investigators (AKB, KTA) and grey literature data abstraction was completed by two patient partners (AN, DW), one trainee (AS) and one investigator (KTA) using standardized data extraction forms to capture information on sex/gender and SPOR’s patient engagement guiding principles of inclusiveness, support, mutual respect, and co-building [24]. The main purpose of the scoping review was to provide an overview of existing POR research/resources and identify evidence gaps and future research/resource needs [30]. The scoping review was broad in scope and primarily focused on identifying and describing the characteristics of the evidence base [34].

2.2.2 Phase 2

The findings from the scoping review were reported at a POR consultation workshop, attended by key stakeholders. Key stakeholders included the research team, CTO, and other collaborators who were purposely invited to include investigators, clinicians, decision/policy makers, industry, health charities, patient organizations, and patient partners/investigators/decision makers from the SPOR Chronic Disease Networks. A culture of learning and interdisciplinary collaboration to build capacity for POR in clinical trials using the guiding principles of inclusiveness, support, mutual respect, and co-building was fostered during the workshop. The results from the broad map of the literature on patient engagement in clinical trials and the results from the 1-day POR consultation workshop were used to inform the development of tools to build capacity for POR and sex/gender knowledge/uptake in clinical trials. Although the format of the tools was uncertain, it was anticipated they might possibly include webinars, guidelines/best practice materials, and workshops. Input and direction was sought from stakeholders on proposed evidence gaps using nominal group techniques (e.g. structured small and large group discussion with ranking and consensus) [35]. As our focus was on obtaining feedback and insight, we did not collect sociodemographic characteristics (socioeconomic status, ethnicity, age, sex and gender, etc.) from stakeholders.

2.2.3 Phase 3

New/early investigators, researchers, clinicians, patient partners and decision makers were purposely invited from the SPOR Chronic Disease Networks to a 1-day new/early investigator POR training day. Two objectives for the day were to (1) establish key concepts, principles, and areas for patient engagement and sex/gender knowledge/uptake in clinical trial research; and (2) disseminate knowledge about the POR tools developed by the Project Team in Phase 2. The day was co-delivered by investigators and patient partners. Content related to POR (history and context, research cycle, patient engagement and levels of participation, team building, governance and decision making) was delivered in the morning using didactic teaching, think/pair/share collaborative learning strategies, and small and large group discussion. The remaining half-day focused on team building and practical tips for utilizing and integrating the POR tools developed in Phase 2 into clinical trials research. Sex/gender knowledge integrated into the decision aids included definitions with recommendations for data collection and analyses [36, 37]. Sex and gender knowledge and the concepts of inclusiveness, support, mutual respect, and co-building were incorporated into small group sessions as new/early investigators, clinicians, and patient partners participated in a mock trial using the POR tools developed in Phase 2. Small groups created inclusive mechanisms for engaging all members in the discussions. Two activities incorporated into the small group sessions included a discussion about sex and gender considerations for the mock trial (e.g. definitions and assessment of sex and gender) and a discussion on the importance of examining differences in outcomes between men and women in the mock trial. Investigative team members facilitated sex and gender discussions within each of the small groups. A large group discussion on the small-group mock case scenario processes followed, focusing on building capacity for sex/gender knowledge/uptake and POR in clinical trials, value placed on the experiential knowledge of patient partners, respectful collaboration, and team function. The day concluded with a POR training day evaluation and an evaluation of the POR tools developed in Phase 2 using the System Usability Scale (SUS) [38]. The SUS has been used across a wide range of user interfaces, including Web pages and Web applications [39]. The 10 five-point Likert questions are scored to provide a point estimate of usability with a reported reliability of 0.85 [39]. In addition, four semi-structured questions were asked to determine users’ overall impression of the POR tools: what they liked and why, what could be improved, and if anything was missing [40]. Anonymized field notes were taken by the investigative team patient partners (TC, AN, DW) and imported into NVivo [41]. Two members of the investigative team (M. Parry, AKB) used an inductive approach [42, 43] to identify major themes that emerged from the data [44, 45].

3 Results

3.1 Phase 1

After duplicates were removed, a total of 7990 records were identified through database searching; 7911 records were excluded primarily because patients were research participants and not research partners, and/or the focus was on recruitment/retention of participants (i.e. not patient partners) in ethnically diverse populations and/or those with low literacy levels. Four studies focused on decision aid support in the informed consent process for individuals who were considering participating in a clinical trial (i.e. increasing recruitment to clinical trials) [46,47,48,49].

A total of 79 studies were included in a qualitative synthesis from our database search. The only published literature found on sex/gender and POR included the Sex and Gender Equity in Research (SAGER) guidelines that included details and comprehensive procedures for reporting of sex and gender in the study design, data analysis, results, and interpretation of findings [50]. Well over 150 websites (e.g. INVOLVE, PCORI) contained a variety of patient engagement documents in various formats targeted to both patient partners and investigators (Fig. 2).

Fig. 2
figure 2

Study selection process

Full size image

Despite the breadth of grey literature on resources to facilitate patient engagement in research, few were directed to clinical trials or sex/gender and POR. Sex and gender resources included the use of testimonials from men and women who were patient partners [51] and investigators [52], and also included online training modules [36, 53], methods for integrating sex and gender into health research [54, 55], and outreach toolkits [56]. SPOR’s patient engagement guiding principle of inclusiveness was fostered through partnership plans, models for participant engagement in clinical trials [57,58,59,60,61,62,63], tools to match patient partners to research teams [63], and outreach and recruitment resources [64]. Support was provided through POR workshops, planning guides [65], briefing notes [66], steering group terms of Refs. [67, 68], role description templates [68], structures for consensus decision making [69], and methods for reimbursement and compensation to patient partners [70,71,72,73]. Mutual respect was offered through documents summarizing the evidence to facilitate team discussions [74], through communications skills training modules [75], and through newsletter templates. Co-building was evidenced through partnership surveys [76, 77] and documents that categorized various patient partner roles in governance, team meetings/working groups [74, 78], workshops/conference attendance [74], authorship, and in identifying patient-reported outcomes (PROs) [79].

3.2 Phase 2

A total of 32 investigators, patient partners, clinicians, trainees, representatives from health charity and patient organizations, research administrators and industry attended the POR consultation workshop. Of those who completed the consultation workshop evaluation (n =14), 43% (n = 6) were patient partners. After small group sessions and large group discussions, attendees directed the team to collate relevant POR information from the scoping review into two web-based decision aids (patient and investigator) to improve sex/gender knowledge/uptake and POR in clinical trials. Decision aid development was guided by the International Patient Decision Aid Standards (IPDAS) [80, 81], User-Centered Design [82] and the Ottawa Decision-Support Framework (ODSF) [83], and was designed to provide education to assist patients and investigators to make informed decisions about engaging as/with a patient partner or investigator in a clinical trial. Five core functionalities of each of the English-only patient and investigator decision aids included: (1) Introduction (get the facts on POR/patient engagement partnerships in clinical trials); (2) My Priorities (what are the priorities to each key end user [patient partner and investigator] in terms of where in the research process/lifecycle to be engaged, as well as the IAP2 level of engagement); (3) Learn More (sex/gender knowledge and resources to plan, engage and evaluate patient engagement partnerships in clinical trials); (4) My Readiness (comparing priorities with perceived benefits and risks to individual end users); and (5) My Decision (decision and next steps, such as finding a patient partner or finding a clinical trial team) [Fig. 3].

Fig. 3
figure 3

Core functionalities of the patient and investigator decision aids

Full size image

Patient and investigator testimonials identified in the Phase 1 grey literature search were incorporated into the Introduction of each of the decision aids. Sex and gender information, links and references also identified in Phase 1 were included in Learn More – How do I Incorporate Sex and Gender in my Clinical Trial? in both of the decision aids. The probability of outcomes was not included in the initial development of either decision aid as this information was not thought to be relevant; these specific decision aids were not developed for patients seeking health treatment or screening decisions. However, priorities clarification was incorporated into each decision aid as potential benefits/risks were thought to be relevant to patient partnerships in clinical trials. Decision aids assist to improve knowledge [84,85,86] and help to clarify personal priorities with the benefits and risks of engaging in a decision [80, 81]. Potential benefits/risks were considered relevant and applicable for patients and investigators in providing insight into how their priorities affect POR decisions. For example, benefits to patients to engage as partners in clinical trials might include the acquisition of knowledge/skills, reduction in loneliness/isolation or hopelessness, and increased emotional support. However, the risks of engaging as a patient partner might also include increased costs, intensified anxiety, or even a worsening of a health condition. The decision aids were designed to be interactive and web-based/digital.

3.3 Phase 3

A total of 88 new/early investigators, researchers, clinicians, health charity and patient organization representatives, patient partners and decision makers attended the new/early investigator POR training day. Of those who completed the new/early investigator training day evaluations (n =52), 29% were patient partners (n =15). The day was co-delivered by the Principal Investigator (PI; M. Parry) and CTO (DR) using didactic teaching, think/pair/share collaborative learning strategies, and large/small group discussion. Content included sex and gender (e.g. definitions, assessment and importance) and POR information (e.g. history and context, research cycle, patient engagement and levels of participation, team building, governance and decision making) [87] and preliminary evaluation of the patient and investigator decision aids (user performance and satisfaction). Median SUS scores for both the patient and investigator decision aids indicated ‘good’ usability. Attendees thought the decision aids were engaging, they increased their understanding of sex and gender, patient engagement and POR, and they would recommend them to others. They suggested more time was needed to properly distill and utilize the sex/gender and POR content of each of the decision aids. Four major themes emerged from the anonymized field notes: (1) usability (e.g. easy to navigate); (2) learnability (e.g. useful information); (3) design (e.g. appealing visual display); and (4) other (e.g. disseminate decision aids to universities and hospitals). Comments included: “I enjoyed that the decision aids were separated for patients and investigators”; “I liked it, it was user friendly and easy to navigate”; and “there could be more interaction and aimed more for mobile devices”. Patient partners recommended all content be presented at an appropriate reading level using plain language. They suggested a glossary for new terms, a section for questions to ask investigators prior to joining a research team, and more interactivity. Investigators thought it would be helpful to bookmark sections and/or have the ability to review specific components of the research process without having to review the entire ‘Learn More’ section. Investigators also suggested more interactivity and recommended a scoring feature for ‘My Readiness’.

4 Discussion

The overall goal of this project was to build capacity and engagement for sex/gender knowledge/uptake and POR in clinical trials using the SPOR Capacity Development Framework [23] and the SPOR Patient Engagement Framework [24]. Interactions with patient partners on the project team as well as patient partners who were part of the POR consultation workshop and the new/early investigator training day were based on SPOR’s patient engagement guiding principles of inclusiveness, support, mutual respect, and co-building [24]. While it was recognized that the patient partners who were part of the project team were likely highly informed about clinical trials and POR, we valued other points of view and sought to include these more broadly by inviting patient partners from health charity and chronic disease networks to the POR consultation workshop and new/early investigator training day. As our focus was on obtaining feedback and insight, we did not collect sociodemographic characteristics (socioeconomic status, ethnicity, age, sex and gender, etc.) from our patient partners. This is recognized as a potential limitation as feedback and insights may have varied by sex/gender. Having clear goals, roles, and expectations for both the workshop and the training day were important for our patient partners [88]. The project team was also committed to providing the appropriate support to patient partners who were involved in project activities. For patient partners who were involved in the workshop and training day, preparation calls were hosted in advance of each session to ensure (1) questions were answered; and (2) possible scientific/medical concepts that may be discussed were explained. It was also important to ensure that jargon was avoided and/or appropriately explained at both the workshop and the training day. The project team listened to patient partner learning needs and ensured these were effectively met to facilitate full participation in the project. (e.g. data abstraction, field notes). Honoraria and expense reimbursements were provided to patient partners to participate in the project [89, 90]. All meetings began with a reminder of the expertise and experiential knowledge of all attendees, and a safe environment was created so that all voices could be heard. As demonstrated by project outputs (e.g. workshop, training day, decision aids), patients were equal partners in collecting, co-designing and co-building resources for building capacity related to sex/gender knowledge/uptake and POR. Patient partners knew their responsibilities and expectations in each phase of the project and they assisted to disseminate results.

Even though 1996 denotes the inception of INVOLVE [32], there may be evidence prior to this date that would have been relevant for inclusion, and this is recognized as a limitation to our scoping review. However, we attempted to offset this limitation by engaging patient partners in defining search terms for the scoping review, a strategy that has demonstrated an increase in the number of citations retrieved in a review by 34% [30, 31]. A large number of studies were excluded from our scoping review, mainly because patients were research participants, not research partners. A few studies focused on decision aid support in the informed consent process for individuals who were considering participating in a clinical trial (i.e. increasing recruitment to clinical trials) [46,47,48,49]. Preference-sensitive decisions, such as participating in a clinical trial, require patients to make a choice when there is no clear evidence that one choice is better than the other. Each option, such as deciding to participate or not participate in a clinical trial, has inherent benefits and risks. The patient must weigh the risk and benefits and consider their individual priorities in making the decision. No studies focused on decision aid support to engage patients as partners on clinical trial teams.

Decision aids are tools that provide information to improve knowledge about expectations [91], benefits and risks [91], options and outcomes [80]. They assist users to gain skills in assessing uncertainties and help clarify personal priorities with the potential benefits and risks of engaging in a decision [80, 81]. Most decision aids to date have been designed to support patients in making decisions about health treatment or screening decisions, with substantial evidence to indicate their beneficial effects to improve knowledge [84,85,86] and expectations regarding priorities/values and choice [85]. In a 2017 Cochrane review to assess the effects of decision aids for individuals facing treatment or screening decisions (105 studies, 31,043 participants), decision aids reduced decisional conflict related to feeling uniformed (mean difference [MD] − 9.28/100; 95% confidence interval [CI] − 12.20 to − 6.36; 27 studies, 5707 participants, high-quality evidence). This means that decision aids helped individuals become more informed about making the right decision. Stacey et al. [85] also found that decision aids reduced the proportion of individuals who were passive in decision making (risk ratio [RR] 0.68; 95% CI 0.55–0.83; 16 studies, 3180 participants, moderate-quality evidence). Potential sex/gender differences in utilization and effectiveness of decision aids were not reported in this systematic review. As noted by Stacey et al. [85], 50% of health treatment and screening decisions do not have one best choice; decisions are considered to be preference-sensitive because there is insufficient evidence to suggest benefits offset risks, and vice versa.

The IPDAS mandates a systematic process for decision aid development that includes consultation with end users. Decision aid development includes end-user engagement in scoping and design, prototype development, alpha testing using iterative cycles, beta testing (field testing) in real-life circumstances, and production of a final version for use and/or further evaluation [91]. At a minimum, decision aids need to improve the (1) quality of the decision-making processes; and (2) quality of the choice that is made (i.e. decision quality) [92]. Core attributes of the quality of the decision-making processes include [92] (1) recognition that a decision needs to be made; (2) being informed about options and benefits/risks; (3) value clarity; (4) discussion about goals/preferences; and (5) involvement in decision making. The quality of the decision is the extent to which end users are informed and make choices that reflect their goals and preferences [80], which include (1) knowledge about options/outcomes; and (2) concordance between the decision and values/priorities that matter most to the end user. The next phase in the development of the patient and investigator decision aids will include refinement/translation and usability testing to assist us to understand the proposed requirements of each decision aid and to identify sex- and gender-specific issues to consider in the development of each of the decision aids.

5 Conclusion

Our digital patient partner and investigator decision aids are the first to provide information technology to deliver sex/gender, POR knowledge, and decision support beyond the traditional decision aids used for health screening and/or treatment decisions. Although the focus of this project has been on building capacity for POR in clinical trials, the decision aids created may be applicable to patients and investigators conducting other research projects. The decision aids have the potential to make a significant contribution to Canada’s Strategy for POR and will support the collaborative efforts of patients and investigators to build a sustainable, accessible and equitable health care system. To improve uptake, we have co-designed the decision aids with patient partners, investigators, clinicians, health charity and patient organization representatives, and decision makers, and made the decision aids available online and open access. Patients are the heart of SPOR [24]; our patient partners have been actively engaged in all phases of this project and will continue to partner and co-lead the next phase of this project to translate, refine and evaluate the decision aids (patient partner and investigator) guided by the IPDAS [80, 81], User-Centered Design [82] and the ODSF. In the next phase of this project, all alpha and beta testing outcomes will be disaggregated and reported by sex. Beta testing the decision aids in the next phase of this project will also assist us to determine the feasibility (field test) of implementing the patient partner and investigator decision aids in practice.