Background

The disconnect between the development of health research and its subsequent utilization in healthcare practice has been well established [1,2,3]. Underutilization of evidence may impact health and functional outcomes in patients [4, 5], and has been attributed to how evidence has been disseminated with the intended audiences [6]. Additionally, research conducted without the involvement of knowledge users, such as clinicians, patients, caregivers, policy-makers or decision-makers, may contribute to its underutilization because critical components of the research process (e.g. setting priorities, establishing research questions, choosing methods, collecting and analysing data) do not incorporate the perspectives and experiences of the knowledge users. The lack of involvement of knowledge users may result in production of evidence that is irrelevant to them [7, 8]. Research is historically within the purview of academia with responsibility for establishing the research questions and agenda, designing and conducting the study, and disseminating the results [6]. At times, this researcher-driven approach develops evidence that is perceived as irrelevant by knowledge users and results in underutilization of evidence in healthcare practice [7,8,9].

Approaches to conducting research that involve a partnership between researchers and knowledge users during the research process are now being employed to develop knowledge that is deemed more relevant to knowledge users [6]. These research partnerships are rooted in approaches to evidence development that actively involve knowledge users in any part of the research process [10,11,12]. Research partnerships aim to develop more meaningful evidence for knowledge users than researcher-driven approaches, thus potentially enhancing implementation and improving health outcomes and the efficiency of a healthcare system or organization [13]. Acknowledging that numerous complementary traditions coexist, such as integrated knowledge translation and community-based participatory research (CBPR), we utilize the term “health research partnerships” and we define it as collaborative research activities specific to health that involve a minimum of (1) one researcher associated with an academic institution and (2) one nonacademic partner such as an organization, clinician, patient, caregiver, policy-maker or decision-maker [7, 10, 12].

Numerous benefits of health research partnerships have been reported in the literature which impact researchers and knowledge users [14,15,16,17]. For instance, in an analysis of reviews on research partnerships across all disciplines, Hoekstra et al. [17] reported increased motivation for research projects, more positive attitudes towards research, increased accessibility to healthcare information and enhanced feelings of empowerment, confidence and being valued. Further benefits include increased participant enrolment rates [15, 16], strengthened social networks [14,15,16] and improved research skills and capacity [15, 17].

The extent of knowledge user involvement may vary within health research partnerships [11, 17], and can be examined using existing criteria, such as the Spectrum of Public Participation developed by the International Association for Public Participation (IAP2) [18]. The IAP2 Spectrum consists of five levels of public participation, namely inform, consult, involve, collaborate and empower, with “inform” representing the lowest level of engagement and “empower” representing the highest (Additional file 1) [18]. The IAP2 Spectrum has been used to classify the level of patient and public participation in selecting and developing patient-reported outcome measures in paediatrics [19].

There have been several calls for research to identify, describe, evaluate and validate theories, models and frameworks (TMFs) of health research partnerships [20,21,22]. This research is needed to explain why research partnerships succeed or fail, to clarify assumptions about research partnerships, and to help understand at what point and the ways in which to engage with knowledge users [7, 22]. Theories, models and frameworks organize concepts, thinking and observations [23,24,25,26]. Furthermore, they offer clarity on various aspects of implementation practice and research, which may explain why they are often grouped together [27]. Models and frameworks are similar in that they are organizational templates that can be used to plan, anticipate challenges, identify performance measures and measure the impact of research partnerships [26, 28]. A theory is a set of connected concepts, definitions and relational statements that present an organized way of observing relationships among variables [24, 25]. A theory can describe, explain and predict a phenomenon [24, 25]. Unlike a model or framework, a theory can explain why a health research partnership was or was not successful or may predict a successful research partnership [22]. Because TMFs can be utilized to deepen our understanding of aspects of health research partnerships, it is necessary to identify, describe, evaluate and validate TMFs of health research partnerships.

Research reviewing and synthesizing TMFs of research partnerships has emerged [7, 29]. Jull et al. [7] sought to identify frameworks of knowledge user engagement, which they defined as “an arrangement in the governance of the research process with those who influence, administer and/or who are active users of healthcare systems and that leads to co-production of knowledge, and associated concepts” (p. 2). Using the Engagement in Health Research Literature Explorer (https://www.pcori.org/engagement/engagement-literature), Jull et al. [7] identified 54 frameworks and 15 concepts (Table 1) of knowledge user engagement that could help researchers and knowledge users operationalize research partnerships. While the concepts identified provide a useful overview of similarities and differences within existing partnership TMFs, Jull et al. [7] did not explicitly identify or describe the characteristics of the identified frameworks, and this research may be needed to evaluate and help select a TMF. Additionally, research to identify and describe TMFs of health research partnerships may advance their use in research and produce more relevant evidence for knowledge users, thus increasing the utilization of evidence in healthcare practice. Therefore, our objectives were threefold: (1) identify TMFs of health research partnerships, (2) describe the characteristics of the identified TMFs of health research partnerships and (3) map each identified TMF to Jull et al.’s [7] 15 concepts of knowledge user engagement.

Table 1 Concepts of knowledge user engagement as described by Jull et al. [7]
Full size table

Methods

Our scoping review followed methodological frameworks outlined by Arksey and O’Malley [30] and Levac et al. [31] The reporting of our scoping review was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Extension for Scoping Reviews (PRISMA-ScR) [32]. We developed a scoping review protocol a priori and published it on the Open Science Framework (https://osf.io/qntym) [33]. The steps in our scoping review are discussed below.

Step 1: establishing the research question(s)

  1. 1.

    What theories, models and frameworks of health research partnerships have been identified and described in the published literature?

  2. 2.

    What are the characteristics of the identified theories, models and frameworks of health research partnerships?

  3. 3.

    What concepts of knowledge user engagement proposed are present in the identified theories, models and frameworks of health research partnerships?

Step 2: identifying relevant studies

We collaborated with a research librarian to develop our search strategy, which included both controlled vocabulary (e.g. Medical Subject Headings) and free text terms informed by previously published literature (e.g. theory, model, framework, CBPR, participatory action research, patient and public involvement, integrated knowledge translation) [7, 34, 35]. We searched MEDLINE (Ovid), Embase and CINAHL (Cumulative Index to Nursing and Allied Health Literature) for articles from January 2005 to June 2021. The time frame for our search reflects the period of increasing publications specific to research partnerships [7, 35]. Trial searches were conducted from 24 April until 14 May 2020. A final search was conducted on 20 May 2020. We completed an updated search on 23 June 2021. Our full Ovid search strategy can be found in Additional file 2. The Ovid search strategy was adapted and applied to Embase and CINAHL.

We also searched the Engagement in Health Research Literature Explorer because it is an open-access database that consists of peer-reviewed articles related to engagement in health research (https://www.pcori.org/engagement/engagement-literature). This online repository of literature was developed by the Patient-Centred Outcomes Research Institute (PCORI), and the collection of articles in the PCORI Explorer is kept up to date with regular searches of PubMed and MEDLINE [36]. For details on the search terms and search strategy that PCORI staff members utilize to search PubMed and MEDLINE for applicable articles, please see: https://www.pcori.org/engagement/engagement-health-research-literature-explorer/engagement-health-research-literature-explorer-supplemental-methods-information. We searched the PCORI Explorer from January 2018 to June 2021 to capture research that was not previously included in Jull et al. [7]. The articles in PCORI can be searched via article topic type, types of stakeholders engaged, and phase(s) of research in which engagement occurred, from identifying research questions to sharing study results [36]. Within article topic type, we searched the Framework, Editorial, Commentary category in the PCORI database because it includes “manuscripts that express a theoretical view on engagement in health research, including scientific commentaries, opinion briefs, or conceptual pieces such as models or frameworks” [36]. Furthermore, we completed a hand search of the supplemental data from the review by Jull et al. [7]. Given the volume of included studies, we did not conduct a grey literature search.

Step 3: selecting the studies

Title and abstract screening included articles that (1) identified as a research partnership (minimum of one researcher associated with an academic institution and one partner such as an organization, clinician, patient, caregiver, policy-maker or decision-maker) [7, 10, 12], (2) referred to a TMF for the partnership, (3) were specific to health, (4) were published between January 2005 and June 2021, and (5) were written in English, the primary language of the research team. We excluded articles if they lacked an abstract or were a protocol paper, conference abstract, thesis, dissertation, commentary, opinion piece or editorial. During screening, we specifically looked for the “index” publication, namely a TMF’s first publication presenting its development as the definitive reference for the TMF [37]. However, not all TMFs were published in a way that it was possible to identify the first publication from the abstract. In these situations, if the article met the inclusion criteria, it was included in level 2 screening [37]. Prior to title/abstract screening, the first author (BT) pilot-tested the screening criteria on 50 articles and refined them to enhance clarity. Three teams of two reviewers completed title/abstract screening. All reviewers met prior to beginning screening to discuss the screening criteria. Each team completed a calibration exercise on 30 randomly selected articles to promote consistency in screening. Conflicts were resolved by consensus.

Full-text screening included index publications if they explicitly described (1) the TMF, (2) how the partner(s) were involved in the development of the TMF and (3) how the TMF informed the research partnership. We excluded the index publications if they were a book or commentary or they could not be retrieved with reasonable effort. Full-text screening occurred in two stages. First, we screened the full texts of index publications identified in title and abstract screening for inclusion. Secondly, we employed an ancestry and snowball search approach to locate the index publication from articles that referenced a TMF [29, 38]. This involved a hand search for the index publications via Google Scholar or our university library [29, 38]. There were no restrictions on when an index publication was published to be included in data analysis. Prior to full-text screening, the first author (BT) pilot-tested the screening criteria on 25 articles and refined them to improve clarity. One reviewer (BT) completed full-text screening. A calibration exercise was completed between three teams of two individuals on 12 randomly selected articles per team to ensure that the one reviewer was consistent in screening. The reviewer met every 2 weeks with the last author (KMS) to discuss concerns with full-text screening until it was completed. Both level 1 and 2 screening were completed on Rayyan (https://rayyan.qcri.org/welcome).

Step4: data charting

An Excel data extraction form was developed a priori and pilot-tested by the first author on 10 randomly selected included articles. Through an iterative process, the data extraction form was revised to include information specific to (1) authors, (2) country of publication, (3) year of publication, (4) title of TMF, (5) intended users, (6) theoretical underpinning of TMF, (7) methodology, (8) methods utilized to develop the TMF, (9) purpose of the TMF, (10) extent of partner involvement in the development of the TMF as per the IAP2 Spectrum [18], (11) phase of research that the TMF related to [7], (12) concepts of knowledge user engagement identified by Jull et al. that the TMF related to [7], and (13) whether the TMF was graphically depicted by a figure or model. One reviewer (BT) completed data extraction on all included articles. A calibration exercise was conducted between two authors (BT and DS) on nine randomly selected articles to ensure the reviewer was accurate and consistent with data extraction. BT and KMS met virtually every 2 weeks to discuss data extraction until it was completed.

Step 5: collating, summarizing and dissemination of results

Descriptive statistics were completed to identify the TMFs of health research partnerships including the number of index publications from which data were extracted. Additionally, we reported on counts and/or frequencies and proportions specific to the characteristics of the TMFs we extracted data on. A narrative synthesis was completed to describe the characteristics of the TMFs. A narrative synthesis is a systematic and transparent analysis approach utilized in reviews to examine and summarize text to explain the findings [39]. The research team employed an iterative process when collating and summarizing the findings to ensure consensus.

Results

Identifying TMF of health research partnerships

See Fig. 1 for our PRISMA flowchart [40]. Thirty index publications were identified after full-text screening. We conducted an ancestry and snowball search for index publications on an additional 75 articles, which yielded another nine index publications. During the ancestry and snowball search we did not know which TMF was referenced in the article until we completed full-text screening. At times, the TMF we located from the ancestry and snowball search had already been identified in previous screening. Once screening was completed, 39 articles which described the development of a model or framework of health research partnerships were included for data analysis [41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79]. No articles describing theories were included. Moving forward we refer to models and frameworks (MFs) only.

Fig. 1
figure 1

PRISMA flowchart [40]

Full size image

Characteristics of MF of health research partnerships

See Table 2 for characteristics of included articles. Twenty-four articles (62%) were published in the United States. Most articles (n = 30, 77%) did not explicitly indicate a methodology. When they did, qualitative methodology was the predominant methodology reported (n = 8, 21%). Only two articles (5%) indicated that the MFs developed were underpinned by theory.

Table 2 Characteristics of included model or framework (n = 39)
Full size table

Table 3 depicts the methods explicitly reported to develop the MFs. Literature review (n = 11, 28%) and meetings (n = 10, 26%) were the predominant methods utilized, whereas systematic review (n = 1, 3%) was the least used. The number of methods utilized to develop a single MF ranged from n = 1 to n = 4. Eight articles (21%) did not report the methods utilized to develop the MFs.

Table 3 Methods utilized to develop model or framework (n = 39)
Full size table

The most frequently reported purpose of the MFs was to guide or manage (n = 14, 36%) a health research partnership. Sustaining the partnership was the least often reported purpose (n = 3, 8%). For more details on the purpose of the MFs, see Table 4.

Table 4 Purpose of model or framework (n = 39)
Full size table

Figure 2 highlights the level of partner involvement in developing the MFs. Most MFs (n = 15, 38%) were developed using collaboration. For details specific to the phase of the research process the MF could be applied to, that is prepare, plan, conduct or apply, see Additional file 3.

Fig. 2
figure 2

Level of knowledge user involvement in developing model or framework (n = 39) based on the IAP2 [18]

Full size image

Concepts of knowledge user engagement

Specific to the 15 concepts of knowledge user engagement, we found that ethics—principles/values (n = 36, 92%) was the concept most often represented in the identified MFs (Table 5). Relational process (n = 31, 79%), knowledge user—prepare, support (n = 26, 67%) and resources (n = 26, 67%) were also commonly represented. Methodology (n = 1, 3%) was the least represented concept. The number of concepts represented in each MF ranged from n = 3 to n = 12. The median of the total number of concepts represented across the 39 MFs was n = 7.

Table 5 MFs (n = 39) mapped to Jull et al.’s 15 concepts of knowledge user engagement [7]
Full size table

Discussion

We conducted a scoping review which identified and described 39 MFs of health research partnerships, but we did not identify any theory. Theory is utilized to predict and explain aspects of phenomena such as the success or failure of health research partnerships [24, 25, 80]. We did not aim to examine the success or failure of health research partnerships, or to identify factors that predict successful partnerships, and this may explain why we did not identify any theory. Furthermore, unlike theory, MFs are organizational templates that may be utilized to guide a health research partnership [26]. Our scoping review sought to identify the TMFs that were utilized to inform aspects of the health research partnership, that is, to guide the steps necessary for a health research partnership, which may also account for why we only identified MFs being used.

All MFs had representation from at least three concepts of knowledge user engagement, and no MFs encompassed all 15 concepts. We found that ethics—principles/values was the most represented concept in the MFs identified in our scoping review (Table 5). Jull et al. [7] described ethics—principles/values as “conduct knowledge user-researcher partnership work in an ethical way demonstrated by reflection on ethical concepts, and/or concern with particular values and research conducted in ways reported as meaningful, respectful, inclusive of those in the research partnership” (p. 7) (Table 1). Our scoping review sought to identify the TMFs which explicitly included concepts which influenced the research partnership, and this might explain why ethics—principles/values was most represented in our study. Relevancy, respect and inclusivity have all been identified as facilitators of health research partnerships [21, 81]. Partners embarking on a collaborative research project and developing an MF to inform the partnership may include aspects of relevancy, respect and inclusivity in the MF knowing they are facilitators of partnerships. Therefore, it might not be unexpected that we found explicit descriptions of ethics—principles/values in nearly all the MFs we identified in our study. We feel this is an encouraging finding, as it suggests that researchers and knowledge users collaborating in health research partnerships position ethical considerations as an important concept underlying their partnerships. While not examined in our scoping review, we speculate that health research partnerships underpinned by ethical principles and values may influence the success of these partnerships and would be a valuable topic for future research.

Like Jull et al. [7], we found variability in the number of concepts of knowledge user engagement represented within the included MFs. Specific to our study, the concepts ranged from 3 to 12 (Table 5). One explanation for this variability may be related to our full-text screening criteria. We included MFs that consisted of concepts to inform aspects of the health research partnership. However, several of the identified MFs also included additional concepts of knowledge user engagement, namely in dissemination, sustainability or evaluation. We did not exclude MFs if they captured these other aspects of knowledge user engagement. For instance, Swarbrick et al. [76] developed the COINED (CO-Researcher INvolvement and Engagement in Dementia) model, and we found that it had the largest number of concepts of knowledge user engagement represented in it (n = 12) (see Table 5) [76]. The COINED model not only included concepts that were partnership-focused (i.e. researcher—prepare, support; knowledge user—prepare, support; relational processes; and ethics—principles/values), but it also included concepts specific to the research process (i.e. research agenda, methods, data collection, analysis, dissemination and evaluation) [76]. Therefore, the COINED model had the largest number of knowledge user concepts represented in it [76]. In contrast, one of the frameworks with the fewest concepts was that of Ward et al. [79]. We mapped four knowledge user concepts represented in the framework: researcher—prepare, support; relational process; ethics—principles/values; and ethics—policy/rules (Table 5) [79]. These four concepts are underpinned by ideas such as power-sharing, trust, respect, inclusivity and developing meaningful research for all partners, which reflect a focus on the partnership as opposed to the research process itself [7]. Because this framework by Ward et al. [79] was focused on relational aspects of the partnership, it only included four concepts of knowledge user engagement and did not include concepts reflective of other aspects of the research process such as methods, data analysis, dissemination or evaluation.

Regardless of the number of concepts of knowledge user engagement identified within each MF, we cannot infer the quality or usability of the MF. Without a quality appraisal of the MFs, we cannot state that one MF is better than another. Instead, we suggest that future research could utilize an established evaluation tool, such as the Centre of Excellence for Partnership with Patients and the Public (CEPPP) evaluation tool, to assess the MFs for scientific rigour, involvement of knowledge users in their development, and their usability [82]. The CEPPP has been utilized in previously published research which evaluated the quality of frameworks for patients and the public involved in research [29]. A quality appraisal of the MFs could provide researchers and knowledge users with information to help them choose an MF appropriate for their health research partnership. Additionally, a quality appraisal of MFs may encourage their utilization, thus facilitating partnerships between researchers and knowledge users.

As one of our objectives was to map the concepts of knowledge user engagement to the identified MFs, we decided that we would extract these concepts only if they were explicitly represented in an MF—that is, the concept of knowledge user engagement had to be clearly represented in either a graphical depiction of the MF or described in the text of the article. We opted for this coding approach to maintain objectivity and provide researchers and knowledge users interested in MFs of health research partnerships with an accurate depiction of the concepts of knowledge user engagement within each MF we identified. As we read an MF, we referred to the descriptions of the concepts provided by Jull et al. [7] and utilized the descriptions to determine whether the concept of knowledge user engagement was explicitly mentioned. For instance, Jull et al. [7] described the concept of methodology as follows: “[d]ecide on the research methodology (approach) or report process to justify the use of the proposed methodology” (p. 7). When we searched for representation of methodology in an MF, we read the text and/or reviewed the graphical depiction specifically looking for the terms “methodology” or “approach” or “report on process”. If we did not find these terms within the MF, we coded the concept as not represented. We acknowledge that this was a strict approach to employ. We believe it may explain why some of the MFs we identified included a smaller number of concepts of knowledge user engagement than other MFs. However, we believe our results mapping the concepts of knowledge user engagement to the MFs are helpful for researchers and knowledge users embarking on a collaborative research project. They can refer to our results for an MF to plan, guide, implement, enhance or sustain the partnership and review the concepts of knowledge user engagement represented in the MFs to determine which MF may meet their needs. The researchers and knowledge users can then seek out the MF for further information about it.

Strengths and limitations

Strengths of our scoping review included our use of the methodological frameworks by Arksey and O’Malley [30] and Levac et al. [31] to guide the systematic approach we undertook to promote rigour for our scoping review. Specifically, we liaised with a research librarian to develop the research question and search strategy which included a relevant time frame, key search terms and multiple databases to ensure we captured the most appropriate articles for inclusion. Additionally, we utilized the PRISMA-ScR to provide guidance on reporting our scoping review [32].

One limitation of our study was deviation from our scoping review protocol. We had planned for two independent reviewers during full-text screening and data extraction to enhance methodological rigour, but title and abstract screening took longer than anticipated due to the high volume of articles included. Reviewers were no longer available to assist with full-text screening and data extraction because they were required for other projects. To maintain rigour, we completed pilot testing and multiple calibration exercises of our full-text screening criteria and data extraction form. Additionally, BT and KMS met every 2 weeks during data extraction to discuss the extraction process. Despite not having two independent reviewers for full-text and data extraction, we are confident our processes for full-text screening and data extraction maintained rigour. A further limitation of our study was the exclusion of non-English articles and articles with no abstracts.

We acknowledge that we did not involve knowledge users in our study. Now that we have identified and described MFs of health research partnerships, we feel it is necessary to better understand knowledge users’ perspectives of MFs that inform the partnership process. Future research could explore knowledge users’ attitudes, beliefs and experiences specific to MFs of the health research partnerships.

Conclusion

Our study aimed to identify and describe the characteristics of TMFs of health research partnerships, and to map concepts of knowledge user engagement to the TMFs. We identified 39 models or frameworks of health research partnerships, which we defined as a partnership between an academically affiliated researcher(s) and non-academically affiliated partner(s). Of significance, no theory of health research partnerships was identified, which may limit the ability to explain or predict successful health research partnerships. Encouragingly, the concept of ethical principles and values was one of the most frequently represented in the MFs. This suggests that ethical considerations are an important concept informing partnerships between researchers and knowledge users and may enhance successful health research partnerships. We believe our findings are valuable to researchers and knowledge users partnering on a research project. The models or frameworks we identified could be sought out by partners and utilized to inform aspects of the health research partnership process, such as guiding or managing a partnership. Ultimately, this may contribute to research that is more relevant to the knowledge users, thus enhancing the utilization of evidence in healthcare practice and improving health outcomes and the efficiency of a healthcare system or organization.