Background

Patients with mental illness and chronic physical conditions are at high risk of poor quality of life and have high medical costs, poor outcomes, and high mortality [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]. Team-based models of care founded on the chronic care model [16,17,18,19,20,21], such as the collaborative care model [22,23,24,25,26,27,28] and the Patient-Centered Medical Home (PCMH) [29,30,31,32,33,34], have gained broad acceptance as efficacious population-based primary care models to improve psychiatric and medical outcomes for patients with concomitant mental and physical illnesses. However, primary care clinics have been slow to adopt these evidence-based models [35,36,37].

Practice facilitation improves implementation of team-based care models but it often does not directly address relational aspects of team culture that can be integral to sustainable practice change. Practice facilitators are health care professionals trained in primary care practice improvement methods who facilitate system-level changes [38,39,40]. Practice facilitation has been shown to improve multiple aspects of team-based care: 1) improved communication across different specialties [38, 41, 42]; 2) increased adoption of practice change [43, 44]; and 3) consensus building [45]. One large, randomized controlled trial found that practice facilitation increased “adaptive reserve”, defined as “the ability to make and sustain change.” [40] Practice facilitation is also an effective training model for practicing healthcare providers [44, 46]. Practice facilitators can be used to train providers in population-based and algorithmic management of mental illness as they often have expertise in specific areas of primary care practice [46, 47].

Although practice facilitation effectively focuses on the practical aspects of practice change, it often does not focus on the relational aspects of team culture. Team climate and relational coordination predict quality of chronic disease care [48,49,50,51]. Relational Coordination is a theory of organizational management that focuses on the interdependent relationships among people working in teams. It identifies three relational domains that affect team functioning and require nuanced communication skills: shared goals, shared knowledge, and mutual respect. Successful implementation and sustainability of team-based models of care require high levels of relational coordination for individuals from differing professional backgrounds to effectively deliver patient care. However, most healthcare professionals, including primary care providers (PCPs), receive only minimal training in team-based communication skills, leaving them ill equipped to foster strong relational coordination within their team.

Although PCPs generally prefer team-based care for patients with mental illness, transitioning to team-based care can be challenging [52,53,54,55,56,57,58,59,60,61,62,63,64,65,66]. Additionally, PCPs also have high levels of uncertainty in their knowledge and skills in caring for patients with mental illness and need more support in caring for these complex patientspatients [67,68,69,70,71]. In a qualitative study of 15 internal medicine PCPs, participants expressed a high level of distress with respect to practicing beyond the scope of their knowledge and clinical skills. They also described challenges in communicating effectively with their patients and consultants when treating patients with concomitant mental and physical illness [67]. Other studies also found high levels of uncertainty in PCPs’ clinical skills to diagnose and treat mental illness [70, 71]. This uncertainty can deter engagement in the management of mental illness and lead to suboptimal care. Thus, PCPs need additional training in both the diagnosis and treatment of mental illness and in working effectively in a team setting.

In order to create meaningful and lasting improvements to interdisciplinary primary care teams, we developed Relational Team Development (RELATED) from principles of practice facilitation and psychology clinical supervision. It aims to improve the care of patients with comorbid mental and physical illness in the primary care setting by addressing dysfunctional team dynamics and closing gaps in PCP skills and knowledge. In the development process we repeatedly revised the intervention with feedback from representatives from the target population, local stakeholders, and national experts. In this paper, we describe the iterative development process in which we incorporated expert and stakeholder feedback and modified the RELATED intervention accordingly.

Methods

The development of RELATED began with phase 0, which was generative work through a qualitative research study to identify needs of the target population (i.e. PCPs) and a literature review to identify core components of an intervention. After delineating this initial framework, we developed RELATED in three phases (see Fig. 1). In the first phase, we conducted a survey to identify factors associated with self-efficacy in team-based care and mental illness management, as well as to assess self-efficacy in these constructs. In the second phase, we conducted two focus groups with PCPs in a clinic similar to the planned intervention clinic to gain feedback on the planned intervention components. In the third phase, we convened an expert panel to further refine the pilot intervention. The Colorado Institutional Review Board (COMIRB) at the University of Colorado (Aurora, Colorado) approved all study procedures.

Fig. 1
figure 1

Iterative Development of RELATED Intervention

Full size image

Phase 0 preparatory work. Qualitative study and literature reviews

The initial framework for RELATED was based on preliminary qualitative research in which we used in-depth semi-structured interviews of 15 internal medicine PCPs working in two academic and three community health clinics to examine perceptions of patient complexity, including co-morbidity of mental and physical illnesses, and to identify domains that PCPs felt affected care of patients they defined as complex [67,68,69]. We invited 34 academic clinic and 28 community health clinic physicians by email to participate in the interviews. We used non-probabilistic sampling to achieve an even distribution of participants by gender, years in practice, and type of practice [67]. (See Additional file 1: Phase 0 Interview Guide) We learned: 1) PCPs have variable competence levels in treating mental illness; 2) Both clinic-level and larger system barriers inhibit PCPs’ ability to care for patients with mental and physical illness; 3) PCPs need additional training in patient communication; and 4) PCPs prefer didactic and experiential training in the management of patients with mental and physical illness [68]. Informed by the needs expressed by PCPs in their management of patients with mental and physical illness in the primary care setting, we performed a review of practice facilitation and psychology clinical supervision models. We aimed to use aspects of psychology clinical supervision to tailor a practice facilitation model to intervene on knowledge and skill gaps of PCPs in caring for the complex patient population.

Phase 1 methods. Survey of Colorado PCPs

To identify factors associated with PCP self-efficacy in the management of mental illness in a team-based model of care, we conducted a cross-sectional mail survey of currently practicing family medicine and internal medicine physicians in the state of Colorado (N = 900). In the survey, we validated two self-efficacy scales: Team-Based Care (TBC) and Mental Illness Management (MIM). The scales were based on Bandura’s social cognitive theory. The MIM scale addressed the content areas of diagnosis and treatment of depression, generalized anxiety disorder, and bipolar depression, and management of concomitant psychiatric and medical illness. The TBC scale addressed the following content areas based on NCQA PCMH 2011 standard: communication within the team, care coordination, population management, self-management support, and continuity of care [72]. They were scored on a 0 to 10 Likert-type scale where 0 is “not at all confident” and 10 is “extremely confident”. The scales were validated using factor analysis, Cronbach’s alpha, and comparisons to existing measures. Then, using multivariable linear regression, the TBC (7 items) and MIM (10 items) scores were assessed as predictors of communication self-efficacy, attitudes toward team care, team climate, implementation of team care, knowledge and experience treating mental illness, and practice characteristics. (See Additional file 2: Phase 1 Survey Instrument) The participants, data collection, and data analysis of this study have been previously described [73, 74].

Phase 2 methods. Focus groups with PCPs

After developing the initial framework of the intervention and revising it based on the survey of PCPs, we conducted focus groups with local PCPs from local stakeholders similar to those who would receive the intervention. Focus groups were used to obtain feedback on the modified RELATED intervention that emerged following survey results. We chose focus groups rather than key-informant interviews so that PCPs could interact and build upon each other’s ideas.

Participants

We recruited PCPs from the University of Colorado Anschutz Outpatient Pavilion Internal Medicine clinic via email and clinic team announcement. Like the clinic for the planned intervention, all PCPs were associated with the University of Colorado School of Medicine. However, they work in a different hospital system. All PCP’s in the clinic were invited to participate.

Data collection

The Principal Investigator (DL) conducted two 60-min focus groups over lunch and provided food as an incentive. DL is a PCP who works in the clinic in which the focus groups were conducted. The focus groups were semi-structured to allow for a systematic and flexible approach [15]. The interview guide was created by DL. Domains addressed in the interview guide included: experiences of team-based care in current clinic, opinions of feasibility and acceptability of components of RELATED intervention, expected barriers and facilitators to implementation of the intervention, and suggested changes to intervention components. (See Additional file 3: Phase 2 Interview Guide) Participants were encouraged to express themselves in their own terms [16]. The focus groups were audio recorded, transcribed and de-identified.

Data analysis

Two research team members (DL and DK) reviewed the transcripts to identify thematic elements in a content analysis. They then outlined the most prominent themes. These results were summarized for the full research team.

Phase 3 methods. Expert panel

Participants

National experts were recruited to guide further refinement of the RELATED intervention and pilot study. Members of the expert panel were chosen based on expertise in the subject area and methods planned for the RELATED intervention. They were chosen to complement the expertise of the research team. The research team had expertise in patients with multiple chronic conditions (LB and DL), mental health in primary care (EL and FD) the implementation of practice change (DN), study design (IB) and biostats methods (MD and LC). Three experts brought specialized expertise in the Chronic Care Model (MP); designing interventions for patients with multiple chronic conditions (CB); and practice measures for mental illness in primary care (KK). All experts who were invited agreed to participate in the panel. They were also deliberately selected to represent both the mental health and medical professions, mirroring the RELATED target patient populations. Experts were recruited through emails and follow-up phone conversations. The experts joined the research team, which beginning with development and continuing through the completion of pilot testing.

Data collection

We conducted two 90-min calls with the full research team, including the expert panel. In advance of the first meeting, all participants received a description of the RELATED intervention in its most updated format, as well as results from the qualitative study, survey, and focus group. The experts were asked to provide feedback from their area of expertise during the first meeting. After this discussion, the PI worked with the research team to modify the RELATED intervention. This modified version was sent to the full research team prior to the second call. The group discussed the modifications and the experts made final recommendations. The PI led both discussions, which were audio recorded, transcribed, and de-identified.

Data analysis

Two research team members (DL and DK) reviewed the transcripts of the expert panel calls and outlined key findings. These were consolidated into themes of recommended changes to the RELATED intervention.

Results

Phase 0 pre-work. Qualitative study and literature reviews

Results from our qualitative study of PCPs enabled development of a conceptual model for the role of PCP self-efficacy in managing mental illness in a team-based setting (Fig. 2). PCPs in the study expressed poor self-efficacy in managing patients with mental illness in their primary care practices. Therefore, we focused on self-efficacy in the conceptual model. Self-efficacy is a measurable, modifiable trait associated with provider behavior. Provider self-efficacy in performing preventive health care education has been linked to performance of guideline-concordant care [75, 76]. We hypothesized that PCP clinical knowledge, mental health resources available in clinic and PCP communication skills influenced PCP MIM self-efficacy and that PCP communication skills, PCP experience with team-based care, and PCP attitudes toward team-based care influenced PCP TBC self-efficacy. These factors became the preliminary targets of the RELATED intervention, to be tested subsequently through the survey of Colorado PCPs (see below).

Fig. 2
figure 2

Conceptual Model of Role of PCP Self-efficacy in the Care of Complex Patients

Full size image

We then conducted literature reviews of practice facilitation and clinical psychology supervision. These constructs were selected due to their known impact on factors similar to those hypothesized in our PCP self-efficacy model. Practice facilitators are health care professionals trained in primary care practice improvement methods who work with clinics to facilitate system-level changes [77]. They have been shown to increase the adoption of evidence-based care in primary care practices [38, 40, 77]. Psychology clinical supervision leads to increased knowledge of and increased acquisition of psychotherapy skills among both psychotherapists and psychiatric nurses [78, 79]. Figure 3, depicts how practice facilitation and psychology clinical supervision interventions were combined to form the basis of the RELATED intervention.

Fig. 3
figure 3

Conceptualization of Role of Practice Facilitator in RELATED

Full size image

Initial RELATED intervention

We used the results of the qualitative study to define targets of the intervention: PCP clinical knowledge, mental health resources available in clinic, PCP communication skills, PCP experience with team-based care, and PCP attitudes toward team-based. We then identified and combined core components of practice facilitation and psychology clinical supervision in the initial RELATED intervention. Figure 3 illustrates the resulting conceptualization of the practice facilitator incorporating roles of traditional practice facilitators and clinical psychotherapy supervision roles. The practice facilitator would have a clinical background in psychology and training in traditional practice facilitation. In the intervention, this practice facilitator would observe PCPs in visits with patients with co-morbid mental and physical illnesses and provide tailored feedback on the above factors.

Phase 1 results. Survey of Colorado PCPs

We had a 49% response rate (441 of 900 surveyed) from PCPs in diverse practice settings. On a 0 to 10 Likert scale mean scores (standard deviation) were 7.7 (1.7) and 7.1 (1.4) for the TBC and MIM scales, respectively. Cronbach’s alpha for the TBC scale was 0.94 and MIM was 0.88 [73]. In multivariable analysis, greater TBC self-efficacy was significantly associated with more favorable attitudes toward team-based care, the presence of a health psychologist in clinic, and greater communication self-efficacy. Solo practice was associated with lower TBC self-efficacy. Greater MIM was significantly associated with higher score on mental illness management knowledge assessment, greater communication self-efficacy and access to psychology referral. Feeling “not at all” or “slightly” prepared to manage bipolar disorder at residency completion was associated with lower MIM self-efficacy [74]. Finally, the multivariable analysis supported a focus on PCP mental illness management knowledge, communication skills, attitudes toward team-based care, and experience of treating serious mental illness such as bipolar disorder.

Modified RELATED intervention

These survey results supported the conceptual model of PCP self-efficacy developed in the qualitative study. We used these results to design content for the practice facilitator to emphasize during the one-on-one coaching sessions. We identified areas of focus for the intervention as communication skills, MIM assessment and management, and attitudes toward engaging their interdisciplinary team.

Phase 2 results. Focus groups with PCPs

We completed two focus groups with a total of 9 PCPs and 1 care manager. Participants were 80% female, 80% Caucasian, 90% non-Hispanic, and averaged an age of 44 (+ 6.7). Of the 9 PCPs, 78% completed residency between 10 and 19 years ago, the other 22% of PCPs completed residency between 5 and 9 years ago. Overall, PCPs had a positive overall impression of the proposed intervention. Participants thought the one-on-one coaching sessions could reasonably be expected to impact PCP behavior with their patients, but they were unlikely to have meaningful effect on PCP interactions with their team. They felt it would help them manage patients with mental illness. They also thought it would be therapeutic for the stress of managing patients, calling it “therapy for doctors”. However, they did not think it would help them work within a team and participate in practice improvement. They felt that a practical team activity would be necessary to influence their ability to work with their teams.

Modified RELATED intervention

From the feedback in the focus groups the idea collectively emerged to add a practice change activity to RELATED in which PCPs could engage with their interdisciplinary team members. This larger clinic-wide activity would provide a real-time opportunity for PCPs to work within their teams to identify a gap in the clinic’s care of patients with mental and physical illness. The team would work together on developing and implementing a clinical practice improvement to address the identified gap in care. This activity would offer the practice facilitator the opportunity to observe and intervene on team dynamics in real time.

Phase 3 results. Expert panel

The RELATED intervention was finalized and made pilot-ready through the expert panel. We developed a detailed structure for the one-on-one coaching sessions and the practice change activity. Burden for the PCPs and clinical team was a primary concern of the expert panel. The group decided that for one-on-one coaching sessions for each PCP should lead the intervention. In addition to the direct intervention effects, these sessions would provide the practice facilitator with the opportunity to learn about clinic culture and develop rapport with the PCPs. The group decided six meetings would be sufficient for the intended impact without creating undue burden on the practice. They also recommended expansion of didactic material out of the one-on-one coaching sessions to be integrated into the Practice Change Activity Team (PCAT) with the whole team, providing important educational content and a novel collective learning experience. Finally, they noted the absence of patients in the intervention and identified the PCAT as an opportunity for PCPs and staff to benefit from their important feedback on the selected change.

Modified RELATED intervention

The one-on-one coaching sessions were truncated to create time for the practice change activity. The latter was termed PCAT and fully developed as a lab for working on team dynamics, including hierarchical power, and applied quality improvement learning. Detailed results of the expert panel can be found in Table 1, which details the components of the RELATED intervention.

Table 1 Components of Relational Team Development (RELATED)
Full size table

Discussion

We describe the iterative development of RELATED as an intervention to enhance the care of patients with mental illness in the primary care setting through one-on-one PCP coaching and a team-based practice change activity. A practice facilitator with a background in clinical psychology delivers the intervention, which combines core components of psychology clinical supervision and practice facilitation into the aforementioned framework. The initial RELATED intervention, which included only the one-on-one coaching of PCPs, was substantively different than the ultimate pilot-ready version, which also included patients and interdisciplinary staff in a practice change activity. This significant change exemplifies the powerful effect of stakeholder involvement.

The importance provider of engagement is increasingly recognized as fundamental to developing research that will be most useful in clinical practice [80, 81]. Shelef, et.al. have described a similar method of engaging stakeholders to meet the needs of a target population [82]. They present a model of collaboratively refining the intervention with repeated input from stakeholders at multiple levels, including representatives of the target population, local stakeholders, and national advisors to develop an asthma intervention [82]. Our development of RELATED can be seen as an example of utilizing this method of engaging multilevel stakeholders. As the primary subjects of the RELATED intervention, we saw PCPs as a key target population in the development of the intervention. We incorporated PCP experience and perspectives from the in the formative development of the framework through qualitative interviews and in the larger survey. We sought local PCP stakeholder feedback through focus groups in a clinic similar to that of the planned pilot intervention. We, then, incorporated feedback from national advisors though our expert panel. This process may be useful to researchers seeking to design interventions with high acceptability in clinical settings.

Although we employed a multilevel stakeholder process involving an expert panel, we did not utilize the Delphi Technique for that panel. The Delphi Technique was initially developed by the RAND Corporation as a method of iteratively sampling expert opinion to eventually come to consensus. One of the primary benefits of the Delphi Techniques is that it allows participants to remain anonymous and, hence, maintains equal representation among experts. [83, 84] We chose to use a more open discussion among experts to encourage common understanding and cross-fertilization of ideas. The rich exchange of ideas in our expert panel led to both a deep understanding of the intention of the RELATED intervention and a valuable and collaborative discussion of suggested changes to meet that goal.

Limitations

There are several limitations of the iterative stakeholder engagement strategy used to develop the RELATED intervention. In our effort to elicit feedback from local stakeholders, we conducted focus groups with a small number of providers from one urban academic clinic. Alternatively, sampling from non-academic and non-urban settings may have increased the generalizability of the results (i.e., the RELATED intervention). Additionally, non-PCP stakeholders were omitted from the process due to our initial definition of PCPs as the primary target population. The healthcare team, more generally, became the focus of the intervention with the development of the PCAT. Therefore, additional focus groups with interdisciplinary team members could have been useful once the PCAT was added as an intervention component. This additional local stakeholder would have provided opportunity for input on the contents and structure of RELATED from a non-PCP perspective. Further, patient stakeholders were not included in the development of RELATED. Although RELATED does not directly intervene with patients, it does ultimately affect patient care.

Future directions

Following completion of RELATED development, a partnership was formed with Denver Health, a safety net hospital with a network of neighboring safety net primary care practices, who agreed to host the piloting of RELATED. This partnership provided an immediate opportunity to increase the generalizability of RELATED by testing and refining it for a safety-net hospital. We are pursuing additional partnerships with another network of safety net primary care practices that have rural reach, which would further generalize RELATED’s scope. Finally, funding is being sought to examine the efficacy of RELATED efficacy in bringing about desired change in PCPs and the relational dynamics of the team in which they practice.

Conclusions

In this manuscript we describe the use of stakeholder engagement in designing RELATED, a brief clinic-wide intervention that combines team-level practice facilitation and PCP-level clinical coaching to help primary care practices implement evidence-based practices for patients with mental illness. In an approach similar to that described by Shelef et.al [82], .we iteratively refined the intervention with repeated input from stakeholders at multiple levels, including representatives of the target population, local stakeholders, and national advisors. This method may serve as a model for the development of interventions to have high acceptability in the primary care setting.