Background

The patient-centered medical home (PCMH) is a model that emphasizes team-based, integrated care [1-3] and has been heralded as a potential solution to cost, quality, and access issues plaguing the US health care system [4-6]. PCMH models are generally characterized by: (a) the provision of comprehensive care by an integrated team of providers (either under a single roof or virtual) responsible for the majority of patients’ clinical needs; (b) a patient-centered philosophy that treats patients and family members as partners; (c) care that is coordinated across the continuum and between settings, that is accessible, and aligned with patient preferences; and (d) a demonstrated commitment to and engagement in quality-improvement (QI) activities [7]. Both private and government entities have invested substantially in PCMH implementation [8-10].

To successfully implement and sustain complex, evidence-based quality-improvement initiatives, organizations must be willing to invest the necessary time and resources [11,12]. Even when such investments are made, efforts may fail or dissemination may occur slower than desired [13]. Like other quality-improvement initiatives, PCMH implementation appears to require substantial assistance and supporting resources [14]. These resources might include policy guidance documents, in-person or virtual learning collaboratives, collections of “on-demand” resources (i.e., “toolkits”), brief daily meetings, registries, and systems for measuring the effectiveness of quality-improvement interventions [15]. The usefulness of a particular resource may differ across PCMH roles, for example, providers versus nurses or care managers versus dietitians. Although some research has evaluated the use of specific online quality-improvement resources [16], it remains unclear which implementation resources are most widely used, by whom, and which of those resources PCMH personnel find most useful. The scarcity of published research on the factors associated with uptake and spread is not unique to the PCMH setting [11].

The Veterans Health Administration (VHA) launched a nationwide PCMH initiative, called the Patient Aligned Care Teams (PACT), in 2010 [17] investing $822 million [9] in additional staff and instituting a nationwide training program and regional learning collaboratives [17]. In implementing PACT across hundreds of medical centers and clinics, VHA leaders aimed to strike a balance between standardization of certain core elements among facilities and opportunities to experiment and implement local solutions [17]. To support implementation, the VHA identified a core group of geographically disparate providers to implement structural changes and test strategies to bolster quality, efficiency, and patient satisfaction. Members of this group served as coaches for other local staff [17]. Five regional demonstration laboratories were established to assess the model’s impact and test methods of improvement with the goal of spreading successful ideas [17]. Although practice facilitation was not a formalized component of the VHA’s implementation strategy, in some cases, regional VHA networks provided internal coaches for PACT teams [18].

The VHA identified an extensive list of PACT-related training needs [19] and made available a wide variety of activities and resources to support PACT implementation. Table 1 lists each of the activities and resources, provides a statement of purpose or goal, and identifies an example for each. These activities and resources were generally intended to diffuse information about PACT, help define roles and responsibilities for PACT teams, improve communication, assist in goal setting, measurement, and process improvement activities, and ensure that PACT teams were able to track patients as they transitioned between care settings (e.g., inpatient to ambulatory). While this was the intended implementation strategy, the actual training and support likely varied; in particular, promised staffing increases to support the new model appear to have fallen short in many instances [19,20].

Table 1 Description of PACT-related activities or resources to support implementation
Full size table

We analyzed data from a national VHA survey to address the following research questions: Which of these activities and resources were most used and most helpful? How did assessments vary across different primary care team members? Our findings may help other health systems in thinking about how to prioritize investments in the development and dissemination of implementation-boosting resources.

Methods

Survey administration

We analyzed data from a nationwide, cross-sectional survey conducted in the VHA during the summer of 2012. Survey administration is described elsewhere [21]. Primary Care and Nursing clinical leadership were asked to distribute the web-based survey to all the staff working in primary care, regardless of whether or not they were a member of a designated PACT teamlet (a teamlet is comprised of a primary care provider, nurse care manager, nursing assistant or licensed nurse practitioner, and administrative associate) [22]. Surveys were anonymous and voluntary. The survey was disseminated to the staff in 20 of 21 VHA regional networks. Sites participating in two of the five regional demonstration laboratories were not surveyed because they fielded similar, but non-anonymous surveys not reported here. Facility-level identifiers were used to link survey data to VHA facility complexity scores, which is an administrative measure used to categorize facilities based on patient volume, scope of clinical services, teaching, and research activities [23].

Respondents were asked to rate the helpfulness of ten different PACT-related activities or resources (i.e., facilitators) encompassing PACT education and training, formal and informal team meetings, quality-improvement methods, measurement tools and data on performance, new models for scheduling, disease registries, and an online quality-improvement toolkit (Table 1). The facilitator items were developed in conjunction with the five regional demonstration laboratories based on early experiences with PACT and were selected to represent activities or resources that were widely applicable and thought to be important to the model. The survey instrument provided only the names of the activities and resources and not the intended goals or purposes nor examples of what their application looked like in the VHA.

Outcomes

The first outcome of interest was whether respondents used a resource or participated in a PACT-related activity. Respondents who rated a resource or activity as “not helpful”, “somewhat helpful”, or “very helpful” were considered to have used or participated in it. Those who responded “not applicable”, “not involved”, or “don’t know” were categorized as non-users/participants. The second outcome was how helpful respondents rated each resource or activity (i.e., its utility). Responses were collapsed into two categories—“not helpful” and “somewhat/very helpful”—, and the sample was restricted to respondents who were categorized as having used or participated in each activity or resource.

Data analysis

To account for nesting and the fact that helpfulness was contingent on resource use, a 3-level random intercept (survey respondents within 678 clinics, within 135 parent VHA health care systems/medical centers) two-part mixed model was estimated for each of the ten resources. The first equation in the two-part model predicted likelihood of use/participation (ten models) while the second part of the two-part model predicted the utility (helpfulness) of those resources (ten models).

Respondent-level covariates included: primary care role (e.g., provider, nurse, technician, etc.); years working in primary care; years working in the VHA; ethnicity; age; sex; having supervisory responsibilities (no supervisory responsibilities versus any, including team leads, first line supervisors, managers, and executives); percent of time spent working in primary care; and PACT teamlet membership. We also included a facility-level covariate for facility complexity, an index of the sophistication, and breadth of care provided at a given VHA medical center [23]. Odds ratios (ORs) and 95% confidence intervals (CIs) are reported. We used listwise deletion for infrequent covariate missingness.

Data analysis was conducted with Microsoft Excel (Redmond, WA, USA), SPSS version 21 (SPSS IBM, New York, USA), and R version 3.1 (R Foundation for Statistical Computing, Vienna, Austria). This study was determined to be a non-human subject research project by the Stanford University Institutional Review Board.

Key results

Survey respondent demographics

There were 6,464 survey respondents. With 13,742 support staff and more than 4,500 providers working in primary care at the end of 2011 [22], survey respondents accounted for approximately 35% of those working in VHA primary care. Majorities of respondents were female (71%), aged 40–59 years (59.9%), and non-Hispanic white (67.9%) (Table 2). A plurality (43.8%) had worked in the VHA for two to ten years, and the majority (84.2%) spent more than 80% of their time working in primary care. Most respondents (59.7%) reported no supervisory responsibilities. Primary care providers, which included physicians, nurse practitioners (NP), and physician assistants comprised 27.4% of the respondents; 17.6% were nurse care managers, which included registered nurses (RNs) and a small number of NPs; 17.3% were licensed practical nurses (LPNs), licensed vocational nurses (LVNs), or certified nursing assistants (CNAs); 11.3% worked in an administrative role; and 4% were medical/health technicians. The majority of respondents (79.3%) reported being assigned to a PACT teamlet.

Table 2 Respondent characteristics,n = 6,464 survey respondents
Full size table

Use and utility of PACT implementation resources

The most widely used resources were teamlet huddles (80.4% of respondents), local education sessions about PACT (75.7%), measurement tools to assess team performance (74.7%), and regular teamlet meetings to discuss improvement (72.5%) (Table 3). Quality-improvement methods to conduct small tests of change were the least used by 42.4% of respondents.

Table 3 Utility of PACT implementation activities or resources,n = 6,464 survey respondents
Full size table

The resources most often rated somewhat or very helpful were “teamlet huddles” (by 90.4% of the respondents who used that resource), “regular teamlet meetings to discuss improvement” (88.3%), “disease registries” (81%), and “local education sessions about PACT” (76.6%). Although less often reported as helpful, two-thirds (66.7%) of those who conducted small tests of change rated as helpful.

Significant predictors of resource use

Accounting for all other covariates, supervisors were significantly more likely (OR range, 1.46 to 1.86) to report the use of each of the ten resources than non-supervisors (Table 4). Non-PACT team members were significantly less likely (OR range, 0.17 to 0.78) to report the use of each resource than PACT team members. Longer tenure in the VHA was positively associated with use of seven out of ten resources.

Table 4 Significant odds ratios (95% confidence intervals) for predictors of resource use 1
Full size table

Relative to primary care providers, RN care managers (RNs working as a core member of a PACT team) were significantly more likely to use each of the ten resources (OR range, 1.23 to 2.14). RN case managers (RNs who provide condition-specific case management, such as diabetes, across teams) were more likely than providers to use four of the ten resources. LPNs, LVNs, and CNAs (OR range, 1.31 to 2.12) and other RNs (OR range, 1.34 to 1.93) also tended to be more likely to report resource use relative to primary care providers. Dieticians, mental health professionals, pharmacists, and social workers were less likely to report resource use than primary care providers.

For five of the ten resources, respondents who worked in primary care less than 20% of the time were significantly less likely (OR range, 0.52 to 0.73) to report use compared to those who worked in primary care more than 80% of the time. Age, race, and facility complexity were not consistently predictive of resource use/participation. See Additional file 1 for the complete listing of odds ratios.

Significant predictors of utility among resource users

For eight of the ten resources, supervisors were less likely to rate the resource as being somewhat or very helpful (OR range, 0.72 to 0.84) than non-supervisors, after accounting for other covariates (Table 5). PACT team membership was not consistently associated with utility ratings. Longer tenure in the VHA tended to be positively associated with rating resources as helpful (OR range, 1.86 for “Scheduling tools” to 6.55 for “Online toolkit”).

Table 5 Significant odds ratios (95% confidence intervals) for predictors of resource utility among resource users 1
Full size table

Across all ten models, respondents working in administrative or clinical associate roles were less likely to find the resources helpful (OR range, 0.31 for “QI methods” to 0.73 for “Scheduling tools”) than providers. For eight of the ten resources, nurse care managers were less likely to rate the resources as helpful than providers. Overall, the time spent working in primary care was not consistently associated with utility ratings. Facility complexity was not significantly associated with utility ratings. See Additional file 2 for the complete listing of odds ratios.

Discussion

Implementing a PCMH is a fundamental redesign of primary care, and we know of few efforts to systematically rate the use and utility of the implementation resources. As this model continues to spread, health systems can benefit from knowing how to package and deliver implementation-support resources to members of PCMH teams [24-26], something relatively little is known about [27].

Teamlet huddles were the most used and most helpful resource. These brief meetings allow PCMH team members to talk about and plan for the daily management of patient demands and clinic workflow, address specific patients’ needs and preferences, and improve the delivery of preventive services [28]. In at least one other study, huddles have been found to be associated with better teamwork and an improved practice environment [28]. The informal nature of these meetings and the fact that they are temporally connected to the day’s workflow may make this especially salient to busy primary care staff.

Quality-improvement methods to conduct small tests of change were the least used, although over 40% of the respondents reported using QI methods, and two-thirds of the users found them to be helpful. PCMH measurement tools were widely used but less often rated as helpful. These findings are striking in light of the centrality of Plan-Do-Study-Act (PDSA) and related quality measurement approaches in many primary care systems and PCMH models. It may be that greater training and/or experience is needed before PDSA and similar approaches are useful. Although not part of this analysis, some VHA regional networks did provide internal coaches. Internal coaches may have been especially helpful in assisting staff on the use of quality-improvement tools and methods and in helping staff see the applied value. We also did not assess the use of more comprehensive approaches to PACT implementation, such as practice facilitation, which has been found to be effective for guideline implementation in other settings [29]. In general, PCMH-implementation leaders may want to consider training and dissemination strategies that emphasize the linkage between measurement activities and PCMH effectiveness to boost uptake. Regular teamlet meetings to discuss improvement were also widely used and perceived as helpful, but patterns of use and utility were less clear than for other resources. Although the online toolkit of care delivery and organization tools was not as widely used or highly rated as the other resources, it represents a relatively inexpensive and “on-demand” resource to facilitate the sharing of best practices across disparate sites [30]. PCMH leaders should consider how to best disseminate online toolkits in the context of their own health system.

Individuals working in administrative roles were more likely than providers to have used or participated in activities intended to improve clinic access (e.g., through proactive scheduling) and quality-improvement methods to conduct small tests of change. However, they were less likely to rate those resources as being helpful than providers. If, in fact, administrative staff is the most appropriate audience for those particular resources, our data suggests there may be a need to revise their content, format, or delivery. On the other hand, if administrative staff is not the intended audience, it may be helpful to clarify for those working in primary care the purpose and intended audience of these resources. Although supervisors were more likely to have used or participated in the various PACT-related activities or resources than non-supervisors, they were less likely to rate them “helpful.” There are several possible reasons for that, including the fact that supervisors may not actually have been learning anything new or that the resources were not optimized to their job functions. Those working in supervisory roles may not have had the ability to “opt out” of participation in certain activities, making them more likely to be exposed regardless of how helpful they found the activities to be. In contrast, support staff may have self-selected those activities and resources they found most appealing. Supervisors may also be more comfortable than non-supervisors in giving a frank appraisal of resources or may simply have more time to devote to exploring new resources than front-line staff.

Although individuals working in nursing roles were consistently more likely to have used the resources or participated in the activities than providers, they were less likely to rate them as being “helpful.” The fact that providers who rated the resources and activities were consistently more likely to rate them as “useful” than other users suggests that there may be a need for additional customization of the intervention-boosting resources if we intend for them to appeal to a more diverse group of individuals.

Depending on roles and responsibilities, individuals may engage differently during the implementation of complex interventions like PCMH. In their analysis of survey data from primary care staff who participated in a virtual learning collaborative, Butler et al. found that those who benefited most from the collaborative were those with prior PCMH training and those who fully engaged in collaborative activities though non-providers and newcomers to the PCMH model found the collaboratives to be less helpful [31]. Similarly, in their qualitative analysis of interviews on PCMH implementation among staff delivering care to older adults, Hoff et al. found that a single, across-the-board implementation approach may not be effective [32]. Having a cadre of implementation resources may be valuable, and knowing what resources to make available when selectively targeting certain groups of individuals may be a more efficient and effective means of ensuring uptake. Our data provides a framework for guiding those decisions.

Facility complexity was of limited predictive utility, which may indicate that more person and workgroup targeted implementation-boosting efforts is warranted.

There are limitations to our cross-sectional study. First, there are important limitations to the generalizability of our findings. The VHA’s highly integrated structure may have contributed to the ease with which a PCMH model could be implemented. Similarly, this may have made it easier to disseminate the resources. Outside the VHA, economic incentives, competition, and other external forces may serve to facilitate PCMH uptake [4], which could not be assessed in this study. Second, there is potential measurement bias related to the survey items which were developed for the PCMH evaluation. A number of the resources identified in the survey had similar sounding names. Respondents may not have been able to accurately differentiate between them resulting in some inaccurate ratings or measurement errors. For example, “Teamlet huddles” versus “Regular (non-huddles) teamlet meetings.” Third, there is potential selection bias. This survey was distributed through clinical leadership and was anonymous and voluntary; therefore, we do not have a defined sample from which to derive a response rate or detect response bias. However, we were able to compare our respondent demographics to primary care respondents to a separate survey fielded annually to all VHA employees, which in 2012 achieved a 62% response rate, and our respondents’ demographics were very similar; our sample had a higher proportion of supervisors and fewer African-Americans, among the four occupations comprising PACTs [21]. Our sample was also representative of primary care clinics, with 69% of VHA primary care sites nationally being represented in our findings. Future research correlating timing of the availability of activities and resources with the larger implementation efforts and outcomes would add value to these types of evaluations.

Future assessments of the use and uptake of implementation-boosting resources would be enhanced by examining trends in use over time. For example, some activities and resources may be most valuable when implementation is beginning while others are important to the long-term sustainability of whatever is being implemented. This information could help inform the allocation of financial and personnel resources needed at the onset to support implementation versus resources needed several weeks or months down the line. Additionally, knowing where the system is most likely to face challenges in adopting new care delivery models or quality programs will inform the customization, timing, and marketing of these types of resources.

Conclusion

How to best implement and spread complex quality improvement and system redesign initiatives like the patient-centered medical home is a challenge. These data shed light on a little-studied but crucial question: What resources do staff find most helpful in implementing PCMH? Teamlet huddles were the most frequently used and liked resources while QI methods to conduct small tests of change, though often promoted in PCMH models, were least frequently used although found helpful by a majority using them. Our data support targeted outreach to those less invested in the host system adopting the innovation, such as those with shorter tenure or in supporting roles. Such outreach will likely be even more important outside of integrated systems where institutional loyalties may be more fragmented. Our analysis of the availability of implementation-boosting resources during the VHA’s rollout is a useful model for policymakers and systems redesign staff charged with setting priorities for the design and dissemination of complex initiatives.