Background

Cardiovascular disease (CVD) is the leading cause of death in the USA [1]. CVD imposes a staggering burden whether measured in life-years lost, diminished quality of life, racial and ethnic health disparities, or direct and indirect healthcare costs. Yet, the Centers for Disease Control and Prevention (CDC) estimates that more than 200,000 deaths from heart disease and stroke could be prevented each year through a concerted, national effort to control CVD risk factors such as blood pressure, cholesterol, and tobacco use.

In September 2011, the US Department of Health and Human Services launched the “Million Hearts” initiative to prevent 1 million heart attacks and strokes by 2017 [2]. In the clinical realm, Million Hearts focuses on the management of the “ABCS”—aspirin use in high-risk individuals, blood pressure control, cholesterol management, and smoking cessation [2]. To support Million Hearts, the US Agency for Healthcare Research and Quality (AHRQ) launched an initiative in 2015 to assist primary care practices in implementing clinical and organizational changes to improve the management of ABCS.

Objectives and aims

Heart Health NOW (HHN), an AHRQ-funded demonstration and evaluation, investigates whether a comprehensive evidence-based quality improvement strategy (which we call primary care practice support) accelerates the implementation of evidence-based guidelines for CVD prevention in primary care practices on a large scale. The specific aims are to evaluate the effect of primary care practice support in small- to medium-sized practices on (1) practice-level delivery of evidence-based CVD prevention, (2) patient-level health outcomes, (3) practice-level implementation of clinical and organizational changes that support delivery of evidence-based CVD prevention, and (4) practice-level capacity to implement future evidence-based clinical guidelines. HHN is a cooperative effort of the University of North Carolina at Chapel Hill (UNC-CH), the North Carolina Area Health Education Center (NC AHEC), and Community Care of North Carolina (CCNC), the primary care case management provider for NC Medicaid.

Methods/Design

Study setting

North Carolina is the tenth largest state in the USA with a population approaching 10 million and a racial and ethnic composition of 72 % white, 22 % black, and 9 % Hispanic [3]. The burden of cardiovascular disease in North Carolina remains large. The cardiovascular death rate was 263 per 100,000 as measured in 2008–2010, accounting for almost one-third of deaths in the state. These data rank North Carolina 32nd among the 50 states for cardiovascular outcomes [4].

Research design

HHN is a stepped wedge, stratified, cluster randomized trial. Each practice will start the trial as a control, receive the primary care practice support intervention at a randomized time point, and then enter a maintenance period 12 months after the start of the intervention. The intervention will be randomized to practices in one of four strata defined by region of the state (east or west) and degree of practice readiness for change (described below). Specifically, 75 practices in each region with a high degree of readiness will be randomized 1:1:1 in blocks of 3 prior to month 8 to receive the intervention at month 9, 11, or 12. An additional 75 practices within each region that have a low degree of readiness or are recruited later will be randomized 1:1 in blocks of 2 prior to month 13 to receive the intervention at month 14 or 16. The sites will be ordered within each stratum based on time of enrollment with the blocking based on this ordering. The study’s statistician will generate the stratified permuted-block random allocation lists using a computer program. As practices are enrolled, the project management system will use the randomization schedule to randomize practices and assign their intervention start date. A practice is considered randomized once an intervention start date has been assigned. Neither study personnel nor participating practices will be blind to randomized start dates.

Practice recruitment

We will recruit 300 small- to medium-sized primary care practices. To be eligible, practices must have ten or fewer providers in a single location and have an electronic health record (EHR). Enrolling practices agree to the following: (1) form a quality improvement team; (2) meet with the practice facilitator up to 8 h a month; (3) send representatives to regional learning collaboratives three times yearly; (4) meet with experts to discuss CVD prevention, CVD risk assessment, and system change strategies; (5) permit the CCNC Informatics Center to process EHR data and provide the practice with cardiovascular care registries and dashboards for clinical decision support and population health management; (6) review data on ABCS measures at least monthly; and (7) use those data to plan, conduct, and assess improvement activities. Enrolling practices complete an information technology and quality improvement readiness assessment, which is used for earlier randomization of high-readiness practices and assignment of practices to more or less intensive practice facilitation.

Primary care practice support intervention

Practice facilitators will work for 12 months with practices’ quality improvement teams to implement four “key drivers” for supporting delivery of evidence-based CVD prevention: enhanced clinical information systems with dashboards on critical measures and drill downs to patient-level care; adoption and use of practice-wide evidence-based protocols; effective use, documentation, and tracking of patient self-management support goals; and optimization of the practice care team. Practices will receive up to 8 h a month of practice facilitation depending on their level of readiness at enrollment. In addition, physician faculty will engage in academic detailing through webinars and practice visits. Detailing will focus on CVD prevention and treatment, CVD risk assessment, and new recommendations for cholesterol and hypertension management. Finally, providers and staff will participate in regional learning collaborative meetings where they can share experiences implementing evidence into practice.

Evaluation framework

Our evaluation draws on an organizational model of innovation implementation [5–9]. The innovation here is evidence-based CVD prevention (including clinical decision support and population health management tools, global risk-based treatment of lipids, new treatment options, and evidence-based smoking cessation). Briefly, effective implementation is a function of the implementation support the practice receives and the policies and practices it employs to support innovation use (see Fig. 1). Practice capacity for QI, readiness for change, and implementation climate moderate the relationship between support, policies and procedures, and effectiveness. Inner and outer contextual factors influencing organizational priorities, capabilities, and opportunities also moderate the relationship. Realization of patient- and practice-level benefits depends on effective implementation of evidence-based CVD prevention.

Fig. 1
figure 1

Conceptual framework guiding the evaluation of Heart Health Now

Full size image

Data collection

Table 1 outlines data collection, measures, sources, and timing for each construct.

Table 1 Theoretical constructs, measures, data sources, and data collection timing
Full size table

Practice facilitator contact logs

Practice facilitators (PFs) will record monthly the frequency, duration, and mode of practice contact. We will use these data to measure the intensity of implementation support provided to practices.

Academic detailing logs

Physician faculty will record visits, calls, and other contacts with practices. Webinar software will track logins and pages viewed by practice staff.

Regional collaborative attendance records

Project staff will keep records of the number and types of staff from each practice that attend regional collaborative meetings.

Key informant surveys

We will survey key informants (KIs) to assess practice capacity for QI, inner context, and outer context. The surveys will be administered at baseline, end-of-intervention, 6 months post-intervention, and 12 months post-intervention for practices enrolled earlier in the study. The KI will be the practice’s provider champion (N = 300).

Provider/staff survey

We will survey providers and staff to assess practice capacity for QI at baseline, end-of-intervention, 6 months post-intervention, and 12 months post-intervention for practices enrolled earlier; organizational readiness to implement change at baseline; and implementation climate at end-of-intervention and 6 months post-intervention. The sample will consist of 3–5 providers and staff members depending on practice size (N = 900–1500).

PF ratings

PFs will assess on a monthly basis practices’ progress in implementing clinical practice and organizational changes to support improvement in CVD prevention (implementation policies and practices). Ratings will focus on the extent to which practices have implemented four key drivers of improved CVD prevention and also level of leadership and team engagement.

Interviews

We will conduct semi-structured KI interviews (N = 300) at end-of-intervention and 6 months post-intervention to obtain supplemental data about organizational changes for evidence-based CVD prevention (implementation policies and practices) and influential inner and outer context events. The KI interview at end-of-intervention will also ask about acceptability of the implementation support received (implementation effectiveness). The KI interview 6 months post-intervention will ask about practice-level outcomes (innovation effectiveness).

EHR/informatics

We will obtain performance data on the ABCS measures (implementation effectiveness) and patients’ intermediate clinical outcomes (innovation effectiveness) from the EHR. The ABCS measures and clinical outcomes will be collected at baseline, during the intervention (quarterly), at the end of the intervention, and post-intervention (6 and 12 months follow-up). The EHR will also supply race/demographic data. EHR data will be routed to the CCNC Informatics Center, where the data will be extracted, normalized, and placed in a standard report used for the evaluation and sent back to the practices.

Patient surveys

We will survey 200 consecutive adult patients in each practice at baseline and at the end of the intervention to obtain patient experience data on communication and shared decision-making (innovation effectiveness). Data will be collected during an office visit using paper-based surveys. Eligible patients must have one or more of the following comorbidities: hypertension/high blood pressure, myocardial infarction/heart attack, angina/coronary heart disease, stroke, smoking, diabetes, and high cholesterol.

Claims data

We will use claims data at baseline, end-of-intervention, and 6 months post-intervention to assess hospitalizations for cardiovascular-related conditions, in-hospital deaths, and mortality for patients seen at participating practices (innovation effectiveness). We will have access to claims data on 85 % of the insured population in NC, specifically those covered by Blue Cross Blue Shield of North Carolina, the State Employee Health Plan, Medicare, and Medicaid. Outcomes will be aggregated to the practice level. For the baseline period, we will examine claims data for the year prior to the intervention. Following the intervention, we will examine available claims during and after the intervention for each practice.

Measures

Implementation support. We will measure primary care practice support intensity as the percentage of contacts with a practice that are in-person. We will also report the actual number of in-person contacts and number of total contacts to account for variability in contacts across practices. We will also track frequency, mode, and duration of contact with academic detailers, as well as number and type of staff per practice attending collaborative meetings.

Practice capacity for QI encompasses two constructs. The first construct, change process capability, refers to a “state of preparedness for change that is influenced by the organization’s previous history of change, its plans for continuous organizational refinement, and its ability through its social and technical systems to initiate and sustain that change” ([10], p. 193). We will measure this construct using the Change Process Capacity Questionnaire (CPCQ) [10]. The second construct, adaptive reserve, refers to “the practice relationship infrastructure; alignment of management functions in which clinical care, practice operations, and financial functions and reflect a consistent vision; facilitative leadership; teamwork; sense-making; a positive work environment; and a culture of learning”([11], p. S37). We will measure this construct using the Practice Adaptive Reserve scale [11, 12].

Organizational readiness for change refers to the extent to which organizational members are psychologically and behaviorally prepared to implement change [6, 7, 13]. We will measure readiness using the Organizational Readiness for Implementing Change scale [13].

Implementation policies and practices are the strategies that an organization puts into place to support innovation use [9, 14]. We will measure implementation policies and practices using the Key Drivers of Implementation Scales (KDIS) [15]. PFs will rate the extent of practice implementation of the four key drivers using a 5-point, behaviorally-anchored scale. Table 2 offers an example of the KDIS for planned care.

Table 2 Key Drivers of Implementation Scale (KDIS) for standardized care processes
Full size table

Implementation climate refers to organizational members’ “shared summary perception of the extent to which their use of a specific innovation is rewarded, supported, and expected within their organization” ([16], p. 1060). We will measure implementation climate using a 6-item scale [17].

Implementation effectiveness refers to the consistency and quality of aggregate or pooled innovation use (e.g., evidence-based CVD prevention) [14]. We will assess implementation effectiveness using nine ABCS measures (see Table 3).

Table 3 Process of care measures for ABCS
Full size table

We will measure acceptability of primary care practice support qualitatively by asking about KIs’ satisfaction with the frequency, duration, and quality of implementation support.

Innovation effectiveness refers to the benefits that result from innovation use (e.g., patient and practice outcomes) [14]. To capture patients’ assessment of communication, we will use selected items from the CAHPS clinician and group survey (see Table 4) [18].

Table 4 Communication questions
Full size table

To capture patients’ assessment of shared decision-making, we will use selected items from the CAHPS patient-centered medical home supplement [19] (see Table 5).

Table 5 Shared decision-making questions
Full size table

In addition, we will assess patients’ healthcare utilization and mortality using claims data. We will measure hospitalizations for stroke, acute myocardial infarction, and angina identified via primary diagnoses and in-hospital mortality for these conditions. For patients covered by Medicare and Medicaid, we will also be able to assess non-hospital related mortality. We will also collect clinical data on patients’ blood pressure, cholesterol levels, smoking status, and CVD risk based on the ASCVD risk calculator developed by the American Cardiology Foundation and the American Heart Association (see Table 6). These clinical data will be extracted and summarized by the CCNC Informatics Center.

Table 6 Practice-level patient outcomes
Full size table

We will measure practice outcomes qualitatively by asking whether evidence-based CVD prevention improved the practice’s standing in a pay-for-performance program, provided them with maintenance of certification credits, improved their payments through the BCBS of NC sponsored Blue Quality Physicians Program, or made them more attractive for recruitment into ACOs/Shared Savings Programs and other global payment programs.

Inner context refers to structural, political, and cultural features of the practice that are directly related to experiences of patients and staff, as well as features of the larger organization (if any) with which a practice is associated [20]. We will capture inner context through our measures of practice capacity for QI, organizational readiness, and implementation climate. In addition, we will assess other structural features, such as practice type (see Table 7).

Table 7 Inner context measures
Full size table

Outer context refers to features of the economic, political, and social context within which an organization resides [20]. Through the KI survey, we will assess the perceived influence of external events using a 3-point Likert scale (no influence to significant influence). In KI interviews, we will explore qualitatively how events rated “2” (some influence) influenced the practice’s efforts to implement evidence-based CVD prevention processes of care.

Data analysis

Aim 1: Evaluate the effect of primary care practice support on delivery of evidence-based CVD prevention

Our primary hypotheses are that primary care practice support will improve the percentage of patients in a practice: (a) with ischemic vascular disease who use aspirin or other antithrombotics; (b) with hypertension who have adequate blood pressure control; (c) with an LDL test performed who are prescribed a recommended dose of statin based on risk status; (d) who were screened for tobacco use and, if identified as a tobacco user, received cessation counseling and/or pharmacological therapy; and (e) who receive aspirin for primary prevention based on a 10-year CVD risk status. Our secondary hypotheses are as follows: (a) practice capacity for QI, organizational readiness to implement change, implementation climate, and contextual factors (inner and outer) will moderate the effect of primary care practice support on evidence-based CVD prevention; and (b) primary care practice support will reduce race-based disparities in evidence-based CVD prevention.

Statistical analysis

The statistical analysis of the primary outcomes for Aim 1 will be based on a generalized linear mixed model (GLMM) for longitudinal binomial data at the practice level. Allowing for the number of patients who are observed within a practice to vary across time, the models will be estimated by maximizing the pseudo-likelihood, specifying practices as clusters, and using empirical variance estimators. For example, the number of hypertensive patients whose blood pressure is under control Y it, out of N it hypertensive patients, will follow a binomial (N it, π it) distribution where, conditional upon a practice-specific random intercept b i0 and slope b i1, πit = E(Y it/N it|b i0,b i1) is the expected proportion of hypertensive patients whose blood pressure is controlled within the i-th practice in the t-th month; t = si (baseline), si + 3, si + 6, si +9, si + 12, si + 18 (6 month post-intervention follow-up), and for some practices si + 24 (12 month post-intervention follow-up). The GLMM is

$$ \mathrm{Logit}\left({\uppi}_{\mathrm{i}\mathrm{t}}\right) = {\beta}_0 + {b}_{\mathrm{i}0} + {\beta}_1{X}_{\mathrm{i}1} + {\beta}_2{X}_{\mathrm{i}2} + {\beta}_{12}{X}_{\mathrm{i}1}{X}_{\mathrm{i}2} + {g}_1\left({t}^{*};\uppsi \right) + {b}_{\mathrm{i}1}t + {g}_2\left(t;\uplambda \right){X}_{\mathrm{i}\mathrm{t}3} + {g}_3\left(t;\uptheta \right){X}_{\mathrm{i}\mathrm{t}4} $$
(1)

where X i1 = 1 indicates east region and 0 for west region; X i2 = 1 indicates higher readiness and 0 for lower readiness stratum; g t(t *;ψ) is a smooth spline function of calendar time (t *) for the secular trend corresponding to the control condition with finite dimensional parameter ψ; X it3 = 1 indicates the active (12 months) intervention period versus 0 otherwise and g 2(t; λ) is function for four piecewise conjoined linear segments each of 3 months duration with λ being the vector of slope parameters; X it4 = 1 is an indicator for the maintenance period versus 0 otherwise and g 3(t;θ) is a function for two piecewise conjoined linear segments the first of 6 months duration to which all practices will contribute data and the second 6 months durations to which only a subset of practices will contribute data as determined by the stepped-wedge design. Finally, b i0 and b i1 are assumed to be bivariately normal random variables with respective variances σ 0 2 and σ 1 2 and covariance σ 01. Primary interest is in the treatment effect describing the log odds of a randomly sampled patient in a practice having the outcome 12 months post-intervention relative to the log odds of a randomly sampled patient from the same practice having the outcome during the pre-intervention period; this log odds ratio is well-defined as a function of λ parameters. Finally, the average causal effect of 12 months of intervention will be calculated as the average difference in proportions, Σi[E(Y it/N it|b i0,b i1,t = si + 12)—E(Y it/N it|b i0,b i1 t = si)]/300; a 95 % confidence interval will be determined by bootstrapping. The random coefficients logistic model (1) will be fitted using SAS Proc GLIMMIX.

The secondary hypotheses will (a) introduce practice capacity for QI, organizational readiness to change, implementation climate, and contextual factors as main effects and effect modifiers in model (1) and introduce race as a main effect and effect modifier to examine whether primary care practice support reduces race-based disparities in the use of evidence-based CVD prevention. Examination of race will require outcome data aggregated at the practice level to be broken out by race.

Statistical power

Power to detect the effect of intervention after 12 months with respect to the outcome of hypertension control is simulated for the stepped-wedge study design assuming 300 practices, and an average of μ = 1200 or 1600 patients per practice (assuming a typical practice has 4 physicians with 300 or 400 patients per physician). For data collection months t = si (baseline), s i + 3, s i + 6, s i +9, s i + 12, and s i + 18 (6 months post-intervention follow-up), power calculations are based on practice-month summaries consisting of 1800 binomial observations (6 × 300). Power is determined by simulating data from a model similar to model (1) but, for simplicity, assuming no temporal trend:

$$ \mathrm{logit}\left\{E\left({Y}_{\mathrm{i}\mathrm{t}}/{N}_{\mathrm{i}\mathrm{t}}\Big|{b}_{\mathrm{i}0},{b}_{\mathrm{i}1}\right)\right\} = {\beta}_0 + {b}_{\mathrm{i}0} + {\beta}_2{X}_{\mathrm{i}2} + {\beta}_3{X}_{\mathrm{i}\mathrm{t}3} + {b}_{\mathrm{i}1}t $$
(2)

where X it3 = 0 if t = si; X it3 = (t – si)/12 for s i < t ≤ s i + 12, and X it3 = 1 for t = s i + 18. This model assumes a linear logistic trend during the active intervention period and a constant slope during the maintenance period equal to the 12-month intervention effect. Additionally, model (2) specifies a practice-specific intercept and slope assumed to be normally distributed with means 0 and variances σ 0 2 = 0.42, σ 1 2 = 0.32 and covariance σ 01 = −0.15, values estimated from preliminary pretest-posttest data on 19 medical practices (not shown). Primary interest is in exp(β 3), the practice-specific odds ratio of having a positive outcome (e.g., blood pressure controlled) after 12 months intervention relative to baseline. The number of patients in the i-th practice assumed to be constant across time (N i = N it for all t) was randomly selected from N i ~ N(μ, μ 7/4) to account for expected variability in practice size. A complete data set is then generated using model (2) with (β 0 = 0.50, β 2 = 0.05, β 3 = 0.05) corresponding to a baseline of 62 % blood pressure control for least ready practices. The fraction of times that H 0: β 3 = 0 is rejected with two-sided α = 0.05 Wald tests under the correctly specified model was taken to be the simulated power for detecting a non-null odds ratio β 3. When the true β 3 = 0, the results represent simulated type I error, which is seen to be close to the nominal 0.05 significance level. The stepped-wedge design has excellent power for the outcome, percentage of hypertensive patients with controlled blood pressure (see Additional file 1).

Aim 2: Evaluate the effect of primary care practice support on patient-level health outcomes

Our primary hypotheses are that primary care practice support will improve the percentage of patients in a practice who indicate that their provider (a) always communicated well with them during the visit, and (b) involved them a lot of the times in shared decision-making about their medications. Our secondary hypotheses are that primary care practice support will decrease (a) the percentage of patients within a practice who were hospitalized for a cardiovascular disease-related condition, (b) the percentage of patients within a practice who were hospitalized for a cardiovascular disease-related condition or died, and (c) race-based disparities in CVD-related hospitalizations and mortality. We will explore whether primary care practice support improves the percentage of patients in a practice whose tobacco use status and overall cardiovascular risk were assessed, whose blood pressure (BP) was adequately controlled, and who received appropriate management with regard to aspirin use, tobacco cessation support, and use of statin therapy.

Statistical analysis

For the primary hypotheses, communication and shared decision-making are binomial outcomes derived from patient surveys administered at baseline (t = si) and at the end of the intervention period (t = si + 12). The model to assess improvement in these outcomes is a simplification of model (1) because there are only two repeated outcomes:

$$ \mathrm{Logit}\left({\uppi}_{\mathrm{i}\mathrm{t}}\right) = {\beta}_0 + {b}_{\mathrm{i}0} + {\beta}_1{X}_{\mathrm{i}1} + {\beta}_2{X}_{\mathrm{i}2} + {\beta}_{12}{X}_{\mathrm{i}1}{X}_{\mathrm{i}2} + {g}_1\left({t}^{*};\uppsi \right) + {b}_{\mathrm{i}1}t + {\beta}_3{X}_{\mathrm{i}\mathrm{t}3} $$
(3)

where X it3 = 1 if t = si + 12 (end-of-intervention period) and X it3 = 0 if t = si (baseline). Note that exp(β 3) is the odds ratio of a positive response (e.g., always communicated well, involved them in shared decision-making a lot of the time) after 12 months of intervention relative to the odds of a positive response at baseline, controlling for secular trends via ψ. The average causal effect of 12 months of intervention versus the control condition will be calculated as the average difference in proportions as described for Aim 1.

For the secondary hypothesis, model (2) will apply to claims data for the percentage of patients (a) hospitalized for a cardiovascular disease-related condition and (b) hospitalized for a CVD-related condition or died. For these outcomes, X it3 = 1 if t = si + 18 (6 months post-intervention) and X it3 = 0 if t = si (baseline). To assess whether primary care practice support decreases race-based disparities in these outcomes, the binomial data will be obtained for each race and practice and time point. Other facility-level patient outcomes are given in Table 4. Model (3) will be fitted to all binomial outcomes (e. g., percentage of patients…) using SAS Proc GLIMMIX. For continuous outcomes (e.g., a 10-year CVD Risk), an identity link function leading to a linear model fitted with SAS PROC MIXED will be used with variance weight 1/N it where N it is the observed number of patients in the i-th practice at the t-th month.

Statistical power

Statistical power for the primary hypotheses is simulated using model (3) with β 1 = β 12 = 0 and dropping g 1(t;ψ) from the model. It is based on a planned constant sample size (denominator) of N = 200 patients/practice at each time point. For the communication outcome, we assumed β 0 = 1.4 corresponding to a baseline rate of 80 % for a low ready practice (X i2 = 0). For the shared decision-making outcome, we set β 0 = −0.8 giving 30 % baseline when X i2 = 0. Finally, we fixed β 2 = 0.05 and β 3 = 0.05, variances σ 0 2 = 0.42 and σ 1 2 = 0.32, and covariance σ 01 = −0.15. These values for variance components are adopted from Aim 1 for lack of preliminary data for Aim 2. They are considered reasonable estimates because the statistical analysis for Aim 2 involves a random effects logistic regression model for repeated binomial data. Power exceeds 98 % for detecting differences as small as 0.04 in proportions between intervention and control (see Additional file 2).

Aim 3: Examine the effect of primary care practice support on implementation of clinical practice and office system changes that support delivery evidence-based CVD prevention

Our primary hypothesis is that primary care practice support will increase implementation of clinical practice and office system changes to improve evidence-based CVD prevention. Our secondary hypotheses are that (a) primary care practice support intensity will increase implementation of clinical practice and office system changes, and (b) practice capacity for QI, organizational readiness to implement change, and contextual factors will moderate the effect of primary care practice support on the implementation of clinical practice and office system changes. We will explore the acceptability of primary care practice support from the perspective of the provider champion.

Statistical analysis

A random coefficients linear mixed model will be applied to the KDIS based on practice facilitator ratings collected monthly throughout the intervention period:

$$ E\left({Y}_{\mathrm{i}\mathrm{t}}\Big|\ {b}_{\mathrm{i}0},\ {b}_{\mathrm{i}1}\right) = {\beta}_0 + {b}_{\mathrm{i}0} + {\beta}_1{X}_{\mathrm{i}1} + {\beta}_2{X}_{\mathrm{i}2} + {\beta}_{12}{X}_{\mathrm{i}1}{X}_{\mathrm{i}2} + {g}_1\left({t}^{*};\uppsi \right) + {b}_{\mathrm{i}1}t + {g}_2\left(t;\uplambda \right)\ {X}_{\mathrm{i}\mathrm{t}3} $$
(4)

where the explanatory variables and parameters are defined as in model (1). In particular, X it3 = 1 for all t = si + 1,…,si + 12, and g 2(t; λ) is function for four piecewise conjoined linear segments each of 3 months duration in the intervention period with λ being the vector of slope parameters, which we expect to be positive indicating an increase in the number of clinical practice and office system changes with an increase in the amount of implementation support. The main hypothesis is an overall positive change in expected outcome between 12 months post-intervention and 1 month post-intervention. The first secondary hypothesis is that primary care practice support intensity (with X it3 re-defined and scaled as a continuous measure ranging from 0 to 1) will be positively associated with the KDIS scale Y it. The second secondary hypothesis will assess practice capacity for QI, organizational readiness to implement change, and contextual factors as effect modifiers on the rates of implementation given by g 2(t; λ).

Statistical power

Power is simulated using a mixed effects linear model similar to model (4) for the KDIS scale with β 1 = β 12 = 0 and dropping g 1(t;ψ) giving an assumption for simplicity of no secular trend:

$$ E\left({Y}_{\mathrm{i}\mathrm{t}}\Big|\ {b}_{\mathrm{i}0},\ {b}_{\mathrm{i}1}\right) = {\beta}_0 + {b}_{\mathrm{i}0} + {\beta}_2{X}_{\mathrm{i}2} + {b}_{\mathrm{i}1}t + {\beta}_3{X}_{\mathrm{i}\mathrm{t}3} $$
(5)

with X it3 = (t-1)/3 for t = 1, 2, 3, and 4. Model (5) differs from model (4) in that we model 3-month averages of KDIS scales producing quarterly data, in accordance with preliminary data reported in Figure 2 of Halladay et al., which is as follows: electronic health record (mean 2.78, sd = 3.17); care protocols (mean 1.71, sd = 4.37); planned care (mean 2.16, sd = 3.84); and self-management support (mean = 1.38, 3.67). It follows their variances are 10, 19.1, 14.7, and 13.5, respectively [15]. Model (5) assumes that the intervention effect on KDIS is linear in time during the yearlong implementation period; under this model, the main hypothesis is an overall positive change in expected outcome between 12 months post-intervention and 3 months post-intervention (β 3 > 0). For power calculations, we set β 0 = 2, β 2 = 0.5, β 3 = 1.0, σ 0 2 = 5 (or 8), σ 1 2 = 4, σ 01 = −2 and σ e 2 = 5 (or 8). Then, under the random coefficients model, Var(Y it) = σ 0 2 + σ 1 2t2 + 2 t σ 01 + σ e 2. Additional file 3 shows that power to show an effect of intervention on KDIS scales exceeds 85 % under the assumptions.

Supplemental analysis

We will use simple statistics to describe the frequency and distribution of positively and negatively coded comments about the acceptability of primary care practice support. We will examine whether acceptability varied by primary care practice support intensity, baseline capacity for QI, and inner and outer contextual features. We will use the qualitative data to explore the reasons why KIs found primary care practice support acceptable (or not) and recommendations they have for future efforts.

Aim 4: Evaluate the effect of practice facilitation on practice capacity to implement future evidence-based clinical guidelines

Our primary hypothesis is that primary care practice support will improve capacity to implement other evidence-based clinical guidelines in the future. Our secondary hypotheses are that (a) contextual factors will moderate the effect of primary care practice support on practice capacity to implement additional evidence-based clinical guidelines, and (b) primary care practice support will generate additional positive practice outcomes.

Statistical analysis

We will analyze adaptive reserve and change process capability with a linear mixed model applied to the measures collected at baseline (t = si), at the end of the intervention period (t = si + 12), and, for the early enrolled practices, at 6 months post-intervention (t = si + 18). The model to assess improvement in these outcomes is a simplification of model (1) because there are only two repeated outcomes:

$$ E\left({Y}_{\mathrm{i}\mathrm{t}}\Big|\ {b}_{\mathrm{i}0,}{b}_{\mathrm{i}1}\right) = {\beta}_0 + {b}_{\mathrm{i}0} + {\beta}_1{X}_{\mathrm{i}1} + {\beta}_2{X}_{\mathrm{i}2} + {\beta}_{12}{X}_{\mathrm{i}1}{X}_{\mathrm{i}2} + {g}_1\left({t}^{*};\uppsi \right) + {b}_{\mathrm{i}1}t + {\beta}_3{X}_{\mathrm{i}\mathrm{t}3}+{\beta}_4{X}_{\mathrm{i}\mathrm{t}4} $$
(6)

where X it3 = 1 if t = si + 12 (end-of-intervention period) and X it3 = 0 otherwise; X it4 = 1 if t = si + 18 (6 months post-intervention) and baseline is the reference time period. Other assumptions are as in model (4). Primary interest is in β 3, the difference in mean measure between baseline and end-of-intervention period.

In secondary analyses, contextual factors will be examined as additional main effects and as statistical interactions with X it3 in order to assess whether there are effect modifiers for the effect of primary care practice support on practice capacity to implement future evidence-based clinical guidelines. To test the hypothesis that primary care practice support generated additional positive practice outcomes, we will calculate 95 % confidence intervals for proportions of practices reporting positive outcomes, both overall and stratified by practice characteristics (inner context). We will also use logistic regression to ascertain whether additional positive outcomes varied as a function of primary care practice support intensity, KDIS scores, ABCS measures, and inner context variables.

Statistical power

For the primary hypotheses, we conservatively calculate power by ignoring any information from individuals 6 months post-intervention and allow a 15 % dropout (N = 255). Additional file 4 shows that our minimal detectable differences are similar to mean change reported by Nutting et al. and Solberg et al., respectively, for adaptive reserve and CPCQ [10, 11].

Supplemental analysis

We will explore the reasons why some practices benefited more than others by identifying themes about actions taken, missed opportunities, influential inner and outer contextual factors, and other situational factors.

Trial status

Practices are being recruited. The high-readiness practices randomized to start in month 9 will receive the intervention beginning January 2016.

Discussion

Significance

Practice facilitation, when combined with other quality improvement interventions, is an effective strategy for supporting the implementation of evidence-based guidelines in primary care settings [21]. However, no previous effort has conducted a controlled trial of practice facilitation at the scale or level of intensity that we propose. Results will have significant implications for healthcare payers, government agencies, and other stakeholders seeking to use practice facilitation, alone or combined with other quality improvement interventions, to support national initiatives to scale up and spread evidence-based clinical guidelines.

Limitations

Recruiting and retaining a large number of practices to engage in a yearlong, comprehensive quality improvement effort poses significant challenges. Potential recruitment barriers include fear of financial risk, reluctance to share data, acquisition by larger health systems, and competing priorities. Potential retention barriers include loss of a key clinician, resignation of a practice manager, and financial difficulties. We will attempt to mitigate these barriers by using practice facilitation to help practice achieve other goals (e.g., PCMH recognition) or enhance billing for services. Also, CCNC and AHEC will leverage their strong relationships with hospitals and health systems to boost recruitment and prevent attrition.

Our stepped-wedge study design affords some control of the effects of secular trends and other factors unrelated to our comprehensive quality improvement strategy. We will supplement this by documenting and including in our statistical analysis temporal events that could affect the implementation and outcomes of the strategy. However, the effects of outer context factors can be challenging to measure and model.

Impact

If successful, our comprehensive quality improvement strategy will have a positive impact on both practices and patients. Hundreds of small- to medium-sized primary care practices, many located in rural and medically underserved areas, will be better prepared for the transition to value-based payment, wherein an increasing proportion of reimbursement is tied to performance on quality of care measures. Likewise, discernible reductions in cardiovascular risk among the estimated 900,000 adult patients served by participating practices would likely prevent thousands of cardiovascular events within 10 years.