Background

Cardiovascular disease (CVD) is a leading cause of death in Canada [1]. Successful management of CVD involves not only the treatment of a specific disease, but also treating and preventing risk factors for CVD, including diabetes, dyslipidemia and hypertension [1–3]. However, the management of CVD is complicated by the large number of clinical practice guidelines available for conditions that contribute to this disease. An article by Ray et al. noted there are also discrepancies in recommendations across guidelines, potentially contributing to low adherence rates [2, 4, 5]. A harmonized guideline by Tobe et al. (2011) found there are over 400 recommendations for managing risk factors for heart disease [3].

Given the complexity of the management of this illness, it is imperative that practitioners use guidelines, and the most appropriate guidelines, in caring for patients with CVD and risk factors for CVD. The impact of guideline implementation has been illustrated previously; a review by Grimshaw and Russell found that using guidelines improved clinical practice [6]. Despite evidence to support the use of guidelines, there remains a gap in their implementation [7].

The dissemination of guidelines alone has little to no effect on practice [8], thus many studies have investigated interventions of varying intensity to increase the uptake of clinical practice guidelines. Numerous studies of interventions to improve the uptake of guidelines in CVD prevention are available. However, their overall impact on guideline adherence and clinical outcomes is unclear. Unverzagt et al. [9] published a systematic review on a similar topic that focused on primary care physicians’ adherence to guidelines, wherein they demonstrated these interventions can have an impact on adherence outcomes. It is important to determine the effect of these interventions on other healthcare providers, as well as determine the impact of these interventions on clinical outcomes, which is yet to be addressed in the literature to our knowledge.

We identified and synthesized the available research evidence about the effectiveness of interventions that target healthcare providers to improve adherence to CVD prevention and treatment guidelines and clinical outcomes. Our secondary objective was to explore characteristics of guideline implementation interventions and contexts that are associated with increased effectiveness. This leads to our research question: what is the most effective intervention to improve the implementation of, uptake of, or adherence to cardiovascular disease-related clinical practice guidelines by healthcare providers in randomized controlled trials?

Methods

As this research did not involve the collection of primary data, we did not seek ethics approval. This review has been registered with PROSPERO 2014:CRD42014010111. Available from http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42014010111

Search

A systematic search was conducted using search terms related to “adherence” and “clinical practice guidelines”, which was refined with the help of a medical librarian. We searched the following databases: PubMed, EMBASE, Cochrane Library (including CENTRAL, DARE and HTAs), PsycINFO, Web of Science and CINAHL (all available years, up to June 2014). Grey literature was also searched, including clinicaltrials.gov to identify potential new studies, ICTRP registry database, and ProQuest thesis database. Our search strategy did not impose any limits on language of publication (Additional file 1).

Inclusion criteria

Study design

The included studies were limited to randomized controlled trials (RCTs). We included all types of RCTs, including cluster RCTs, and nested designs.

Population

Studies that enrolled any registered healthcare providers were included. Subgroups of interest for our analyses included comparing physician participants to other healthcare providers (non-physicians). We excluded trials if less than 75 % of the participants included were certified, regulated healthcare providers.

Intervention

All studies that evaluated the impact of an intervention on the implementation of, uptake of, or adherence to a clinical practice guideline by a health care provider were included. The guideline of concern had to relate to the prevention or management of CVD, including risk factor management for any of: diabetes, dyslipidemia or hypertension. Guideline definitions were based on authors stating a guideline to be such. A study was deemed to be about the implementation or adherence to a guideline if the trial report explicitly stated that improving use of a clinical practice guideline was the focus of the intervention. Types of interventions included: academic detailing, audit and feedback, educational sessions, continuing medical education (CME) sessions, and ‘other’ (such as reminders or decision support systems).

Comparison group

We selected studies that included at least one control group. Comparison groups included usual care, a similar guideline implementation intervention of differing intensity or duration than the main intervention group, or no intervention (receipt of the intervention at a different time than the intervention group, such as after data collection).

Primary outcomes

We included trials that reported both a measure of guideline adherence and at least one clinical outcome. Measures of adherence included self-reported adherence, prescription review, and chart review. We included studies reporting any relevant clinical outcomes and considered the following groups of outcomes for analyses: mortality, hospitalizations, quality of life, and disease targets. Outcomes assessed at similar time points were combined in our analysis as short term (3–6 months), and long term (7 months or longer).

Study selection and data extraction

Articles were screened based on title and abstract using the inclusion criteria, then based on full text by two independent reviewers. Discrepancies were resolved by consensus.

Data from included articles was extracted in duplicate by independent extractors. We extracted study characteristics (study design, setting and population), a description of the intervention (the type of intervention, providers, and resources involved), comparison intervention, risk of bias, outcome measurement and results, and funding for the study. Risk of bias was assessed using the Cochrane Risk of Bias tool for RCTs [10]. All discrepancies between extractors were resolved through consensus. Data was managed using spreadsheets created for each extractor. Authors were contacted after data extraction and consensus meetings were completed to request missing data and to check the accuracy of our extractions.

Data analysis

We conducted descriptive analyses of included studies. We conducted meta-analyses (MA) for outcome results when there was sufficient clinical homogeneity across the studies. Clinical homogeneity was based on similar study characteristics (intervention type, outcome and follow-up point of interest). Meta-analyses were conducted in Review Manager (RevMan 5), using a random effects model and forest plots were generated. Intraclass correlation coefficient (ICC) for cluster RCTs were used in our meta-analyses to calculate the effective sample size to ensure the effect of clustering was taken into account in our analyses, as per the Cochrane Handbook [11]. A Z-test was used to assess statistical significance of meta-analysis results and a p < 0.05 was considered significant.

The adherence and patient outcomes were measured as both dichotomous and continuous outcome measures. Odds ratios (OR) and 95 % confidence intervals (CI) were calculated for use in the MA for dichotomous outcomes. Standardized mean differences (SMD) were used for continuous outcomes, as outcomes measuring the same construct were measured on different scales. Most continuous outcome analyses looked at the differences in mean change of each group from baseline, and this value was used in the MA, though some trials reported follow up results in each group, wherein we calculated the change score for each group to use in our MA. In order to impute the standard deviation for the change score in this instance, the standard deviation of the change score from another similar study was used.

We conducted sensitivity analyses to determine the robustness of our results, comparing the results of our analyses including and excluding studies with imputed standard deviations, and excluding studies with high risk of bias (greater than 3 domains rated as high risk of bias). We conducted subgroup analyses considering participant subgroups (physician participants and other healthcare providers), and considering the condition that was the focus of the guideline in the study (acute and chronic CVD conditions or risk factors). We planned to create funnel plots to investigate potential publication bias if at least ten studies were included for a given outcome, however this was not possible. We present a summary of the overall strength of evidence available using GRADE Summary of Findings tables produced using GRADEpro.

Results

Results of the search

We identified 12,255 potentially relevant unique citations. We excluded 12,033 citations during the initial abstract and title screening. We reviewed 222 full text publications and included 38 studies in the review [12–54] (Fig. 1).

Fig. 1
figure 1

PRISMA flow chart of study inclusions

Full size image

Included studies (Table 1)

Eighteen studies took place in the USA, 14 were completed in Europe (the Netherlands, Italy, England, and Norway), two took place in Canada, one in South Africa, one in Brazil, one in Asia-Pacific area, and one in the Virgin Islands. Thirty-five studies included an intervention to improve physician use of guidelines and ten of those studies included a nurse as a target for the intervention; two studies focused on nurses alone, and one study focused on pharmacists. The most common intervention type was educational focused intervention (18/38), followed by audit and feedback (9/38), academic detailing focused interventions (4/38), comprehensive interventions that included education, audit and feedback and an academic detailing component (2/38), and “other” interventions that did not fall into any pre-designated category (8/38). Seven trials included more than one intervention group. All studies included an adherence outcome, as per our inclusion criterion. Disease target was the most common clinical outcome reported (33/38 trials), followed by mortality (11/38). Hospitalization and quality of life data were also reported in 8/38 and 6/38 trials, respectively.

Risk of bias in included studies

Risk of bias summary graphs and tables were created using RevMan (Figs. 2 and 3). Risk of bias was often assessed as unclear due to poor reporting of a methodological procedure. The majority of trials (33/38) were cluster RCTs, therefore additional risk of bias criteria were included for these studies. Random sequence generation was most often assessed to be low risk of bias, while blinding of participants was most commonly rated as high risk of bias.

Fig. 2
figure 2

Risk of bias summary table for each study. Green indicates a low risk of bias, yellow indicates unclear risk of bias and red indicates high risk of bias, as assessed by reviewers using the Cochrane risk of bias tool

Full size image
Fig. 3
figure 3

Risk of bias summary graph, summarized for each domain. Green indicates a low risk of bias, yellow indicates unclear risk of bias and red indicates high risk of bias, as assessed by reviewers using the Cochrane risk of bias tool

Full size image

Effects of interventions

Education intervention

Seventeen trials tested an education-focused intervention and were included in a meta-analysis. These trials overall favoured the intervention, and one meta-analysis was statistically significant. Seven trials (2545 subjects) reported mortality outcomes, three of which reported mortality at a short term time point with an overall odds ratio of 0.54 (95 % CI 0.2 to 1.42). Four trials reported mortality at a long term time point with an overall odds ratio of 0.48 (95 % CI 0.11 to 1.98). Four trials (979 subjects) reported hospitalizations as an outcome at a long term time point. The overall odds ratio for this outcome was 0.88 (95 % CI 0.54 to 1.41). Six trials (2145 subjects) reported disease target results at a short term time point (SMD = −0.32 (95 % CI −0.71 to 0.07)) and five trials (2732 subjects) reported this outcome at a long term time point (SMD = −0.09 (95 % CI −0.24 to 0.07)) (Fig. 5). Seventeen trials reported adherence outcome data, six (2306 subjects) reported dichotomous data at a short term time point (OR = 2.11 (95 % CI 0.90 to 4.97)), four trials (322 subjects) reported continuous data at a short term time point (Fig. 4) and eight trials (6019 subjects) reported dichotomous data at a long term time point (OR = 1.05 (95 % CI 0.82 to 1.34)) (Fig. 6).

Fig. 4
figure 4

Forest plot of education intervention comparison for continuous adherence outcome at a short term time point (<6 months)

Full size image

Audit and feedback

Nine trials included an intervention that focused on audit and feedback, and seven of those trials reported data sufficient to be included in meta-analyses. Three trials (2240 subjects) reported disease target results at a long term time point with an overall effect of −0.44 SMD (95 % CI −1.05 to 0.17) (Fig. 5). Six trials (2983 subjects) reported adherence data at a long term time point with an overall odds ratio of 1.39 (95 % CI 0.88 to 2.21) (Fig. 6).

Fig. 5
figure 5

Summary disease target outcome forest plot for three comparisons measured by standardized mean difference, with point estimate and 95 % CIs

Full size image
Fig. 6
figure 6

Summary adherence outcome forest plot for five comparisons measured by odds ratio, with point estimates and 95 % CIs

Full size image

Academic detailing

Four trials (6017 subjects) included academic detailing as the focus of the intervention and all of these trials reported data that was included in a meta-analysis for adherence outcome. The overall odds ratio for this comparison was 1.32 (95 % CI 0.88 to 1.96) (Fig. 6).

Other interventions

Eight trials included an intervention whose focus did not fit these previous groups. Four trials (1782 subjects) included a decision support tool as the focus of their intervention. The overall odds ratio for this comparison was 1.19 (95 % CI 0.83 to 1.70) (Fig. 6).

Sensitivity and subgroup analyses

Sensitivity analysis investigating the impact of imputed standard deviations in continuous data was possible in the education intervention outcome for the disease target outcome at a short term time point. The pooled SMD from six studies in this comparison was −0.32 (95 % CI −0.71 to 0.07), while the estimate from the sensitivity analysis, with studies that included imputed standard deviation removed, was −0.27 (95 % CI −0.71 to 0.17). Another sensitivity analysis, investigating the impact of high risk of bias studies on the overall estimate was possible for the meta-analysis of the effect of education on short term adherence outcomes. The pooled odds ratio was 2.36 (95 % CI 0.86 to 6.51) before studies with high risk of bias were excluded, and 3.65 (95 % CI 0.53 to 25.15) after studies with high risk of bias were excluded.

We compared results in studies that targeted physicians only in their intervention to interventions that involved non-physician healthcare providers alone or in addition to physicians with subgroup analysis. This subgroup analysis was possible in seven comparisons, and the subgroups of physician participants alone frequently had less heterogeneity than when grouped with all studies, suggesting participants may be a source of heterogeneity (Additional file 2: Figure S1). Another subgroup analysis we conducted compared results in studies that focused on an acute cardiovascular condition to a chronic cardiovascular condition. Five comparisons showed inconsistent results although the heterogeneity was reduced in at least one of the two subgroups in all comparisons.

GRADE summary of findings tables

The overall quality of evidence identified in this systematic review was moderate to very low due to high risk of bias, imprecision, and heterogeneity (Table 2). The most patient important outcome of mortality had moderate quality of evidence associated, indicating the results may be interpreted with some confidence.

Table 1 Characteristics of included studies
Full size table
Table 2 Summary of findings table for educational interventions
Full size table

Discussion

Statement of principal findings

We have focused on interventions aimed at improving adherence to CVD guidelines. Overall studies are variable in their conclusions on whether the intervention was effective, though our quantitative analysis supports that interventions trend towards having an impact on adherence to guidelines and patient outcomes. One comparison of an education intervention for the adherence outcome was statistically significant, indicating this area of study deserves further consideration, as these interventions may help improve both adherence to guidelines, and more importantly, patient outcomes. Our results were robust where sensitivity analyses were possible. Subgroup analyses (participant and condition) reduced the statistical heterogeneity but there was inconsistency in the subgroup with the larger effect for each analysis. In some cases, the physician subgroup favoured the intervention to a greater degree than the non-physician subgroup, but in other comparisons the opposite was true. The same results were found for the condition subgroup (acute vs. chronic condition). The confidence in these recommendations ranged from moderate to very low based on a GRADE summary of findings due to imprecision, risk of bias, and inconsistency.

Strengths and weaknesses of the review

Our systematic review has several strengths, including that it was comprehensive in inclusion of studies. We included all types of healthcare providers in order to illustrate the impact these interventions can have on both physicians and non-physicians, which is increasingly important for multidisciplinary teams required for complex diseases such as CVD. We limited our study inclusion to those that reported both adherence and a patient outcome, as interventions must improve both in order to be clinically useful. All screening, data extraction, and risk of bias assessment was done in duplicate with trained reviewers to ensure the reproducibility of these results. Our quantitative analysis was pre-specified to avoid finding spurious results due to post hoc analyses. We minimized the number of comparisons that were made while ensuring comparisons had fairly good clinical homogeneity to maintain the strength of those conclusions. We also contacted authors for missing data and to verify the accuracy of our data extractions of their trial, thus we have confidence in this data.

However, this review has limitations. The first relates to the quality of reporting in trials. Reporting of risk of bias domains was poor in many trials, making it difficult to assess risk of bias. There was also significant heterogeneity in the studies’ interventions and characteristics making combining results in a meta-analysis difficult, leading to small numbers of studies included in each comparison. Meta-analyzing results was further complicated by uncertainty of the exact nature of some interventions due to limited descriptions of interventions available in publications. This also limited our ability to assess publication bias, so we were unable to determine the effect that might have on our confidence in these results.

Comparison to similar reviews

A systematic review on CVD guideline implementation strategies in primary care physicians by Unverzagt et al. reported similar conclusions on the effectiveness of education and reminder system interventions to improve adherence [9]. Our review extends these findings, illustrating the impact at the patient level on mortality, hospitalizations, quality of life and disease targets, and to different healthcare providers.

Similar to our findings, a review by Grimshaw et al. on guideline implementation noted overall the most effective interventions tend to include specific educational interventions and patient specific reminders at point of care [6].

Meaning of the review results

These results indicate there is some evidence to support the use of some interventions to improve healthcare provider adherence to CVD guidelines. Despite the limitations in the studies in this review, a trend of interventions improving adherence and patient outcomes was noted, supporting that these interventions may be more effective than passive guideline dissemination strategies. However, more studies are needed to strengthen these conclusions.

The majority of interventions included were multifaceted, which some reviews have suggested provide positive outcomes more frequently than single interventions [53–55]. However, our results were not consistent with these; we found these interventions have limited effects, which may be related to the number of components in a given intervention, as only two interventions included all of the types of interventions. A review by Squires et al. found there is ambiguity in the evidence of whether multifaceted interventions are more effective than single interventions, which is in agreement with our inconsistent findings [55].

Another possible reason for the overall small effect sizes may relate to the complexity of the management of CVD. This includes treating and preventing multiple risk factors in patients, such as diabetes, hypertension and dyslipidemia [56]. Most guidelines address only one of these diseases, and this may contribute to the small improvements found in this review. Given the multifactorial nature of CVD, it needs to be treated with guidelines that acknowledge this. Using harmonized CVD guidelines such as C-CHANGE is an important step that needs to be taken in CVD guideline implementation intervention trials to ensure the best, most comprehensive care is provided to patients [1]. This is also an important consideration as to why CVD guideline implementation strategies must differ from strategies used in treating simpler diseases such as pneumonia or asthma [57, 58].

Unanswered questions and future research

It would be beneficial for more high quality studies on this topic to be conducted to improve the strength of our recommendations, given the low confidence in most of these estimates due to a small number of studies included in each MA. Interventions should be fully described so they are not only reproducible, but future reviews are able to confidently determine homogeneous groups for meta-analyses. Future reviews on this topic should also define clinically important differences to determine whether the effects are not only statistically significant, but clinically significant as well.

Conclusions

Interventions to improve adherence to CVD guidelines can be effective at improving both adherence and patient outcomes, and are often more effective than guideline dissemination alone. Interventions that focused on healthcare provider education demonstrated statistically significant improvements. The overall quality of evidence available in this review was low, but several patient important outcomes including mortality were supported by moderate to high quality evidence.