Predictors of the Growing Influence of Clinical Practice Guidelines
- First Online:
- Cite this article as:
- O’Malley, A.S., Pham, H.H. & Reschovsky, J.D. J GEN INTERN MED (2007) 22: 742. doi:10.1007/s11606-007-0155-y
- 42 Views
Despite the proliferation of clinical practice guidelines (CPGs), physicians have been slow to adopt them.
Describe changes in the reported effect of CPGs on physicians’ clinical practice over the past decade, and identify the practice characteristics associated with those changes.
Design and Participants
Longitudinal and cross-sectional analyses of rounds 1–4 of the Community Tracking Study Physician Survey, a nationally representative survey, conducted periodically between 1996 and 2005.
The cross-sectional outcome was the reported effect of CPGs on the physician’s practice (very large, large, moderate, small, very small, and no effect). The longitudinal outcome was the change in reported effect of CPGs between two consecutive rounds for panel respondents. Independent variables included changes in physicians’ practice characteristics (size, ownership, capitation, availability of information technology (IT) to access guidelines, whether quality measures and profiling affect compensation, and revenue sources).
The proportion of primary care physicians reporting that CPGs had a very large or large effect on their practice increased significantly from 1997 to 2005, from 16.4% to 38.7% (P < .0001). The corresponding change for specialists was 18.9% to 28.2% (P < .0001). In longitudinal multivariate analyses, practice characteristics associated with an increase in effect of CPGs included acquiring IT to access guidelines, an increase in the impact that quality measures and profiling have on compensation, and an increase in the proportion of practice revenue under capitation or derived from Medicaid.
Promotion of wider adoption of health IT, and financial incentives linked to validated quality measures, may facilitate further growth in the impact of CPGs.
KEY WORDSpractice guidelinesfinancial incentivesquality
The past two decades have seen a proliferation of clinical practice guidelines (CPGs) development and dissemination ( http://www.guideline.gov; http://mdm.ca/cpgsnew/cpgs/index.asp); yet, uptake of CPGs among practicing physicians has been slow.1–3 As CPGs are a resource that provide physicians with information on evidence-based management that may improve quality of care, it is useful to better understand which factors are associated with their uptake.
Prior studies have examined barriers to physicians’ adherence to CPGs, such as their knowledge and attitudes about CPGs.4–5 There have also been numerous studies of interventions to improve uptake of individual guidelines among particular subgroups of physicians.6–7 The extent to which changes in modifiable characteristics of physicians’ practices predict changes in CPG uptake across all types of physicians has received less attention.
We examined whether the reported effect of CPGs has changed over time, and the practice characteristics that might be associated with such change. We hypothesized that having access to practice infrastructure to facilitate the navigation of evidence-based information, as well as financial incentives to meet quality standards, would increase adoption of CPGs. Physicians have an enormous amount of medical evidence that they must navigate and digest, a task that is greatly facilitated by information technology (IT). In addition, prior work has demonstrated that performance-based financial incentives, as well as the very act of measurement, likely increase physicians’ awareness of the processes of care being measured.8
We analyzed data from all four rounds of the Community Tracking Study (CTS) Physician Survey, a telephone survey of nationally representative samples of U.S. physicians conducted in 1996–1997, 1998–1999, 2000–2001, and 2004–2005. The physician sample was drawn from the American Medical Association and the American Osteopathic Association master files and included active, nonfederal, office-based, and hospital-based physicians who spent at least 20 hours a week in direct patient care. Residents and fellows, as well as radiologists, anesthesiologists, and pathologists were excluded.
Rounds 1–3 each included approximately 12,000 physicians. Because of funding cuts, round 4 included a smaller, but more statistically efficient and still nationally representative sample of 6,628 physicians. Cross-sectional survey response rates ranged from 52% to 65%, which are relatively high for surveys of physicians, and the lowest of these (52% in round 4) has been demonstrated to not bias estimates relative to the highest response rate of 65% from round 1.9
The second, third, and fourth rounds of the survey included physicians sampled in the previous round as well as new physicians. This approach resulted in three panels of physicians, spanning two consecutive rounds each, allowing us to track changes in responses from individual physicians between rounds. To create the first panel, 9,353 of the 12,385 respondents (76%) to the first round were contacted in the second round. Of these, a total of 915 had become ineligible because they had retired, worked fewer than 20 hours per week, or were not locatable. Of these, 7,092 completed the round 2 survey. After excluding those physicians whose specialty changed between rounds or who had missing data for key variables, there were 7,057 remaining in panel 1 for our analysis. Applying a similar process in later rounds resulted in 8,487 physicians in panel 2 (75% response rate) and 4,401 physicians in panel 3 (76% response rate). Item nonresponse for each round was typically less than 3%, and less than 0.1% for our dependent variable. Further details about the survey have been previously published and are available at http://www.hschange.org/index.cgi?data=04.
Our outcome was the reported effect of CPGs on the physician’s practice of medicine. In each round of the survey, physicians were asked: “How large an effect does your use of formal, written practice guidelines such as those generated by physician organizations, insurance companies or HMOs, or government agencies have on your practice of medicine? Would you say that the effect is very large, large, moderate, small, very small, or no effect at all?” These responses were coded on a scale from 6 to 1. (If a physician said that he/she uses his/her own guidelines, then the interviewer would say: “In this question, we are only interested in the use of formal, written guidelines such as those generated by physician organizations, insurance companies or HMOs, or other such groups.)
We analyzed the original six-category variable for cross-sectional analyses. For longitudinal analyses, the outcome variable is the “change in effect of guidelines” from one round to the next. We calculated values for this change for panel respondents by taking the difference in reported effect of guidelines for the same respondent in consecutive rounds of the survey, with the resulting value ranging from −5 to +5.
We evaluated the extent to which changes in physicians’ practice characteristics were associated with the effect that they reported CPGs had on the care they provided. We considered four types of factors that prior studies have suggested play a role in guideline adoption: (1) the practice environment, (2) relevant IT, (3) exposure to financial incentives based on performance, and (4) revenue sources.
Practice environment included practice size (<10 vs 10 or more physicians), practice ownership (physician is full or part owner of practice vs an employee), and percentage of practice revenue paid on a capitated or other prepaid basis. The question on IT addressed the availability of a computer or other forms of IT in the practice to access CPGs. We also asked physicians whether “specific measures of quality of care, such as rates of preventive care services delivered,” and “results of practice profiling comparing their pattern of using medical resources to treat patients with that of other physicians” were explicitly considered in determining their compensation. Response options to both compensation items were yes/no.
Finally, respondents indicated the percentage of their practice revenue derived from Medicaid and Medicare. Because managed care penetration varies for Medicaid and Medicare populations, we felt it was important to control for these types of insurance in the multivariate models. In 2004, approximately 12% of Medicare beneficiaries were enrolled in managed care plans versus 60% of Medicaid enrollees in managed care (http://www.cms.hhs.gov/apps/media/press/release.asp?Counter=1783; http://www.cms.hhs.gov/MedicaidManagCare/). Changes in the proportion of revenue from these sources might alter the proportion of one’s patient panel under managed care processes and hence influence a practitioner’s need to be responsive to CPGs. Whereas the CTS survey has items on specific practice type (physician group practice, hospital practice, group model HMO, etc.) there was little movement of physicians between types of settings from one round to the next, and thus too little variation for this characteristic to be an informative independent variable.
We defined independent variables using a set of dummy variables for each practice characteristic, reflecting changes in responses to the relevant survey question between two consecutive rounds, for individual physicians in each panel. For continuous variables (e.g., percentage of revenue from Medicaid), the difference in percentage was calculated. If the difference from the earlier to later round exceeded 5 percentage points, it was categorized as either an increase or decrease depending on its direction; otherwise, the value was categorized as no change. For categorical variables (e.g., availability of IT to access guidelines, Yes/No) the change from earlier to subsequent round was noted as, either an increase (if one gained IT access), a decrease (if one lost IT access), or no change (if one’s IT access remained the same) from the first round of the panel to the next.
We also characterized physicians in cross-sectional analyses in terms of their age, sex, years in practice, board certification status, medical education (United States/Canada vs elsewhere), and primary specialty. When we distinguish primary care physicians (PCPs) from specialists, the PCP category includes general internal medicine, family practice, general practice, geriatrics, and pediatricians. Specialists include medical or surgical specialists.
Because PCPs and specialists may operate under different practice characteristics, face different degrees of pay-for-performance pressures, and be exposed to different numbers of CPGs,10 we stratify analyses by PCP versus specialists. Percentages were weighted to be nationally representative and to account for the complex sample design.
We used the panel data to construct a first differences model. This allowed us to examine whether changes in practice characteristics were associated with changes in the reported effect of CPGs on physicians’ clinical practice over time. These longitudinal analyses have the advantage over cross-sectional analyses of avoiding bias because unobserved physician preferences and characteristics are held constant, with each physician in effect acting as his or her own control. We also included baseline ratings of the “effect of CPGs on one’s clinical practice” in the regression model to control for possible bias because of floor and ceiling effects, and the possibility that different physicians may tend to rate the CPG effect more or less highly under similar conditions. Similar baseline indicators were used for the IT variable and the profiling variable to account for the presence of those characteristics at baseline. (A physician who already had IT in the earlier round and kept it in the latter round would be classified as no change as would a physician who lacked IT in both rounds. The inclusion of the baseline indicator variable accounts for the fact that these two subgroups with no change are not equivalent.)
Because bivariate relationships between the independent and dependent variables were consistent across the three panels, and to maximize statistical power, we combined all three panels in multivariate analyses. Multivariate ordinary least squares regression allowed us to examine the independent effect of change in each practice characteristic, holding all other factors constant. Whereas we were not powered to run separate change models for PCPs versus specialists, we did include a dummy variable to indicate whether the respondent was a PCP or a specialist. We also controlled for secular trends by including dummy variables indicating each panel. All analyses were conducted using SAS version 9.1 (SAS Institute Inc, Cary, NC, USA) and SUDAAN version 9.0 analytic software (Research Triangle Park, NC, USA), which used the appropriate weights and accounted for the complex sample design as well as for the nonindependence of observations when the same physician was included in multiple panels.
Trends in Reported Effect of CPGs on Practice
Characteristics of the Physician Sample
Primary care physicians, n (%)*
Specialists, n (%)*
1996–1997 (n = 7,210)
1998–1999 (n = 7,264)
2000–2001 (n = 7,673)
2004–2005 (n = 3,291)
1996–1997 (n = 5,318)
1998–1999 (n = 5,040)
2000–2001 (n = 4,733)
2004–2005 (n = 3,337)
Median age (SE)
Years in practice
General internal medicine
US or Canada
Changes in Effect of Clinical Practice Guidelines (CPGs) Between 1997–1999, 1999–2001, and 2001–2005
Change in score
1997–1999 (Panel 1), % (SE)
1999–2001 (Panel 2), % (SE)
2001–2005 (Panel 3), % (SE)
PCPs (n = 4,025)
Specialists (n = 3,032)
PCPs (n = 5,104)
Specialists (n = 3,383)
PCPs (n = 2,352)
Specialists (n = 2,049)
Predictors of Increased CPG Effect on Clinical Practice
Multivariate Predictors of Changes in Physicians’ Report of the Effect of Clinical Practice Guidelines (CPGs) on Their Clinical Practice, 1997–2005
Changes in practice environment
Moved from smaller to larger (small is <10 physicians)
Moved from larger to smaller
Owner to nonowner
Nonowner to owner (owner includes part-owner)
Increased percentage of practice revenue under capitation
Decreased percentage of practice revenue under capitation
Change in IT access
IT to access guidelines
Gained IT as a means for accessing guidelines
Lost access to IT for accessing guidelines
Changes in whether quality monitoring affects compensation
Quality measures affect compensation
Quality measures began to affect compensation
Quality measures no longer affected compensation
Profiling affects compensation
Profiling began to affect compensation
Profiling no longer affected compensation
Changes in revenue sources
Proportion of revenue from Medicaid
Increased percentage of Medicaid revenue
Decreased percentage of Medicaid revenue
Proportion of revenue from Medicare
Increased percentage of Medicare revenue
Decreased percentage of Medicare revenue
Specialist (reference group is primary care generalists)
Covariate to adjust for secular trends
Panel 1 (reference group)
An increase in the percentage of a practice’s capitated revenue was associated with an increase in the reported effect of CPGs (P < .001). A much smaller but significant association was present between a decreased percentage of capitated revenue and an increase in the reported effect of CPGs. Gaining IT as a means for accessing guidelines was also associated with an increase in reported effect of CPGs (P < .001).
Physicians whose compensation was affected by performance on quality measures experienced an increase in effect of CPGs on their clinical practice (P = 012). Physicians for whom profiling began to be used to determine part of one’s compensation also had an increase in the reported effect of CPGs (P < .001). Consistent with this, a decreased impact of profiling on compensation was associated with a decreased reported effect of CPGs (P < .001).
An increase in the percentage of revenue from Medicaid was associated with an increase in the effect of CPGs (P < .001). Changes in proportion of revenue from Medicare were not associated with CPG effect. We also observed a significant secular trend in the increase in CPG effect over time (Table 3).
The proportion of physicians reporting that CPGs had a substantial effect on their practice has increased significantly in recent years, likely reflecting the proliferation of new guidelines (http://www.guideline.gov; http://mdm.ca/cpgsnew/cpgs/index.asp) and increasing pressures on physicians to invest in infrastructure and activities to systematically improve the quality of care. We found that PCPs were more likely than medical or surgical specialists to report that CPGs had a substantial effect on their practice. We observed, however, that the strongest associations with increased impact of guidelines were with practice-level factors. Specifically, increased adoption of IT to access guidelines, greater exposure to capitation, exposure to quality performance and profiling-based compensation, and a higher proportion of practice revenue derived from Medicaid were all associated with an increased effect of CPGs.
Primary care physicians were more likely than medical or surgical specialists to report that CPGs had a substantial effect on their practice. This difference may be due in part to the larger number of CPGs relevant to primary care, which encompasses a broad range of acute and chronic conditions and preventive services.10 For similar reasons, PCPs have had greater exposure than specialists to quality measurement and performance-based incentives, which we found were also associated with reporting a high impact of CPGs.11
An increase in the percentage of practice revenue under capitation was associated with an increase in the reported effect of CPGs. Whereas the percentage of physicians in practices having capitated contracts with health plans dropped from 62% to 50% between 1999 and 2001; since 2001, the use of capitated contracts has remained steady.11 Capitation may be associated with reports of a substantial effect of CPGs because health plans using capitation have for years also imposed quality-based reporting and incentives to counteract the anti-quality incentives to reduce services to patients. In addition, practices must achieve a certain size and organizational capacity to assume risk. This larger size involves the capacity to measure the performance of physicians, making it relatively simple to compare physicians’ treatment patterns with CPGs.
We also found that physicians who were newly exposed to compensation based on performance on quality measures and profiling were more likely to report increases in the effect of CPGs. This finding may reflect either the motivating influence of financial incentives to more fully adopt care recommendations, or the isolated effect of quality measurement, which in itself may increase their awareness of, and attention to, guidelines.12,13 From 2001 to 2005, the proportion of physicians in group practice whose compensation was based in part on quality measures increased from 17% to 20%. Financial incentives tied to physicians’ individual productivity are much more prevalent and have consistently affected 70% of physicians in non-solo practice since 1996.11
There were two seemingly counterintuitive findings that deserve comment. We found that decreased exposure to capitation and (although only marginally significant) the loss of quality measures as a factor used in their compensation were associated with a small increase in effect of CPGs. These results suggest the possibility that once in an environment that promotes the use of CPGs, physicians grow to accept their validity and usefulness, such that the influence of CPGs on their treatment patterns continue to grow even in the absence of external incentives.
The observed association between increases in the proportion of revenue from Medicaid and increases in the reported effect of CPGs is intuitive; it likely reflects the confluence of several trends affecting Medicaid patients. In particular, the heavy penetration of managed care plans in Medicaid, and efforts by the Centers for Medicare and Medicaid Services have both promoted adoption of CPGs and performance measurement. During the time period we studied, care for Medicaid patients became increasingly concentrated in larger groups and institution-based practices.14 Such providers may have more resources and infrastructure to support incorporation of guidelines into routine care through IT and quality improvement efforts.15
We identified a consistent association over time between gaining IT access to CPGs and an increase in the reported effect of CPGs on one’s clinical practice. The use of IT to access guidelines increased among all physician specialties and practice types between 2001 and 2005.16 Given the multitude of scientific studies that physicians must navigate, having computerized access to guidelines in the office setting may help them to synthesize the evidence base.
We acknowledge several study limitations. Physicians tend to overreport their adherence to CPGs;17 however, it is unlikely that there was a differential bias in this among individual physicians across different survey rounds or categories of practice characteristics. And we cannot ascertain whether physicians’ reports that CPGs have a substantial effect on their practice necessarily indicate that they have adopted the recommendations. It is also important to be mindful that current quality and cost measurement techniques, such as profiling, fail to capture many aspects of quality of medical care.18 Much of what physicians do is neither observed nor easily measurable; and, in the care of older patients with multiple comorbidities, strict adherence to CPGs may not always be appropriate.18–22 Nonetheless, CPGs are an important tool. A recent study of physician leaders in large multispecialty medical groups identified the use of evidence-based medicine as one of the most important organizational factors for quality improvement.23
Despite the recent expansion in the proportion of U.S. physicians who report that CPGs affect the care they provide, a substantial minority still report that CPGs have little to no impact on their care. Our findings suggest that promotion of wider adoption of health IT and financial incentives tied to valid measures of clinical and cost performance may facilitate further growth in the impact of formal guidelines as an important lever for improving the quality of patient care.
The CTS Physician Survey was supported by a grant from the Robert Wood Johnson Foundation to the Center for Studying Health System Change. Thanks to Michelle Banker for her assistance with MEDLINE searching and table preparation. Thanks as well to the anonymous reviewers whose comments contributed to the revised version of this manuscript.
Conflict of interest statement