Influence of Continuing Medical Education on Rheumatologists’ Performance on National Quality Measures for Rheumatoid Arthritis

Over the past decade, research examining the quality of care provided to patients with rheumatoid arthritis (RA) has indicated notable deficits, including low or variable rates of guideline-directed prescription of disease-modifying antirheumatic drugs (DMARDs) and vaccinations [1, 2, 3, 4, 5]. Evidence suggesting gaps in care led to the development of tools to monitor and improve the quality of care for patients with RA [4, 6]. Through a collaborative project that began nearly a decade ago, the American College of Rheumatology (ACR) worked with the American Medical Association’s Physician Consortium for Performance Improvement to develop a set of 6 process-based quality measures for RA [6]. These measures are included in the Centers for Medicare and Medicaid Services (CMS) Physician Quality Reporting System (PQRS) [7]. The RA measures comprise DMARD prescription, tuberculosis (TB) screening within 6 months prior to initiation of a new biologic medication, disease activity assessment and classification, functional status assessment, prognosis assessment and classification, and glucocorticoid management.

Designed to improve the quality, accountability, and transparency of healthcare, the PQRS program originally provided incentive payments for eligible healthcare professionals (those who received payments under the Medicare Physician Fee Schedule) who met criteria for reporting quality measures. As of 2015, the program imposes increasing reimbursement penalties for eligible professionals who have not reported PQRS measure data according to CMS requirements [8].

The terminology, principles, and implementation methods of quality improvement (QI) have not historically been taught in United States undergraduate and graduate medical schools. For practicing physicians, most QI education and training programs are based in academic medical centers and large health systems [9]. In all settings, however, physicians are currently responding to new requirements for participation in QI programs and reporting quality measures for accountability and value-based payment incentives. To realize the potential for improving the quality of healthcare, continuing education may be a key strategy for addressing QI-related knowledge, attitudinal, and practice gaps. We developed and provided continuing education activities designed to support community-based rheumatologists in improving performance on PQRS quality measures for RA and additional measures for patient counseling and assessments of medication side effects and adherence, which are related to priorities of the National Quality Strategy (NQS) [10]. To assess the influence of the education, we conducted baseline and post-education chart audits and compared rates of documented performance on these measures.

The QI education program and outcomes study were approved by an independent institutional review board (Sterling IRB, Atlanta, GA; IRB ID #4534). This article does not contain any new studies with human or animal subjects performed by any of the authors.

The 20 participating rheumatologists were located in the Northeast (n = 6), South (n = 5), Midwest (n = 4), and West (n = 5). They reported treating an average of 41 patients with RA per week. The analysis included 160 baseline charts (mean = 8 per physician; range = 4–9) and 160 post-education charts (mean = 8 per physician; range = 4–9). The comparisons of samples of charts abstracted by the 4 different reviewers indicated agreement for at least 90% of chart variables.

As presented in Fig. 1, there were significantly higher mean rates of performance on 4 of the 6 PQRS quality measures for RA in the post-education versus baseline charts. The absolute percentage increases for these 4 measures were as follows: tuberculosis screening before biologic DMARD therapy (24%, p < 0.001); assessment and classification of disease activity (23%, p < 0.001); assessment of functional status (9%, p = 0.01); and assessment and classification of disease prognosis (23%, p = 0.005). At baseline, 99% of patient charts indicated prescription of DMARD therapy; adherence to this measure was not significantly different in the post-education charts. For the sixth PQRS measure, documentation of a glucocorticoid management plan, only 1 patient in the post-education sample met the eligibility requirement of prolonged high-dose glucocorticoid therapy; thus, analysis for this measure was precluded.

Open image in new window

For 2 of the 3 patient counseling measures and for assessments of side effects and medication adherence, there were significantly higher mean rates of post-education versus baseline chart documentation (Fig. 2). The absolute percentage increases for these 4 measures were as follows: counseling about medication (9%, p = 0.02); counseling about quality of life (17%, p = 0.01); assessment of medication side effects (22%, p < 0.001); and assessment of medication adherence (40%, p < 0.001). The proportion of charts indicating that patients were adherent to their medications was significantly greater in the post-education (82%) versus baseline (48%) review (p < 0.001). For counseling about lifestyle modifications, the mean percentage increase after education did not reach statistical significance (7%, p = 0.06).

Open image in new window

Previous reports of suboptimal performance on quality measures for RA have motivated leaders in the rheumatology community to call for programs to improve the quality of RA patient care [4, 6]. This pragmatic study provides preliminary evidence for a positive influence of accredited education on improving performance on PQRS quality measures for RA and on additional measures aligned with NQS priorities. To our knowledge, this is the first study to report on QI-focused educational interventions for improving adherence to these measures. In addition, the study is unique in providing data on performance rates for the full set of PQRS RA measures among community-based rheumatologists.

With the exception of DMARD treatment, baseline rates of documented performance on the PQRS measures were low to moderate, ranging from 18% for TB screening before initiating a biologic DMARD to 74% for functional status assessment. In an analysis of the 2009 ACR Rheumatology Clinical Registry (RCR), Kazi et al. reported considerably higher rates of adherence to 5 quality measures among 240 rheumatology providers who submitted data for 7806 patients with RA: assessment and classification of disease activity (100%), DMARD treatment (93%), TB screening (92%), assessment of functional status (79%), and assessment and classification of disease prognosis (78%) [11]. One explanation for the higher rates of adherence reported by Kazi et al. is that the providers were self-selected members of the RCR. In addition, one of the main goals of this registry is to give providers a mechanism with which to document and report PQRS measures. In an analysis of 2005–2008 HEDIS data for more than 90,000 RA patients enrolled in Medicare managed care plans, Schmajuk et al. found that 63% of the patients had received a DMARD [5]. The authors reported that DMARD receipt varied considerably across patient groups, with the lowest rates reported for older individuals, black patients, and patients with low socioeconomic status. The study focused on health system performance, thus reflecting DMARD use across both rheumatology and primary care practices. Recently, Desai et al. reported a study involving reviews of 438 charts of RA patients in an academic medical center rheumatology practice [12]. Assessment of disease activity and functional status were documented in 29% and 75% of the charts, respectively.

Our findings indicate that a program of quality-focused education was associated with significant improvements in documented performance on 4 of the 6 PQRS measures for RA and on additional measures involving patient counseling and assessments of medication side effects and adherence. For several measures the improvements were substantial. However, post-education performance rates indicate gaps and room for improvement for most of the measures, including assessment of disease prognosis (54%), assessment of disease activity (63%), TB screening before initiating a biologic DMARD (42%), counseling for lifestyle modifications (46%) and quality of life (68%), and assessment of medication adherence (65%).

Our observations of the rheumatologists’ discussions during the audit-feedback and small-group webinar sessions may offer insight into the suboptimal post-education performance rates. These discussions addressed the barriers and challenges the physicians face in aligning their practices with national quality measures. For example, some participants commented that the methods necessary to assess and classify disease activity and prognosis are time-consuming and more appropriate for clinical trials than for office visits. Regarding the measure of TB screening, a common response from participants was that their electronic health records lack structured fields for recording this measure. Several rheumatologists commented that, despite the lack of documentation in their charts, they always screen patients for TB before initiating biologic DMARD treatment. These responses reflect the need for adaptations of electronic medical records to efficiently collect key quality and safety data in a standardized manner.

Several limitations of the study should be considered in interpreting the results. Because the endpoints were process-based quality measures, the performance of the same physicians was assessed using different patient charts in the baseline and post-education audits. This design afforded some control over participant-related extraneous variables. However, conclusions regarding the direct effect of the educational interventions are limited by the lack of a control group of rheumatologists who did not participate in the educational activities and whose charts were audited over the same time periods. An interrupted time series trial with a control group would be a stronger design for more reliably assessing whether improvements were attributable to the education rather than to secular trends.

Due to the rheumatologists’ gaps in documenting patient characteristics, especially classifications of disease activity, we were not able to determine whether patients whose charts were audited in the baseline and post-education period were matched for variables that might have influenced quality of care. Other potentially confounding factors, which may influence quality of care and, therefore, should ideally be matched across samples, include number of patient visits and patients’ income, health literacy, and comorbidities. Another limitation is that the post-education follow-up period was only 6 months. A longer follow-up may have resulted in higher post-education rates of adherence to the quality measures; in contrast, it is possible that performance on quality measures may revert to baseline over time without continual reinforcement. Finally, we were not able to determine whether the changes observed between the baseline and follow-up period related directly to improving performance on these quality measures or to better chart documentation. However, chart documentation is a critical element of the ability to assess compliance with these measures. Moreover, documentation of these measures is essential for guiding care processes, including evaluating treatment effectiveness, informing treatment decisions, and providing essential information to promote patient safety.

This pragmatic study was not designed to determine the extent to which the different educational activities influenced performance on the quality measures. The continuing education literature has indicated that conventional formats, such as didactic lectures and print media, generally do not lead to persistent changes in complex physician behaviors [13, 14, 15, 16, 17]. In a meta-analysis of 140 studies on chart audit and feedback as an educational intervention for healthcare professionals, the authors concluded that this method can elicit small but meaningful improvements in clinical performance [18]. The greatest improvements occur when feedback is offered by a supervisor or respected colleague and accompanied by specific goals or action plans for quality improvement. We designed the audit-feedback sessions and small-group webinars accordingly.

As suggested by Saag et al., the connection between process-based RA quality measures and patient outcomes and, therefore, the rationale for aligning clinical practice with the measures can be established with evidence from clinical trials and well-designed observational studies [4]. The development of the PQRS quality measures for RA was strongly influenced by the 2008 ACR guidelines [6]. A recent study reported that rheumatologists’ documentation of PQRS measures for disease activity assessment and functional status was not significantly associated with 2-year radiographic progression [12]. However, as acknowledged by the authors, this study was limited partly because the rates of documenting the quality measures were low and RA outcome measures were not consistently defined. New studies are thus needed to understand relationships between process-based quality measures and patient outcomes. The results of these studies will guide revisions of current quality measures and the development of new ones, including outcomes-based measures of disease activity and function [19]. Evolving value-based care delivery models may pose some barriers associated with extra time demands for performing, documenting, and reporting quality measures. Thus, new QI programs and studies are needed to develop strategies for facilitating workflow to enable quality-driven care.

The present study provides preliminary evidence for the potential for QI-focused education to help rheumatologists align their practices with evidence-based and consensus quality measures. Additional research will be necessary to identify and optimize educational interventions that yield significant and sustainable improvements in the quality of care for patients with RA.