The QI education program and outcomes study were approved by an independent institutional review board (Sterling IRB, Atlanta, GA; IRB ID #4534). This article does not contain any new studies with human or animal subjects performed by any of the authors.
Physician Recruitment and Baseline Chart Selection
Twenty community-based rheumatologists were recruited to participate in the chart audits and educational activities. Given documented rheumatology workforce shortages in the United States, we sought to sample from states with adequate numbers of rheumatologists for the study. Using Centers for Disease Control and Prevention (CDC) surveillance data, we identified states with high ratios of rheumatologists to patients with arthritis (CDC data do not distinguish numbers of patients with rheumatoid arthritis or osteoarthritis). From the states with the highest ratios of rheumatologists to patients, we identified and recruited participants through internal or purchased lists of practicing rheumatologists, whom we contacted by postal mail, fax, or email. Rheumatologists were enrolled in the order of their expressed interest in the educational program and study. We aimed to recruit approximately equal numbers of rheumatologists from the Northeast, South, Midwest, and West.
The study was designed to review 160 baseline charts of adult patients (aged 18 years and older) who had a diagnosis of RA for at least 1 year (indicated by ICD-9 codes 714.0, 714.1, 714.2, or 714.81 from billing data) and at least 1 visit with the participating rheumatologist between 12/1/2012 and 11/30/2013. Administrative staff for each of the 20 rheumatologists selected an oversample of up to 12 charts that met inclusion criteria, with the goal of obtaining an average of 8 charts per rheumatologist. This number was determined partly by pragmatic considerations including the limited time commitment that the practices could devote to identifying charts and funding restrictions.
Eligible charts were selected by reviewing consecutive patients with the most recent office visits, working backward from the index date of 11/30/2013. In the baseline period, 3 practices provided fewer than the targeted 8 charts (n = 4, 6, and 7). The rheumatologists in these practices were enrolled in the educational program; thus, their charts were included in the analysis. To compensate for these practices to reach the targeted 160 charts for baseline review, we included 9 charts from 7 other practices. These practices were selected through a process that balanced the number of charts from the 4 geographical regions.
Each practice received a $500 administrative fee to reimburse costs for staff resources. This fee, which comprised a $250 resource allocation for each of the 2 chart abstraction periods (baseline and post-education), covered costs for identifying and pulling patient charts based on eligibility criteria, as well as coordinating with the chart abstractors.
Baseline Retrospective Chart Abstraction and Analysis
Charts that met inclusion criteria were retrospectively abstracted by 1 of 4 trained medical record reviewers. Paper charts were made available for review onsite, or they were copied and sent to the chart abstractors for offsite review. Electronic charts were accessed remotely or onsite based on the preference and capability of the practice. The reviewers completed their abstraction of baseline charts between December 2013 and February 2014. To assess inter-rater reliability, each reviewer compared samples of their colleague’s charts through an internal quality assurance process. The assessment was based on numbers of chart variables for which the reviewers agreed in their abstraction.
The charts were abstracted for patient demographics and the rheumatologists’ documented performance on the 6 quality measures for RA included in the 2013 and 2014 PQRS programs (Table 1). In addition, charts were abstracted for (1) documentation of patient counseling about medication benefits/risks and adherence, lifestyle modifications, and quality of life; (2) assessment of RA medication side effects; and (3) assessment of RA medication adherence. For the latter measure, charts were reviewed for whether adherence was assessed (yes or no) and for documentation of adherence status (adherent or nonadherent). These counseling and assessment measures are related to NQS priorities for ensuring that patients are engaged in their healthcare, improving communication, promoting effective prevention and treatment practices, or making care safer [10]. Through structured chart review, each rheumatologist’s performance on the measures was recorded for analysis in Statistical Package for the Social Sciences (SPSS, IBM Corporation, NY, USA), version 22.
Table 1 PQRS quality measures for RA
Educational Activities
After the baseline chart review, the rheumatologists participated in a series of educational activities that were accredited by the Accreditation Council for Continuing Medical Education. The first activity was an online audit-feedback session (45 min), which was presented individually to each rheumatologist by a medical chart review expert. During these sessions, each physician’s baseline rates of performance on the PQRS quality measures were presented and compared with the de-identified mean rates of the other 19 rheumatologists in the study. The sessions were designed to support participants in identifying areas for improvement, focusing especially on measures for which baseline performance rates were low. The presenter engaged the participant in discussing barriers to performing and documenting the quality measures, as well as in identifying strategies for improvement. In addition, the feedback addressed the rheumatologist’s documentation of the patient counseling measures as well as medication side effects and adherence.
Within 4 weeks of the audit-feedback activity, each rheumatologist participated in a 45-min webinar with 4 other peers in the cohort. The 5 small-group webinars were led by an expert rheumatologist who guided discussions of strategies for improving performance on RA quality measures. One of the co-authors of this article (E. Ruderman) served as faculty presenter for these webinars. The discussions addressed the evidence-based rationale for applying the quality measures in clinical practice; approaches to improving patient assessment, treatment, and management based on the measures; and strategies for appropriately documenting performance on the measures. To reinforce learning, the educational program also included a variety of online- and mobile-accessible accredited activities in an RA QI toolkit. These included a 10-page monograph that presented the evidence-based rationale for the quality measures as well as a 12-page monograph and a 30-min video addressing interprofessional approaches to achieving high standards for the quality of RA care. The 20 rheumatologists’ participation in the audit-feedback and small-group webinar activities was confirmed through roll call. For the 3 online and mobile-accessible activities, all of the rheumatologists self-reported their participation.
Post-Education Retrospective Chart Abstraction and Analyses
Six months after each rheumatologist completed the educational activities, follow-up chart audits (n = 160) were conducted according to the same methods described for the baseline reviews. In each practice, charts were identified for patients with RA who had at least 1 visit with their physician in the post-education period. The number of post-education charts was matched to each rheumatologist’s number of baseline charts. Between August and October 2014, the post-education charts were retrospectively abstracted for documentation of the PQRS quality measures and NQS-related clinical processes during the 6-month period following each rheumatologist’s completion of the educational activities.
Statistical Analysis
Using SPSS, Chi-square tests were performed to analyze the differences between baseline and post-education frequencies of chart documentation for the PQRS quality measures for RA and the additional measures for patient counseling and assessments of medication side effects and adherence. p values less than 0.05 were considered significant.