INTRODUCTION

In 2007, the Centers for Medicare and Medicaid Services’ (CMS) introduced Physician Quality Reporting Initiative (PQRI), now known as Physician Quality and Reporting System (PQRS). The purpose of PQRS was to give providers “the opportunity to assess the quality of care they provide to their patients, helping to ensure that patients get the right care at the right time,” and also allow providers to “quantify how often they are meeting a particular quality metric.”1 Starting in 2011, the program provided incentives for reporting, but this year it began requiring all providers who treat Medicare patients to report quality measures or incur a 1.5 % penalty from standard reimbursement. Payment penalties are allotted based on whether or not the provider reported measures 2 years prior—so penalties in 2015 are based on reporting in 2013 and penalties in 2017 are based on reporting in 2015.1 This change represents a monumental step for the American healthcare system because it makes reporting mandatory by one of the largest payers in healthcare. It also represents a transition from a fee-for-service system to a pay-for-performance one, or what the federal government calls the value-based purchasing model—in other words, with PQRS tying performance measurement to Medicare reimbursement, CMS is making quality of care a national priority and a priority that likely will be present for a long time. Furthermore, PQRS’s impact will likely be even greater because its data will be used to implement value-based purchasing programs, and the commercial sector is likely to model these CMS payment reforms in the next few years.

It is important to note that CMS makes all decisions about the quality measures included in PQRS. CMS chooses the measures from different quality measure lists created by many different measure developers, including the National Committee for Quality Assurance (NCQA), American Medical Association’s Physician Consortium for Performance Improvement (AMA-PCPI), specialty societies (like American Board of Internal Medicine and Society of Thoracic Surgeons), Health Resources and Services Administration (HRSA), and CMS itself.1 CMS also chooses the number of measures and diversity of specialties to which measures apply, both of which have increased as PQRS’s mission has broadened since its inception.1 Thus, CMS is ultimately in control of the types of measures and specialties that PQRS assesses.

As a whole, quality measurement is a controversial topic, with many physicians and researchers expressing criticism of quality measurement programs. They believe that they create additional administrative costs and are thus expensive to participate in.2–7 They believe that they do not lead to quality improvement.4,8,9 Nearly 84 % of physicians do not believe that Medicare quality reporting programs enhance their ability to provide patients with high quality care. Over half of physicians are extremely concerned that measures do not apply to specialty care.9

However, there are no studies systematically analyzing PQRS measures themselves. Given the fact that Medicare is the largest healthcare payer and that PQRS affects thousands of physicians and millions of patients, understanding the type of measures that CMS has chosen to use in PQRS and how this selection has changed over time is essential, because this gives an understanding of the different measures that providers can then choose from to sufficiently report to PQRS. This information is important for physicians, policymakers, and patients to better understand how quality of care is currently being measured, where gaps in measurement exist, and thus, how measurement can be improved in the future. In this study, we analyzed the 2011 and 2015 PQRS measures to characterize the kinds of measures used, which is reflective of the physicians and patient disease populations being assessed.

METHODS

Data Source

We obtained the 2011 and 2015 PQRS measures using publicly available data from the CMS website.1 Data on this website include instructions for how providers can implement PQRS and measure lists for all years since 2011. Measure lists describe significant information about each measure, including a detailed measure description, the measure developer, and reporting options for each measure (claims, registry, electronic health record, etc.) as well as each measure’s National Quality Strategy (NQS) priority category, starting in 2013.1 The 2011 PQRS measure list is the earliest list available on the CMS website, with 2011 being the year during which incentives for reporting began. The 2015 PQRS measure list was the most current list at the time that this analysis was conducted.

Measure Categorization

We categorized measures according to specialty, type, and NQS category. For specialty, measures were categorized into five groups: internal medicine, pediatrics, obstetrics/gynecology, surgery, and other specialties. These categories were selected by the authors to be broad and comprehensive enough to give meaningful information about the physician specialties that PQRS measures. Specialties were designated as a subspecialty of internal medicine according to the American Board of Internal Medicine’s classifications.10 Measures were classified as other if they did not fit in the category of internal medicine, pediatrics, obstetrics/gynecology or surgery, and included measures for specialties like radiology, pathology, neurology, and psychiatry. Measures could apply to multiple specialties and were documented accordingly.

We also categorized the type of measure as structure, process, outcome or other according to the Donabedian model for assessing healthcare quality.11 Measures were already designated into these categories by CMS in the 2015 list. For this categorization, structure measures describe the characteristics of the health care facility, such as regulations about the physical plant or certifications, process measures describe the activities done in a health care facility like tests ordered or treatments prescribed, and outcome measure describe the results achieved after receiving treatment from a healthcare facility, such as having high blood pressure controlled to a certain level.

Finally, we analyzed measures according to the six NQS domains as designated by CMS: effective clinical care, patient safety, communication and care coordination, person- and caregiver-centered experience and outcomes, efficiency and cost reduction and community/population health.12 Since NQS categorization was not already provided by CMS, we categorized the 2011 list, designating measures with the same categorization as 2015 when they were the same both years. Table 1 shows ten example PQRS measures and how they were coded according to specialty, type, and NQS category.

Table 1. Sample Coding of Select PQRS Measures
Full size table

The first author (B.F.) coded all of the measures and was the primary reviewer. The second author (T.B.) reviewed all of the coding. When there was disagreement about the measure’s coding designations, the two authors discussed the measure to come to a consensus.

Analysis

We calculated the percentage of measures in each category (specialty categories, Donabedian types, and NQS domains). We also examined the percentage of structure, process, and outcome measures and the percentage of each NQS category within each specialty category. We performed a longitudinal analysis comparing measure characteristics (specialty, Donabedian type, and NQS category) from 2011 to 2015 and used the Pearson Chi-squared test to test for significant changes in the characteristics from 2011 to 2015. For the 2015 list, we used the Pearson Chi-squared test to compare specialty differences in the measure type and NQS category. All analyses were performed using Stata software, version 13 (StataCorp).

RESULTS

The 2015 PQRS includes 254 performance measures with 146 (57.5 %) assessing internal medicine specialties, 60 (23.6 %) assessing surgery, 14 (5.5 %) assessing obstetrics/gynecology, 30 (11.8 %) assessing pediatrics, and 75 (29.5 %) assessing other specialties (Table 2). When we categorized measures by type, we found that ten (3.9 %) assess structures, 169 (66.5 %) assess processes, 74 (29.1 %) assess outcomes, and one (0.4 %) assesses other measures. For the NQS priorities, 138 (54.3 %) focus on effective clinical care, 34 (13.4 %) on patient safety, and 37 (14.6 %) on communication and care coordination. The three other NQS priorities (14 person- and caregiver-centered experiences and outcomes, 16 efficiency and cost reduction and 15 community/population health) together account for 17.7 % of measures (Table 2).

Table 2. PQRS Measure Characteristics, 2011 and 2015
Full size table

The 2011 PQRS includes 198 performance measures, where 126 (63.6 %) assess internal medicine, 36 (18.2 %) assess surgery, ten (5.1 %) assess obstetrics/gynecology, 23 (11.6 %) assess pediatrics, and 55 (27.8 %) assess other specialties. When categorized by measure type, four (2.0 %) assess structures, 169 (85.4 %) assess processes, 25 (12.6 %) assess outcomes and zero (0 %) assess other measures. For the NQS priorities, 129 (65.2 %) focus on effective clinical care, 18 (9.1 %) on patient safety, and 28 (14.1 %) on communication and care coordination. The three other NQS priorities (six person- and caregiver-centered experiences and outcomes, seven efficiency and cost reduction and ten community/population health) together account for 11.6 % of measures (Table 2).

Compared with 2011, the 2015 measures had no significant specialty differences, with the majority of measures applying to internal medicine (63.6 % vs. 57.5 %). The type of measures changed significantly between 2011 and 2015, with fewer process measures (85.4 % vs. 66.5 %, p < 0.001) and more outcome measures (12.6 % vs. 29.1 %, p < 0.001) in 2015 versus 2011. There was no significant change in NQS measure categories between 2011 and 2015 (Table 2). When we looked at subcategories of measure types and NQS categories within each specialty in 2015, we found statistically significant differences in the measure type: surgery had the highest percentage of outcomes measures (61.1 %), whereas only 21.7 % of internal medicine measures were outcomes, and obstetrics/gynecology measures had 5.9 % outcome measure with only one outcome measure (Table 3). For NQS categories, internal medicine had the highest percentage of effective clinical care measures (68.5 %), whereas only 22.2 % of surgery measures were for effective clinical care. Surgery had the highest percentage of patient safety (31.9 %) and communication and care coordination measures (27.8 %) compared with internal medicine (5.4 % and 6.5 %) (Table 3).

Table 3 Characteristics of Measures by Specialty Type, 2015
Full size table

DISCUSSION

Our analysis demonstrates both progress and room for improvement in CMS’s PQRS performance measure selection. The 2015 measures represent progress because they assess quality for many specialties, particularly those that have been lacking measures in earlier PQRS lists or lists by other organizations.4,7,13,14 Approximately half of the measures assess surgery, pediatrics, obstetrics/gynecology or other specialties; the other half assess internal medicine. The specialty distribution is more even in the 2015 measure list, compared with that of 2011. Nonetheless, the fact that half of the measures are still considered applicable to internal medicine seems to support the common criticism that quality measurement efforts disproportionally assess primary care. A closer examination of the internal medicine measures demonstrates that they assess many internal medicine subspecialties. The 2015 list contains 56 more measures than 2011 and many additional measures for diseases like inflammatory bowel disease, rheumatoid arthritis, hepatitis, adult sinusitis, sleep apnea, and dementia.1,13 By including these different diseases, the PQRS list applies to many more specialties, even subspecialties within internal medicine, than lists used by private payers such as the Healthcare Effectiveness Data and Information Set (HEDIS).14

Notably, CMS included a sizeable amount of outcomes in the 2015 measure list. This is evident especially compared to 2011 and more specifically for surgery. This is significant because outcome measures often directly measure events with the clearest importance in clinical management and treatment. For example, patients and physicians alike are most interested in preserving quality of life, survival after serious conditions, avoiding hospital readmissions, and having positive medical experiences, in general.4,15

Despite this progress, our analysis demonstrates that CMS still needs to address some key aspects of quality with its PQRS measures. The current measures do not evenly address all NQS domains, focusing on clinical effectiveness with little emphasis on patient-centeredness, efficiency, and population health. This representation has remained relatively unchanged between the 2011 and 2015 lists. This is significant because the six NQS priorities appear to be CMS’s means of addressing the Institute of Medicine’s (IOM) six aims for the twenty-first century health care system as enumerated in the 2001 Crossing the Quality Chasm report.16 Many would argue that the US healthcare system still must adequately address these aims. Although different specialties may necessitate measurement predominantly by one NQS priority over another, there are certainly issues related to each priority that should be measured and that providers should strive to improve upon. CMS must continue to develop measures that address the three least represented NQS priorities for each of the major medical specialties. As an evolving program, with wide-reaching effects, PQRS has the potential to have a significant impact on better achieving all six NQS priorities more evenly.

The measures also lack outcomes for non-surgical specialties. Predominant use of process measures is problematic because studies show that only some processes correspond with meaningful patient outcomes.15,17

CMS must embrace its role as a leader in quality measure development and start designing outcome measures for PQRS specifically for the medical specialties that lack them. CMS also might consider collaborating specifically with additional medical specialty societies, in addition to its other measure developing collaborators, to advance measure development for medical subspecialties and non-internal medicine specialties. For internal medicine, CMS or its measure-developing collaborators might consider developing outcome measures around diagnostic quality and morbidity/mortality from chronic diseases. For obstetrics/gynecology, CMS might be really innovative, because other measure endorsers and developers like the National Quality Forum (NQF) and American Medical Association’s Physician Consortium for Performance Improvement (AMA-PCPI) either do not list obstetrics/gynecology outcomes or solely measure pregnancy care.18 Outcomes related to menopause management and hysterectomy, for example, might matter to many patients affected by PQRS (i.e., older adults). Although measure development will not be easy, CMS must use its broad relevance to many medical specialties through PQRS to be an innovator in the quality measurement field.

Undoubtedly, issues will arise transitioning to more quality measurement, particularly increased outcomes. They require more complicated risk adjustment, sufficient sample size, and carry the risk of surveillance bias.17 Another concerning issue would be incentivizing providers to avoid high-risk patients, a concern that 82 % of physicians believe could be a real consequence of performance measurement and reporting.19,20 As physicians become more rewarded for their performance, they may start to only treat patients who promise good outcomes, such as young healthy patients with the greatest access to care.21,22 This unintended consequence must be kept in mind when designing and implementing outcomes measures more broadly. It might require additional adjustments or incentives for physicians serving high-risk populations.

CMS must be heralded for its accomplishments in continuing to develop improved PQRS measures, while still being encouraged to fine-tune its approach to better measure quality. Moving forward, it will be important to understand whether increasing numbers and scope of measures have effects on physicians and patients. Nonetheless, in the next few years, due to its linkage of reimbursement and measure reporting, PQRS has great potential to provide an important impetus and opportunity for CMS to better meet national quality measurement needs and ultimately ensure long-term improved quality of care for all Americans.