Advertisement

Journal of General Internal Medicine

, Volume 33, Issue 8, pp 1221–1223 | Cite as

Impact of Teaching Intensity and Sociodemographic Characteristics on CMS Hospital Compare Quality Ratings

  • Wenshuai WanEmail author
  • C. Jason Liang
  • Richard DuszakJr.
  • Christoph I. Lee
Concise Research Reports

KEY WORDS

health services research quality assessment Medicare sociodemographic factors risk adjustment 

INTRODUCTION

In July 2016, the Centers for Medicare & Medicaid Services (CMS) released “Hospital Compare” ratings for 3576 U.S. hospitals to help consumers compare hospitals’ quality of care, which was distilled into a single overall rating between 1 and 5 stars based on 57 process- and outcomes-based clinical measures.1,2 A recent publication found differences in overall ratings by characteristics including a hospital’s teaching status.3 Professional groups (e.g., the American Hospital Association) have expressed concern that the current ratings scheme unfairly penalizes teaching hospitals, especially those serving low-income and otherwise disadvantaged patient populations.4

Our objective was to build a statistical model that adjusted  for a more complete set of covariates than previous studies to analyze discrepancies in Hospital Compare ratings between teaching and non-teaching hospitals. Informed by previous findings, we attempted to more fully capture key aspects of an academic mission-driven hospital, including trainee development, research, and uncompensated care.3,5 We based our model on U.S. geographic healthcare markets to assess the potential impact of adjusting overall quality ratings for hospital characteristics, including their locations and the sociodemographic characteristics of their surrounding patient populations.

METHODS

Publicly available data sets were merged from CMS Hospital Compare, CMS Inpatient Prospective Payment System, and the U.S. Census. Children’s hospitals were excluded due to exemption from data reporting. Sociodemographic indicators were mapped to hospital location by health service area (HSA). Our study was granted exempt status by the Institutional Review Board of the Hospital of the University of Pennsylvania.

The relationships between Hospital Compare ratings and various predictors were analyzed by univariate and multivariate linear regression models using R, version 3.4.0. We performed sensitivity analysis using two propensity score matching methods (multivariate logistic regression and random forests) to account for non-linear and non-additive associations.

RESULTS

In a multivariate linear regression model adjusting for hospital characteristics, geography, and sociodemographic factors, each increase of 1 resident (trainee) per 10 beds, a continuous measure of teaching intensity, was associated with a 0.09 Hospital Compare star rating decrease (p < 0.01, Table 1). Figure 1 also shows the correlation between increasing teaching intensity, as captured by resident-to-bed ratio, and overall mean Hospital Compare rating.
Table 1

Association Between Hospital, Location, and Sociodemographic Characteristics and Hospital Compare Ratings

 

Univariate linear regressions

Multivariate linear regressiona

Coefficientb

Std Error

p value

Coefficient

Std Error

p value

Hospital level predictors

 Resident-to-bed ratio

− 1.390

0.089

< 0.001*

− 0.899

0.103

< 0.001*

 Emergency services

− 0.318

0.101

0.002*

− 0.243

0.089

0.006*

 Ownership

  Nonprofit

Reference

 

Reference

 

  Government

− 0.223

0.043

< 0.001*

− 0.116

0.040

0.004*

  Proprietary

− 0.235

0.039

< 0.001*

− 0.207

0.037

< 0.001*

 # beds

− 0.001

< 0.001

< 0.001*

< 0.001

< 0.001

0.477

 % occupied (inpatient)

− 0.008

0.001

< 0.001*

− 0.003

0.001

0.002*

 Locationc

  South Atlantic

Reference

 

Reference

 

  Mid Atlantic

− 0.251

0.057

< 0.001*

− 0.201

0.061

0.001*

  New England

0.315

0.081

< 0.001*

0.198

0.080

0.014*

  East North Central

0.435

0.052

< 0.001*

0.400

0.055

< 0.001*

  East South Central

0.063

0.061

0.300*

0.086

0.059

0.140

  West North Central

0.424

0.065

< 0.001*

0.138

0.064

0.032*

  West South Central

0.292

0.053

< 0.001*

0.203

0.055

< 0.001*

  Mountain

0.221

0.069

0.001*

0.116

0.075

0.124

  Pacific

0.034

0.056

0.544

0.157

0.072

0.029*

Sociodemographic predictorsd

 Race

  Non-Hispanic White

Reference

 

Reference

 

  African-American

− 0.012

0.001

< 0.001*

− 0.004

0.001

0.013*

  Hispanic

− 0.007

0.001

< 0.001*

− 0.005

0.006

0.408

  Asian

− 0.006

0.003

0.014*

0.006

0.003

0.042*

 Non-English speakinge

− 0.009

0.001

< 0.001*

0.007

0.006

0.287

 Age > 65

− 0.001

0.004

0.848

− 0.025

0.005

< 0.001*

 Education

  GED

− 0.003

0.002

0.177

< 0.001

0.005

0.925

  Bachelor’s

0.017

0.002

< 0.001*

0.026

0.006

< 0.001*

 Unemployment

− 0.126

0.011

< 0.001*

− 0.044

0.015

0.003*

 Poverty

  Individual

− 0.022

0.003

< 0.001*

− 0.019

0.014

0.190

  Household

− 0.021

0.003

< 0.001*

0.013

0.015

0.384

 Abandoned Homesf

− 0.003

0.003

0.327

0.005

0.003

0.098

*Statistically significant to p < 0.05

aAdjusted for all hospital characteristics, geography, and sociodemographic factors in Table 1

bCoefficients measured on Hospital Compare scale between 1 and 5 stars, i.e., 0.1 correlates with 0.1 higher overall quality star rating

cRegion categorized by Medicare Hospital Inpatient Prospective Payment System

dFrom 2010 U.S. Census (Summary File 1) and 2006–2010 American Community Survey by percent of population per health service area (HSA)

eMeasured based on Spanish only fluency in U.S. Census

fPercent of residential units without ownership, occupancy, or management for > 2 months annually

Fig. 1

Hospital compare ratings based on resident-to-bed ratio. Increased teaching intensity, represented by higher resident-to-bed ratios, was associated with a decreasing mean overall Hospital Compare rating (between 1 and 5 stars) at U.S. teaching hospitals.

Two propensity score sensitivity analyses, which matched teaching and non-teaching hospitals on all geographic and sociodemographic variables in Table 1 (except resident-to-bed ratio) produced similar results. With multivariate logistic regression and random forest propensity matching, teaching hospitals had 0.32 (95% CI, 0.24, 0.39) and 0.23 (95% CI, 0.16, 0.31) lower Hospital Compare ratings, respectively (out of a possible 5 stars).

Other non-teaching-related measures were also independently associated with significant differences in Hospital Compare ratings when all hospital and sociodemographic factors were included in a single, multivariate linear regression (Table 1). Hospitals offering emergency services were associated with lower ratings by 0.24 stars (p < 0.01). Location by U.S. region was highly associated with differences in hospital ratings. Compared to the largest cohort of hospitals (South Atlantic, n = 568), cohorts in the East North Central or Mid-Atlantic received up to 0.40 higher or 0.20 lower ratings, respectively (p < 0.01 for both).

In addition, HSA-level sociodemographic factors were independently associated with differences in hospital ratings (Table 1). Age, race, unemployment, and education (bachelor’s degree) of populations within a hospital’s HSA all contributed to variations in quality rating (p < 0.05 for race, others p < 0.01).

DISCUSSION

Our analysis suggests that an institution’s Hospital Compare score is strongly influenced by differences in hospital characteristics and sociodemographic factors. Even after adjusting for their location and surrounding patient populations, Hospital Compare ratings are lower for academic hospitals, and this rating penalty increases with higher teaching intensity. In addition, hospitals providing emergency services and those surrounded by a higher density of patients with sociodemographic risk factors also receive lower overall ratings.

While our study does not fully explain how higher teaching intensity, as measured by resident-to-bed ratio, is associated with poorer Hospital Compare ratings, it may be that academic hospitals are called upon to provide a larger spectrum of services or are receiving incomplete adjustment for clinical factors (e.g., disease severity or comorbidities). Policymakers must ensure that quality measurement, as embodied in Hospital Compare ratings, does not penalize academic hospitals for treating vulnerable and challenging patients, or for providing postgraduate medical education.

Notes

Funding Information

Wenshuai Wan, MD received funding during the research project from a Ruth L. Kirschstein National Research Service Award (T32).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.

References

  1. 1.
    Centers for Medicare & Medicaid Services. CMS Releases First Ever Hospital Compare Star Ratings. 2015 Apr 16. Available at: http://www.cms.gov/newsroom/mediareleasedatabase/press-releases/2015-press-releases-items/2015-04-16.html. Accessed 21 March 2018.
  2. 2.
    Centers for Medicare & Medicaid Services. First Release of the Overall Hospital Quality Star Rating on Hospital Compare. 2016 Jul 27. Available at: http://www.cms.gov/Newsroom/MediaReleaseDatabase/Fact-sheets/2016-Fact-sheets-items/2016-07-27.html. Accessed 21 March 2018.
  3. 3.
    DeLancey JO, Softcheck J, Chung JW. Associations Between Hospital Characteristics, Measure Reporting, and the Centers for Medicare & Medicaid Services Overall Hospital Quality Star Ratings. JAMA. 2017;317(19):2015–2017.CrossRefGoogle Scholar
  4. 4.
    Pollack R. Statement on Centers for Medicare and Medicaid Services (CMS) overall hospital star ratings. 2016 Jul 27. Available at: http://www.aha.org/presscenter/pressrel/2016/160727-pr-starratings2.shtml. Accessed 21 March 2018.
  5. 5.
    Fiscella K, Burstin HR, Nerenz DR. Quality measures and sociodemographic risk factors: to adjust or not to adjust. JAMA. 2014;312(24):2615–6.CrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2018

Authors and Affiliations

  • Wenshuai Wan
    • 1
    Email author
  • C. Jason Liang
    • 2
  • Richard DuszakJr.
    • 3
  • Christoph I. Lee
    • 4
    • 5
    • 6
  1. 1.Department of Radiology Hospital of the University of PennsylvaniaPhiladelphiaUSA
  2. 2.Biostatistics Research BranchNational Institute of Allergy and Infectious Diseases, National Institutes of HealthBethesdaUSA
  3. 3.Department of Radiology and Imaging SciencesEmory University School of MedicineAtlantaUSA
  4. 4.Department of RadiologyUniversity of Washington School of MedicineSeattleUSA
  5. 5.Department of Health ServicesUniversity of Washington School of Public HealthSeattleUSA
  6. 6.Hutchinson Institute for Cancer Outcomes ResearchSeattleUSA

Personalised recommendations