Abstract
Can administrative claims data, Z, serve as a surrogate for better clinical data, X, when assessing institutional performance? We consider an analysis of I hospitals, each of which involves an adjusted outcome. In the i th hospital, we denote the true association between the outcome and the risk factors using one data source by θ i (X), the true association between the outcome and the risk factors using the other data source by γ i (Z), and assume we have estimates of each available. Within hospital i, the estimated association parameters are jointly normally distributed such that conditional on γ i (Z), a simple linear relationship exists between θ i (X) and γ i (Z). Methods are illustrated using mortality rates for 181,032 elderly US heart attack patients treated at 4322 hospitals. We find a strong linear relationship between the hospital standardized mortality rates adjusted by risk factors found in administrative claims data and rates adjusted by risk factors found in medical charts (posterior mean [95% interval] for slope: 0.997 [0.965,1.028]). However, the absolute and relative differences between the two sets of rates increase as hospital volume increases. For typically-sized standard deviations of claims-based rates, there is reasonable certainty of quality problems when the hospital’s claims-based rate is 0.72 times or smaller than the national mean or 1.45 times or greater than the national mean. Fewer hospitals are classified as either low-mortality or high-mortality hospitals when using claims-based estimates compared to chart-based estimates.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abrams, K.R., Spiegelhalter, D., Myles, J.P.: Bayesian Approaches to Clinical Trials and Health Care. John Wiley & Sons Inc., New York, NY (2004)
Ash, A., Ellis, R., Pope, J., et al.: Using diagnoses to describe populations and predict costs. Health Care Financing Review 21, 7–28 (2000)
Austin, P.C., Tu, J.V.: Comparing clinical data with administrative data for producing acute myocardial infarction report cards. J.R.Statist. Soc. A 169, 115–126 (2006)
CMS Premier Hospital Quality Incentive Demonstration (2006). See http://www.cms.hhs.gov/HospitalQualityInits/35_HospitalPremier.asp#TopOfPage, last accessed March 3, 2006
Daniels, M.J., Hughes, M.D.: Meta-analysis for the evaluation of potential surrogate markers. Statist. Med. 16, 1965–1982 (1997)
Efron, B.: The Jackknife, the Bootstrap and Other Resampling Plans. Society for Industrial and Applied Mathematics, Vol. 38. Philadelphia, PA, (1982)
Gelman, A.: Prior distributions for variance parameters in hierarchical models. Bayesian Anal. 1, 1–19 (2005)
Gilks, W.R., Thomas, A., Spiegelhalter, D.J.: A language and program for complex Bayesian modelling. Statistician 43, 169–178 (1994)
Horton, N., Laird, N., Murphy, J., et al.: Multiple informants: Mortality associated with psychiatric disorders in the Stirling County Study. Am. J. Epidemiol. 154(7), 649–656 (2001)
Horton, N., Laird, N., Zahner, G.E.P.: Use of multiple informant data as a predictor in psychiatric epidemiology. Int. J. Psychiatric Res. 8, 6–18 (1999)
Jencks, S., Cuerdon, T., Burwen, D. et al. Quality of medical care delivered to Medicare beneficiaries: A profile of state and national levels. J. Am. Med. Assoc. 289(3), 305–312 (2003)
Kass, R.E., Wasserman, L.: A reference bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J. Am. Statist. Ass. 90(431), 928–934 (1995)
Krumholz, H., Chen, J., et al.: Comparing AMI mortality among hospitals among patients 65 years of age and older: evaluating methods of risk adjustment. Circulation, 2986–2992 (1999)
Krumholz, H., Wang, Y. et al.: Administrative claims model for profiling hospital performance for AMI. Circulation 113, 1683–1692 (2006)
Lee, K.L., Woodlief, L., Topol, E., et al.: Predictors of 30-day mortality in the era of reperfusion for acute myocardial infarction: Results from an international trial of 41,021 patients. Circulation 91, 1659–1668 (1995)
Molenberghs, G., Burzykowski, T., Alonso, A., et al. A perspective on surrogate endpoints in controlled clinical trials. Stat. Methods Med. Res. 13, 177–206 (2004)
Morris, C.N.: Parametric empirical Bayes inference. J. Am. Statist. Ass. 78:47–65 (1983)
Normand, S.-L., Glickman, M.E., Sharma, R.G., McNeil, B.J.: Using admission characteristics to predict short-term mortality from myocardial infarction in the elderly: Results from the Cooperative Cardiovascular Project. J. Am. Med. Assoc. 275(17), 1322–1328 (1996)
Normand, S.-L., Glickman, M.E., Gatsonis, C.: Statistical methods for profiling providers of medical care: Issues and applications. J. Am. Statist. Ass. 92, 803–814 (1997)
Pashos, C.L., Newhouse, J.P., McNeil, B.J.: Temporal changes in the care and outcomes of elderly patients with acute myocardial infarction, 1987 through 1990. J. Am. Med. Assoc. 270(15), 1832–1836 (1993)
Pepe, M.S., Whitaker, R.C., Seidel, K.: Estimating and comparing univariate associations with application to the prediction of adult obesity. Statist. Med. 18, 163–173 (1999)
Prentice, R.L.: Surrogate markers in clinical trials: Definition and operational criteria. Statist. Med. 8, 431–440 (1989)
Schmitz, R .: Medicare Quality Monitoring System (MQMS) Report: Acute Myocardial Infarction (AMI) 1992–2001. Mathematica Policy Research, Inc. Washington DC (2003)
United States Congress. One Hundred Sixth Congress of the United States of America, Amendment of Title IX of the Public Health Service Act, January 6, 1999
Acknowledgements
The authors thank Jennifer Mattera, Amy Rich, Yongfei Wang, Deron Galusha, Inyoung Kim; and the clinical experts (Jeptha Curtis, Robert McNamara, Mikhail Kosiborod). The analyses upon which this publication is based were performed under Contract Number 500-02-C001, entitled “Utilization and Quality Control Quality Improvement Organization for the State (commonwealth) of Colorado”, sponsored by the Centers for Medicare & Medicaid Services (formerly Health Care Financing Administration), Department of Health and Human Services. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US government. The authors assume full responsibility for the accuracy and completeness of the ideas presented. Dr. Normand’s effort was partially supported by Grant MH54693 from the National Institute of Mental Health.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Normand, SL.T., Wang, Y. & Krumholz, H.M. Assessing surrogacy of data sources for institutional comparisons. Health Serv Outcomes Res Method 7, 79–96 (2007). https://doi.org/10.1007/s10742-006-0018-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-006-0018-8