Journal of General Internal Medicine

, Volume 19, Issue 5, pp 444–450 | Cite as

Consistency of performance ranking of comorbidity adjustment scores in canadian and U.S. utilization data

  • Sebastian Schneeweiss
  • Philip S. Wang
  • Jerry Avorn
  • Malcolm Maclure
  • Raia Levin
  • Robert J. Glynn
Original Articles


OBJECTIVE: The performance of standard comorbidity scores to control confounding is poorly defined in health care utilization data across elderly populations. We sought to evaluate and rank the performance of comorbidity scores across selected U.S. and Canadian elderly populations using health care utilization databases.

DESIGN: Cross-population validation study.

PARTICIPANTS: Study participants were residents age 65 years or older who had prescription drug coverage through state-funded programs selected from several large health care utilization databases available to the investigators: British Columbia, BC (N=141,161), New Jersey, NJ (N=235,881), and Pennsylvania, PA (N=230,913).

MEASUREMENTS: We calculated 6 commonly used comorbidity scores for all subjects during the baseline year (1994 for NJ and PA, and 1995 for BC). These included scores based on diagnoses (Romano, Deyo, D’Hoore, Ghali) and prescription drugs (CDS-1, CDS-2). The study outcome was 1-year mortality. The performance of scores was measured by c-statistics derived from multivariate logistic regression that included age and gender.

MAIN RESULTS: Across these 4 large elderly populations, we found the same rank order of performance in predicting 1-year mortality after including age and gender in each model: Romano (c-statistic 0.754 to 0.771), Deyo (c-statistic 0.753 to 0.768), D’Hoore (c-statistic 0.745 to 0.760), Ghali (c-statistic 0.733 to 0.745), CDS-1 (c-statistic 0.689 to 0.738), CDS-2 (c-statistic 0.677 to 0.718), and age and gender alone (c-statistic 0.664 to 0.681). Performance was improved by an average of 6% by adding the number of different prescription drugs received during the past year.

CONCLUSIONS: Performance ranking of 6 frequently used comorbidity scores was consistent across selected elderly populations. We recommend that investigators use these performance data as one important factor when selecting a comorbidity score for epidemiologic analyses of health care utilization data.

Key words

comorbidity adjustment confounding (epidemiology) prediction claims data methods 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Schneeweiss S, Maclure M. Use of comorbidity scores for control of confounding in studies using administrative databases. Int J Epidemiol. 2000;29:891–8.PubMedCrossRefGoogle Scholar
  2. 2.
    Greenfield S, Nelson EC. Recent developments and future issues in the use of health status assessment measures in clinical setting. Med Care. 1992;30:MS23-MS41.PubMedCrossRefGoogle Scholar
  3. 3.
    Iezzoni LI. Risk Adjustment for Measuring Healthcare Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997.Google Scholar
  4. 4.
    Roos LL, Sharp SM, Cohen MM, Wajda A. Risk adjustment in claims-based research: the search for efficient approaches. J Clin Epidemiol. 1989;42:1193–1206.PubMedCrossRefGoogle Scholar
  5. 5.
    Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol. 1993;46:1075–9.PubMedCrossRefGoogle Scholar
  6. 6.
    Romano PS, Roos LL, Jollis JG. Further evidence concerning the use of a clinical comorbidity index with ICD-9-CM administrative data. J Clin Epidemiol. 1993;46:1085–90.CrossRefGoogle Scholar
  7. 7.
    von Korff M, Wagner EH, Saunders K. A chronic disease score from automated pharmacy data. J Clin Epidemiol. 1992;45:197–203.CrossRefGoogle Scholar
  8. 8.
    Clark DO, von Korff M, Saunders K, Baluch WM, Simon GE. A chronic disease score with empirically derived weights. Med Care. 1995;33:783–95.PubMedCrossRefGoogle Scholar
  9. 9.
    Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–9.PubMedCrossRefGoogle Scholar
  10. 10.
    D’Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson Comorbidity Index. Methods Inf Med. 1993;32:382–7.PubMedGoogle Scholar
  11. 11.
    D’Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson index with administrative data bases. J Clin Epidemiol. 1996;49:1429–33.PubMedCrossRefGoogle Scholar
  12. 12.
    Ghali WA, Hall RE, Rosen AK, Ash AS, Moskowitz MA. Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data. J Clin Epidemiol. 1996;49:273–8.PubMedCrossRefGoogle Scholar
  13. 13.
    Melfi C, Holleman E, Arthur D, Katz B. Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data. J Clin Epidemiol. 1995;48:917–26.PubMedCrossRefGoogle Scholar
  14. 14.
    Poses RM, Smith WR, McClish DK, Anthony M. Controlling for confounding by indication for treatment. Are administrative data equivalent to clinical data? Med Care. 1995;33:AS36-AS46.PubMedGoogle Scholar
  15. 15.
    Schneeweiss S, Walker AM, Glynn RJ, Maclure M, Dormuth C, Soumerai SB. Outcomes of reference drug pricing for angiotensin-converting enzyme inhibitors. N Engl J Med. 2002;346:822–9.PubMedCrossRefGoogle Scholar
  16. 16.
    Schneeweiss S, Seeger J, Maclure M, Wang P, Avorn J, Glynn RJ. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol. 2001;154:854–64.PubMedCrossRefGoogle Scholar
  17. 17.
    Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chron Dis. 1987;40:373–83.PubMedCrossRefGoogle Scholar
  18. 18.
    Elixhauser A, Steiner C, Harris R, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36:8–27.PubMedCrossRefGoogle Scholar
  19. 19.
    Schneeweiss S, Wang PS, Avorn J, Glynn RJ. Improved comorbidity adjustment for predicting mortality in Medicare populations. Health Serv Res. 2003;38:1103–20.PubMedCrossRefGoogle Scholar
  20. 20.
    Zhang JX, Iwashyna TJ, Christakis NA. The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims. Med Care. 1999;37:1128–39.PubMedCrossRefGoogle Scholar
  21. 21.
    American Society of Health-System Pharmacists: AHFS Drug Information 96. Bethesda, Md; 1996.Google Scholar
  22. 22.
    Wang PS, Walker A, Tsuang M, Orav EJ, Levin R, Avorn J. Strategies for improving comorbidity measures based on Medicare and Medicaid claims data. J Clin Epidemiol. 2000;53:571–8.PubMedCrossRefGoogle Scholar
  23. 23.
    Yuan Z, Cooper GS, Einstadter D, Cebul RD, Rimm AA. The association between hospital type and mortality and length of stay. Med Care. 2000;38:231–45.PubMedCrossRefGoogle Scholar
  24. 24.
    Lessler JT, Harris BSH. Medicaid data as a source for postmarketing surveillance information, final report. Research Triangle Park, NC: Research Triangle Institute; 1984.Google Scholar
  25. 25.
    Iezzoni LI, Foley SM, Daley J, Hughes J, Fisher ES, Heeren T. Comorbidities, complications, and coding bias: does the number of diagnosis codes matter in predicting in-hospital mortality? JAMA. 1992;267:2197–203.PubMedCrossRefGoogle Scholar
  26. 26.
    Fisher ES, Whaley FS, Krushat M, et al. The accuracy of Medicare’s hospital claims data: progress has been made, but problems remain. Am J Public Health. 1992;82:243–8.PubMedCrossRefGoogle Scholar
  27. 27.
    Anderson GM, Kerluke KJ, Pulcins IR, Hertzman C, Barer ML. Trends and determinants of prescription drug expenditures in the elderly: data from the British Columbia Pharmacare Program. Inquiry. 1993;30:199–207.PubMedGoogle Scholar
  28. 28.
    Roos LL, Sharp SM, Cohen MM. Comparing clinical information with claims data: some similarities and differences. J Clin Epidemiol. 1991;44:881–8.PubMedCrossRefGoogle Scholar
  29. 29.
    Williams JI, Young W. Inventory of studies on the accuracy of Canadian health administrative databases. Technical Report, Institute for Clinical Evaluative Sciences (ICES); December 1996.Google Scholar
  30. 30.
    Fowles JB, Lawthers AG, Weiner JP, et al. Agreement between physician’s office records and Medicare part B claims data. Health Care Financ Rev. 1995;16:189–99.PubMedGoogle Scholar
  31. 31.
    Romano PS, Mark DH. Bias in the coding of hospital discharge data and its implications for quality assessment. Med Care. 1994;32:81–90.PubMedCrossRefGoogle Scholar
  32. 32.
    Glynn RJ, Monane M, Gurwitz JH, Choodnovskiy I, Avorn J. Agreement between drug treatment and a discharge diagnosis of diabetes. Am J Epidemiol. 1999;149:541–9.PubMedGoogle Scholar
  33. 33.
    Ash AS, Shwartz M. Evaluating the performance of risk-adjustment methods: dichotomous outcomes. In: Iezzoni LI, ed. Risk Adjustment for Measuring Healthcare Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997.Google Scholar
  34. 34.
    Wilson PWF, D’Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–47.PubMedGoogle Scholar
  35. 35.
    Hosmer DW, Lemeshow S. Applied Logistic Regression, 2nd ed. New York, NY: Wiley; 2000.Google Scholar
  36. 36.
    Liebetrau AM. Measures of Association. Quantitative Application in the Social Sciences. Vol. 32. Beverly Hills, Calif: Sage Publications; 1983.Google Scholar
  37. 37.
    Maclure M, Schneeweiss S. Causation of bias: the episcope. Epidemiology. 2001;12:114–22.PubMedCrossRefGoogle Scholar
  38. 38.
    Redelmeier DA, Tan SH, Booth GL. The treatment of unrelated disorders in patients with chronic medical diseases. N Engl J Med. 1998;338:1516–20.PubMedCrossRefGoogle Scholar
  39. 39.
    Glynn RJ, Monane M, Gurwitz JH, Choodnovskiy I, Avorn J. Aging, comorbidity, and reduced rates of drug treatment for diabetes mellitus. J Clin Epidemiol. 1999;52:781–90.PubMedCrossRefGoogle Scholar
  40. 40.
    Glynn RJ, Knight EL, Levon R, Avorn J. Paradoxical relations of drug treatment with mortality in older persons. Epidemiology. 2001;12:682–9.PubMedCrossRefGoogle Scholar
  41. 41.
    Moons KGM, Stijnen T, Michel BC, et al. Application of treatment thresholds to diagnostic test evaluation: an alternative to the comparison of areas under receiver operating characteristics curves. Med Decis Making. 1997;17:447–54.PubMedCrossRefGoogle Scholar
  42. 42.
    Hannan EL, Kilburn H, Lindsey ML, Lewis R. Clinical versus administrative data bases for CABG surgery. Does it matter? Med Care. 1992;30:892–907.PubMedCrossRefGoogle Scholar
  43. 43.
    Grover S, Coupal L, Hu X-P. Identifying adults at increased risk of coronary disease: how well do the current cholesterol guidelines work? JAMA. 1995;274:801–6.PubMedCrossRefGoogle Scholar
  44. 44.
    Wilson PWF, D’Agostino RB Sr., Levy L, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837–47.PubMedGoogle Scholar

Copyright information

© Society of General Internal Medicine 2004

Authors and Affiliations

  • Sebastian Schneeweiss
    • 5
    • 1
  • Philip S. Wang
    • 5
  • Jerry Avorn
    • 5
  • Malcolm Maclure
    • 1
    • 2
  • Raia Levin
    • 5
  • Robert J. Glynn
    • 5
    • 3
    • 4
  1. 1.Department of EpidemiologyHarvard School of Public HealthBoston
  2. 2.University of VictoriaVictoriaCanada
  3. 3.Division of Preventive MedicineBrigham and Women’s Hospital and Harvard Medical SchoolUSA
  4. 4.Department of BiostatisticsHarvard School of Public HealthBoston
  5. 5.Division of Pharmacoepidemiology and PharmacoeconomicsBrigham and Women’s Hospital and Harvard Medical SchoolBoston

Personalised recommendations