How Reliable Are County and Regional Health Rankings?
- 412 Downloads
Surveillance system indicators of morbidity, mortality, or behaviors are used to provide regional information for ever-smaller areas, most recently for ranking counties. These rankings are thought to provide information about the relative standing of regions and provide information about problem areas and the success of programs. We investigate the ability of such rankings to reliably assess health. We assess the reliability of several ranked health indices used at the county level and the consistency of an index's quality across different states. Reliability is assessed using an index of reliability, simulations, and the ability of an index to consistently identify the top 10 % (worst) counties within a state. There is marked variability across measures to provide consistent ranks, across states, and, many times, across states for a particular measure. A few health measures do consistently well, e.g., Teen Birth, Chlamydia, and Years of Potential Life Lost rates. While a few health rankings appear worthy of use for policy and in the identification of local problems, in general, they are not consistent across regions.
KeywordsHealth Services Research Quality Indicator Health Status Indicators Health Policy
This project was funded, in part, by the Iowa Consortium for Substance Abuse Research and Evaluation.
Conflict of Interest
None of the authors have any financial disclosures.
- Evans, M., Hastings, N., & Peacock, B. (1993). Statistical distributions (2nd ed.). New York: John Wiley & Sons.Google Scholar
- Guilford, J. P. (1954). Psychometric methods. New York: McGraw-Hill.Google Scholar
- Hofer, T. P., Hayward, R. A., Greenfield, S., Wagner, E. H., Kaplan, S. H., & Manning, W. G. (1999). The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease. JAMA: The Journal of the American Medical Association, 281, 2098–2105. doi: 10.1001/jama.281.22.2098.CrossRefGoogle Scholar
- Krieger, N., Chen, J. T., Waterman, P. D., Soobader, M.-J., Subramanian, S. V., & Carson, R. (2002). Geocoding and monitoring of us socioeconomic inequalities in mortality and cancer incidence: Does the choice of area-based measure and geographic level matter? American Journal of Epidemiology, 156, 471–482. doi: 10.1093/aje/kwf068.PubMedCrossRefGoogle Scholar
- Krieger, N., Chen, J. T., Waterman, P. D., Soobader, M.-J., Subramanian, S. V., & Carson, R. (2003). Choosing area based socioeconomic measures to monitor social inequalities in low birth weight and childhood lead poisoning: The public health disparities geocoding project (US). Journal of Epidemiology and Community Health, 57, 186–199. doi: 10.1093/aje/kwf068.PubMedCrossRefGoogle Scholar
- Louis, T. A., Jabine, T. B., & Gerstein, M. A. (2003). Statistical issues in allocating funds by formula. Washington, D.C.: National Academies Press.Google Scholar
- Marshall, E. C., Sanderson, C., Spiegelhalter, D. J., & McKee, M. (1998). Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates Commentary: How robust are rankings? The implications of confidence intervals. British Medical Journal, 316, 1701–1705. doi: 10.1136/bmj.316.7146.1701.PubMedCrossRefGoogle Scholar
- McGinnis, J.M. (2010). Observations on incentives to improve population health. Preventing Chronic Disease, 9, A92. Retreived from http://www.cdc.gov/pcd/issues/2010/sep/10_0078.htm.
- Muhuri, P.K., & Ducrest, J.L. (2007). Block grants and formula grants: A guide for allotment calculations. Rockville, MD: SAMHSA, Office of Applied Studies Retrieved from http://oas.samhsa.gov/BG_documentation_070809_final_psg.pdf.
- Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.Google Scholar
- Oliver, T.R. (2010). Population health rankings as policy indicators and performance measures. Preventing Chronic Disease, 5, A101. Retrieved from http://www.cdc.gov/pcd/issues/2010/sep/10_0040.htm.
- Peppard, P. E., Kindig, D. A., Dranger, E., Jovaag, A., & Remington, P. L. (2008). Ranking community health status to stimulate discussion of local public health issues: The Wisconsin county health rankings. American Journal of Public Health, 98, 209–212. doi: 10.2105/AJPH.2006.092981.PubMedCrossRefGoogle Scholar
- Vila, P., Booske, B., & Remington, P. (2006). Measuring mortality in the Wisconsin county health. Population Health Institute Technical Report., (Department of Population Health Sciences). Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.169.3689&rep=rep1&type=pdf