Abstract
It is believed that diabetes risk scores need to be ethnic specific. However, this prerequisite has not been tested. We examined the performance of several risk models, developed in various populations, in a Europid and a South Asian population. The performance of 14 published risk prediction models were tested in two prospective studies: the Australian Diabetes, Obesity and Lifestyle (AusDiab) study and the Mauritius non-communicable diseases survey. Eight models were developed in Europid populations; the remainder in various non-Europid populations. Model performance was assessed using area under the receiver operating characteristic curves (discrimination), Hosmer–Lemeshow tests (goodness-of-fit) and Brier scores (accuracy). In both AusDiab and Mauritius, discrimination was highest for a model developed in a mixed population (non-Hispanic white and African American) and lowest for a model developed in a Europid population. Discrimination for all scores was higher in AusDiab than in Mauritius. For almost all models, goodness-of-fit was poor irrespective of the ethnicity of the development cohort, and accuracy was higher in AusDiab compared to Mauritius. Our results suggest that similarity of ethnicity or similarity of diabetes risk may not be the best way of identifying models that will perform well in another population. Differences in study methodology likely account for much of the difference in the performance. Thus, identifying models which use measurements that are clearly described and easily reproducible for both research and clinical settings may be more important.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
International Diabetes Federation (2013) IDF Diabetes Atlas 6th edition. In. IDF
Knowler WC, Barrett-Connor E, Fowler SE et al (2002) Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med 346:393–403
Pan XR, Li GW, Hu YH et al (1997) Effects of diet and exercise in preventing NIDDM in people with impaired glucose tolerance. The Da Qing IGT and diabetes study. Diabetes Care 20:537–544
Lindstrom J, Louheranta A, Mannelin M et al (2003) The finnish diabetes prevention study (DPS): lifestyle intervention and 3-year results on diet and physical activity. Diabetes Care 26:3230–3236
Ramachandran A, Snehalatha C, Mary S, Mukesh B, Bhaskar AD, Vijay V (2006) The indian diabetes prevention programme shows that lifestyle modification and metformin prevent type 2 diabetes in Asian Indian subjects with impaired glucose tolerance (IDPP-1). Diabetologia 49:289–297
Stratton IM, Adler AI, Neil HA et al (2000) Association of glycaemia with macrovascular and microvascular complications of type 2 diabetes (UKPDS 35): prospective observational study. BMJ 321:405–412
UK Prospective Diabetes Study Group (1998) Tight blood pressure control and risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 38. BMJ 317:703–713
UK Prospective Diabetes Study (UKPDS) Group (1998) Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet 352:837–853
Noble D, Mathur R, Dent T, Meads C, Greenhalgh T (2011) Risk models and scores for type 2 diabetes: systematic review. BMJ 343:d7163
Dunstan DW, Zimmet PZ, Welborn TA et al (2002) The Australian diabetes, obesity and lifestyle study (AusDiab)—methods and response rates. Diabetes Res Clin Pract 57:119–129
Magliano DJ, Barr EL, Zimmet PZ et al (2008) Glucose indices, health behaviors, and incidence of diabetes in Australia: the Australian diabetes, obesity and lifestyle study. Diabetes Care 31:267–272
Soderberg S, Zimmet P, Tuomilehto J et al (2004) High incidence of type 2 diabetes and increasing conversion rates from impaired fasting glucose and impaired glucose tolerance to diabetes in Mauritius. J Intern Med 256:37–47
Soderberg S, Zimmet P, Tuomilehto J et al (2005) Increasing prevalence of Type 2 diabetes mellitus in all ethnic groups in Mauritius. Diabet Med 22:61–68
Briganti EM, Shaw JE, Chadban SJ et al (2003) Untreated hypertension among Australian adults: the 1999–2000 Australian diabetes, obesity and lifestyle study (AusDiab). Med J Aust 179:135–139
World Health Organization (2006) Definition and diagnosis of diabetes mellitus and intermediate hyperglycaemia: report of a WHO/IDF consultation in. World Health Org, Geneva
Stern MP, Williams K, Haffner SM (2002) Identification of persons at high risk for type 2 diabetes mellitus: do we need the oral glucose tolerance test? Ann Intern Med 136:575–581
Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36
Harrison DA, Brady AR, Parry GJ, Carpenter JR, Rowan K (2006) Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom. Crit Care Med 34:1378–1388
Hosmer DW, Lemeshow S (2000) Applied logistic regression. John Wiley & Sons, New York
Paul P, Pennell ML, Lemeshow S (2013) Standardizing the power of the Hosmer–Lemeshow goodness of fit test in large data sets. Stat Med 32:67–80
Hosmer DW, Hjort NL (2002) Goodness-of-fit processes for logistic regression: simulation results. Stat Med 21:2723–2738
Gerds TA, Cai T, Schumacher M (2008) The performance of risk prediction models. Biom J 50:457–479
Moons KG, Kengne AP, Grobbee DE et al (2012) Risk prediction models: II. External validation, model updating, and impact assessment. Heart 98:691–698
Janssen KJ, Moons KG, Kalkman CJ, Grobbee DE, Vergouwe Y (2008) Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 61:76–86
Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD (2004) Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 23:2567–2586
Toll DB, Janssen KJ, Vergouwe Y, Moons KG (2008) Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 61:1085–1094
Lin JW, Chang YC, Li HY et al (2009) Cross-sectional validation of diabetes risk scores for predicting diabetes, metabolic syndrome, and chronic kidney disease in Taiwanese. Diabetes Care 32:2294–2296
Rosella LC, Mustard CA, Stukel TA et al (2012) The role of ethnicity in predicting diabetes risk at the population level. Ethnicity Health 17:419–437
Rahman M, Simmons RK, Harding AH, Wareham NJ, Griffin SJ (2008) A simple risk score identifies individuals at high risk of developing Type 2 diabetes: a prospective cohort study. Fam Pract 25:191–196
Lindstrom J, Tuomilehto J (2003) The diabetes risk score: a practical tool to predict type 2 diabetes risk. Diabetes Care 26:725–731
Schulze MB, Hoffmann K, Boeing H et al (2007) An accurate risk score based on anthropometric, dietary, and lifestyle factors to predict the development of type 2 diabetes. Diabetes Care 30:510–515
Wilson PW, Meigs JB, Sullivan L, Fox CS, Nathan DM, D’Agostino RB Sr (2007) Prediction of incident diabetes mellitus in middle-aged adults: the Framingham offspring study. Arch Intern Med 167:1068–1074
Aekplakorn W, Bunnag P, Woodward M et al (2006) A risk score for predicting incident diabetes in the Thai population. Diabetes Care 29:1872–1877
Chen L, Magliano DJ, Balkau B et al (2010) AUSDRISK: an Australian type 2 diabetes risk assessment tool based on demographic, lifestyle and simple anthropometric measures. Med J Aust 192:197–202
Sun F, Tao Q, Zhan S (2009) An accurate risk score for estimation 5-year risk of type 2 diabetes based on a health screening population in Taiwan. Diabetes Res Clin Pract 85:228–234
Schmidt MI, Duncan BB, Bang H et al (2005) Identifying individuals at high risk for diabetes: the atherosclerosis risk in communities study. Diabetes Care 28:2013–2018
Mohan V, Deepa R, Deepa M, Somannavar S, Datta M (2005) A simplified Indian diabetes risk score for screening for undiagnosed diabetic subjects. J Assoc Physicians India 53:759–763
Acknowledgments
The AusDiab study co-coordinated by the Baker IDI Heart and Diabetes Institute is enormously grateful to Anne Allman from the Baker IDI Heart and Diabetes Institute, Robert Atkins from Monash University, Stan Bennett from the Australian Institute of Health and Welfare, Annaliese Bonney from the Baker IDI Heart and Diabetes Institute, Steven Chadban from the University of Sydney, Max de Courten from the Baker IDI Heart and Diabetes Institute, Marita Dalton from the Baker IDI Heart and Diabetes Institute, Terrance Dwyer from the Murdoch Children’s Research Institute, Royal Melbourne Hospital, Hassan Jahangir from the Baker IDI Heart and Diabetes Institute, Damien Jolley from Monash University, Dan McCarty from the Baker IDI Heart and Diabetes Institute, Adam Meehan from the Baker IDI Heart and Diabetes Institute, Nicole Meinig from the Baker IDI Heart and Diabetes Institute, Shirley Murray from the Baker IDI Heart and Diabetes Institute, Kerin O’Dea from the University of Melbourne, Kevin Polkinghorne from Monash University, Patrick Phillips from the Queen Elizabeth Hospital, Adelaide, Clare Reid from the Baker IDI Heart and Diabetes Institute, Alison Stewart from the Baker IDI Heart and Diabetes Institute, Robyn Tapp from the Baker IDI Heart and Diabetes Institute, Hugh Taylor from the Centre for Eye Research, Australia, Theresa Whalen from the Baker IDI Heart and Diabetes Institute and Fay Wilson from the Baker IDI Heart and Diabetes Institute and the Victorian Government’s OIS Program for their invaluable contribution to the set-up and field activities of AusDiab. The baseline Mauritius NCD surveys were funded by the World Health Organization, Baker IDI Heart and Diabetes Institute, the University of Newcastle upon Tyne (UK), and the National Public Health Institute, Finland, and by the National Institutes of Health (Grant DK-25446). We are most grateful to the staff at the Ministry of Health and Quality of Life in Mauritius for conducting both the baseline and the follow-up study. We thank the participants for volunteering their time to be involved in the study. We acknowledge, in particular, the work of Gary Dowse, Department of Health, Western Australia; Max De Courten, University of Copenhagen, Copenhagen, Denmark; Ray Sparks, Department of Pathology, Monash University, Melbourne, Victoria, Australia; Pierrot Chitson, Ministry of Health, Mauritius. JES is supported by a National Health and Medical Research Council (NHMRC) Senior Research Fellowship. DJM and SKT are supported by NHMRC Grants. S.S. is supported by Grants from the Vasterbotten Country Council and the Swedish Heart and Lung Foundation.
Conflict of interest
Stephanie K Tanamas, Dianna J Magliano, Beverley Balkau, Jaakko Tuomilehto, Sudhir Kowlessur, Stefan Söderberg, Paul Z Zimmet, Jonathan E Shaw declare that they have no conflict of interest.
Human and animal rights
All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5).
Informed consent
Informed consent was obtained from all participants included in the study.
Author information
Authors and Affiliations
Corresponding author
Additional information
Managed by Massimo Porta.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Tanamas, S.K., Magliano, D.J., Balkau, B. et al. The performance of diabetes risk prediction models in new populations: the role of ethnicity of the development cohort. Acta Diabetol 52, 91–101 (2015). https://doi.org/10.1007/s00592-014-0607-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00592-014-0607-x