Advertisement

Acta Diabetologica

, Volume 52, Issue 1, pp 91–101 | Cite as

The performance of diabetes risk prediction models in new populations: the role of ethnicity of the development cohort

  • Stephanie K. Tanamas
  • Dianna J. Magliano
  • Beverley Balkau
  • Jaakko Tuomilehto
  • Sudhir Kowlessur
  • Stefan Söderberg
  • Paul Z. Zimmet
  • Jonathan E. Shaw
Original Article

Abstract

It is believed that diabetes risk scores need to be ethnic specific. However, this prerequisite has not been tested. We examined the performance of several risk models, developed in various populations, in a Europid and a South Asian population. The performance of 14 published risk prediction models were tested in two prospective studies: the Australian Diabetes, Obesity and Lifestyle (AusDiab) study and the Mauritius non-communicable diseases survey. Eight models were developed in Europid populations; the remainder in various non-Europid populations. Model performance was assessed using area under the receiver operating characteristic curves (discrimination), Hosmer–Lemeshow tests (goodness-of-fit) and Brier scores (accuracy). In both AusDiab and Mauritius, discrimination was highest for a model developed in a mixed population (non-Hispanic white and African American) and lowest for a model developed in a Europid population. Discrimination for all scores was higher in AusDiab than in Mauritius. For almost all models, goodness-of-fit was poor irrespective of the ethnicity of the development cohort, and accuracy was higher in AusDiab compared to Mauritius. Our results suggest that similarity of ethnicity or similarity of diabetes risk may not be the best way of identifying models that will perform well in another population. Differences in study methodology likely account for much of the difference in the performance. Thus, identifying models which use measurements that are clearly described and easily reproducible for both research and clinical settings may be more important.

Keywords

Risk prediction model Diabetes Performance Discrimination Calibration 

Notes

Acknowledgments

The AusDiab study co-coordinated by the Baker IDI Heart and Diabetes Institute is enormously grateful to Anne Allman from the Baker IDI Heart and Diabetes Institute, Robert Atkins from Monash University, Stan Bennett from the Australian Institute of Health and Welfare, Annaliese Bonney from the Baker IDI Heart and Diabetes Institute, Steven Chadban from the University of Sydney, Max de Courten from the Baker IDI Heart and Diabetes Institute, Marita Dalton from the Baker IDI Heart and Diabetes Institute, Terrance Dwyer from the Murdoch Children’s Research Institute, Royal Melbourne Hospital, Hassan Jahangir from the Baker IDI Heart and Diabetes Institute, Damien Jolley from Monash University, Dan McCarty from the Baker IDI Heart and Diabetes Institute, Adam Meehan from the Baker IDI Heart and Diabetes Institute, Nicole Meinig from the Baker IDI Heart and Diabetes Institute, Shirley Murray from the Baker IDI Heart and Diabetes Institute, Kerin O’Dea from the University of Melbourne, Kevin Polkinghorne from Monash University, Patrick Phillips from the Queen Elizabeth Hospital, Adelaide, Clare Reid from the Baker IDI Heart and Diabetes Institute, Alison Stewart from the Baker IDI Heart and Diabetes Institute, Robyn Tapp from the Baker IDI Heart and Diabetes Institute, Hugh Taylor from the Centre for Eye Research, Australia, Theresa Whalen from the Baker IDI Heart and Diabetes Institute and Fay Wilson from the Baker IDI Heart and Diabetes Institute and the Victorian Government’s OIS Program for their invaluable contribution to the set-up and field activities of AusDiab. The baseline Mauritius NCD surveys were funded by the World Health Organization, Baker IDI Heart and Diabetes Institute, the University of Newcastle upon Tyne (UK), and the National Public Health Institute, Finland, and by the National Institutes of Health (Grant DK-25446). We are most grateful to the staff at the Ministry of Health and Quality of Life in Mauritius for conducting both the baseline and the follow-up study. We thank the participants for volunteering their time to be involved in the study. We acknowledge, in particular, the work of Gary Dowse, Department of Health, Western Australia; Max De Courten, University of Copenhagen, Copenhagen, Denmark; Ray Sparks, Department of Pathology, Monash University, Melbourne, Victoria, Australia; Pierrot Chitson, Ministry of Health, Mauritius. JES is supported by a National Health and Medical Research Council (NHMRC) Senior Research Fellowship. DJM and SKT are supported by NHMRC Grants. S.S. is supported by Grants from the Vasterbotten Country Council and the Swedish Heart and Lung Foundation.

Conflict of interest

Stephanie K Tanamas, Dianna J Magliano, Beverley Balkau, Jaakko Tuomilehto, Sudhir Kowlessur, Stefan Söderberg, Paul Z Zimmet, Jonathan E Shaw declare that they have no conflict of interest.

Human and animal rights

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5).

Informed consent

Informed consent was obtained from all participants included in the study.

Supplementary material

592_2014_607_MOESM1_ESM.docx (14 kb)
Supplementary material 1 (DOCX 14 kb)

References

  1. 1.
    International Diabetes Federation (2013) IDF Diabetes Atlas 6th edition. In. IDFGoogle Scholar
  2. 2.
    Knowler WC, Barrett-Connor E, Fowler SE et al (2002) Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med 346:393–403PubMedCrossRefGoogle Scholar
  3. 3.
    Pan XR, Li GW, Hu YH et al (1997) Effects of diet and exercise in preventing NIDDM in people with impaired glucose tolerance. The Da Qing IGT and diabetes study. Diabetes Care 20:537–544PubMedCrossRefGoogle Scholar
  4. 4.
    Lindstrom J, Louheranta A, Mannelin M et al (2003) The finnish diabetes prevention study (DPS): lifestyle intervention and 3-year results on diet and physical activity. Diabetes Care 26:3230–3236PubMedCrossRefGoogle Scholar
  5. 5.
    Ramachandran A, Snehalatha C, Mary S, Mukesh B, Bhaskar AD, Vijay V (2006) The indian diabetes prevention programme shows that lifestyle modification and metformin prevent type 2 diabetes in Asian Indian subjects with impaired glucose tolerance (IDPP-1). Diabetologia 49:289–297PubMedCrossRefGoogle Scholar
  6. 6.
    Stratton IM, Adler AI, Neil HA et al (2000) Association of glycaemia with macrovascular and microvascular complications of type 2 diabetes (UKPDS 35): prospective observational study. BMJ 321:405–412PubMedCentralPubMedCrossRefGoogle Scholar
  7. 7.
    UK Prospective Diabetes Study Group (1998) Tight blood pressure control and risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 38. BMJ 317:703–713PubMedCentralCrossRefGoogle Scholar
  8. 8.
    UK Prospective Diabetes Study (UKPDS) Group (1998) Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet 352:837–853CrossRefGoogle Scholar
  9. 9.
    Noble D, Mathur R, Dent T, Meads C, Greenhalgh T (2011) Risk models and scores for type 2 diabetes: systematic review. BMJ 343:d7163PubMedCentralPubMedCrossRefGoogle Scholar
  10. 10.
    Dunstan DW, Zimmet PZ, Welborn TA et al (2002) The Australian diabetes, obesity and lifestyle study (AusDiab)—methods and response rates. Diabetes Res Clin Pract 57:119–129PubMedCrossRefGoogle Scholar
  11. 11.
    Magliano DJ, Barr EL, Zimmet PZ et al (2008) Glucose indices, health behaviors, and incidence of diabetes in Australia: the Australian diabetes, obesity and lifestyle study. Diabetes Care 31:267–272PubMedCrossRefGoogle Scholar
  12. 12.
    Soderberg S, Zimmet P, Tuomilehto J et al (2004) High incidence of type 2 diabetes and increasing conversion rates from impaired fasting glucose and impaired glucose tolerance to diabetes in Mauritius. J Intern Med 256:37–47PubMedCrossRefGoogle Scholar
  13. 13.
    Soderberg S, Zimmet P, Tuomilehto J et al (2005) Increasing prevalence of Type 2 diabetes mellitus in all ethnic groups in Mauritius. Diabet Med 22:61–68PubMedCrossRefGoogle Scholar
  14. 14.
    Briganti EM, Shaw JE, Chadban SJ et al (2003) Untreated hypertension among Australian adults: the 1999–2000 Australian diabetes, obesity and lifestyle study (AusDiab). Med J Aust 179:135–139PubMedGoogle Scholar
  15. 15.
    World Health Organization (2006) Definition and diagnosis of diabetes mellitus and intermediate hyperglycaemia: report of a WHO/IDF consultation in. World Health Org, GenevaGoogle Scholar
  16. 16.
    Stern MP, Williams K, Haffner SM (2002) Identification of persons at high risk for type 2 diabetes mellitus: do we need the oral glucose tolerance test? Ann Intern Med 136:575–581PubMedCrossRefGoogle Scholar
  17. 17.
    Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36PubMedCrossRefGoogle Scholar
  18. 18.
    Harrison DA, Brady AR, Parry GJ, Carpenter JR, Rowan K (2006) Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom. Crit Care Med 34:1378–1388PubMedCrossRefGoogle Scholar
  19. 19.
    Hosmer DW, Lemeshow S (2000) Applied logistic regression. John Wiley & Sons, New YorkCrossRefGoogle Scholar
  20. 20.
    Paul P, Pennell ML, Lemeshow S (2013) Standardizing the power of the Hosmer–Lemeshow goodness of fit test in large data sets. Stat Med 32:67–80PubMedCrossRefGoogle Scholar
  21. 21.
    Hosmer DW, Hjort NL (2002) Goodness-of-fit processes for logistic regression: simulation results. Stat Med 21:2723–2738PubMedCrossRefGoogle Scholar
  22. 22.
    Gerds TA, Cai T, Schumacher M (2008) The performance of risk prediction models. Biom J 50:457–479PubMedCrossRefGoogle Scholar
  23. 23.
    Moons KG, Kengne AP, Grobbee DE et al (2012) Risk prediction models: II. External validation, model updating, and impact assessment. Heart 98:691–698PubMedCrossRefGoogle Scholar
  24. 24.
    Janssen KJ, Moons KG, Kalkman CJ, Grobbee DE, Vergouwe Y (2008) Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 61:76–86PubMedCrossRefGoogle Scholar
  25. 25.
    Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD (2004) Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 23:2567–2586PubMedCrossRefGoogle Scholar
  26. 26.
    Toll DB, Janssen KJ, Vergouwe Y, Moons KG (2008) Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 61:1085–1094PubMedCrossRefGoogle Scholar
  27. 27.
    Lin JW, Chang YC, Li HY et al (2009) Cross-sectional validation of diabetes risk scores for predicting diabetes, metabolic syndrome, and chronic kidney disease in Taiwanese. Diabetes Care 32:2294–2296PubMedCentralPubMedCrossRefGoogle Scholar
  28. 28.
    Rosella LC, Mustard CA, Stukel TA et al (2012) The role of ethnicity in predicting diabetes risk at the population level. Ethnicity Health 17:419–437PubMedCentralPubMedCrossRefGoogle Scholar
  29. 29.
    Rahman M, Simmons RK, Harding AH, Wareham NJ, Griffin SJ (2008) A simple risk score identifies individuals at high risk of developing Type 2 diabetes: a prospective cohort study. Fam Pract 25:191–196PubMedCrossRefGoogle Scholar
  30. 30.
    Lindstrom J, Tuomilehto J (2003) The diabetes risk score: a practical tool to predict type 2 diabetes risk. Diabetes Care 26:725–731PubMedCrossRefGoogle Scholar
  31. 31.
    Schulze MB, Hoffmann K, Boeing H et al (2007) An accurate risk score based on anthropometric, dietary, and lifestyle factors to predict the development of type 2 diabetes. Diabetes Care 30:510–515PubMedCrossRefGoogle Scholar
  32. 32.
    Wilson PW, Meigs JB, Sullivan L, Fox CS, Nathan DM, D’Agostino RB Sr (2007) Prediction of incident diabetes mellitus in middle-aged adults: the Framingham offspring study. Arch Intern Med 167:1068–1074PubMedCrossRefGoogle Scholar
  33. 33.
    Aekplakorn W, Bunnag P, Woodward M et al (2006) A risk score for predicting incident diabetes in the Thai population. Diabetes Care 29:1872–1877PubMedCrossRefGoogle Scholar
  34. 34.
    Chen L, Magliano DJ, Balkau B et al (2010) AUSDRISK: an Australian type 2 diabetes risk assessment tool based on demographic, lifestyle and simple anthropometric measures. Med J Aust 192:197–202PubMedGoogle Scholar
  35. 35.
    Sun F, Tao Q, Zhan S (2009) An accurate risk score for estimation 5-year risk of type 2 diabetes based on a health screening population in Taiwan. Diabetes Res Clin Pract 85:228–234PubMedCrossRefGoogle Scholar
  36. 36.
    Schmidt MI, Duncan BB, Bang H et al (2005) Identifying individuals at high risk for diabetes: the atherosclerosis risk in communities study. Diabetes Care 28:2013–2018PubMedCrossRefGoogle Scholar
  37. 37.
    Mohan V, Deepa R, Deepa M, Somannavar S, Datta M (2005) A simplified Indian diabetes risk score for screening for undiagnosed diabetic subjects. J Assoc Physicians India 53:759–763PubMedGoogle Scholar

Copyright information

© Springer-Verlag Italia 2014

Authors and Affiliations

  • Stephanie K. Tanamas
    • 1
  • Dianna J. Magliano
    • 1
  • Beverley Balkau
    • 1
    • 2
    • 3
  • Jaakko Tuomilehto
    • 4
    • 5
    • 6
  • Sudhir Kowlessur
    • 7
  • Stefan Söderberg
    • 1
    • 8
  • Paul Z. Zimmet
    • 1
  • Jonathan E. Shaw
    • 1
  1. 1.Baker IDI Heart and Diabetes InstituteMelbourneAustralia
  2. 2.U1018, Centre for Research in Epidemiology and Population HealthInsermVillejuifFrance
  3. 3.Paris-Sud 11 UniversityVillejuifFrance
  4. 4.Diabetes Prevention UnitNational Institute for Health and WelfareHelsinkiFinland
  5. 5.King Abdulaziz UniversityJeddahSaudi Arabia
  6. 6.Centre for Vascular PreventionDanube-University KremsKremsAustria
  7. 7.Ministry of Health and Quality of LifePort LouisIsland of Mauritius
  8. 8.Department of Public Health and Clinical Medicine, Heart CentreUmeå UniversityUmeåSweden

Personalised recommendations