Skip to main content
SpringerLink
Log in

Performance of the UKPDS Outcomes Model 2 for Predicting Death and Cardiovascular Events in Patients with Type 2 Diabetes Mellitus from a German Population-Based Cohort

  • Original Research Article
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Background and Objective

Accurate prediction of relevant outcomes is important for targeting therapies and to support health economic evaluations of healthcare interventions in patients with diabetes. The United Kingdom Prospective Diabetes Study (UKPDS) risk equations are some of the most frequently used risk equations. This study aims to analyze the calibration and discrimination of the updated UKPDS risk equations as implemented in the UKPDS Outcomes Model 2 (UKPDS-OM2) for predicting cardiovascular (CV) events and death in patients with type 2 diabetes mellitus (T2DM) from population-based German samples.

Methods

Analyses are based on data of 456 individuals diagnosed with T2DM who participated in two population-based studies in southern Germany (KORA (Cooperative Health Research in the Region of Augsburg)-A: 1997/1998, n = 178; KORA-S4: 1999–2001, n = 278). We compared the participants’ 10-year observed incidence of mortality, CV mortality, myocardial infarction (MI), and stroke with the predicted event rate of the UKPDS-OM2. The model’s calibration was evaluated by Greenwood–Nam–D’Agostino tests and discrimination was evaluated by C-statistics.

Results

Of the 456 participants with T2DM (mean age 65 years, mean diabetes duration 8 years, 56% male), over the 10-year follow-up time 129 died (61 due to CV events), 64 experienced an MI, and 46 a stroke. The UKPDS-OM2 significantly over-predicted mortality and CV mortality by 25% and 28%, respectively (Greenwood–Nam–D’Agostino tests: p < 0.01), but there was no significant difference between predicted and observed MI and stroke risk. The model poorly discriminated for death (C-statistic [95% confidence interval] = 0.64 [0.60–0.69]), CV death (0.64 [0.58–0.71]), and MI (0.58 [0.52–0.66]), and failed to discriminate for stroke (0.57 [0.47–0.66]).

Conclusions

The study results demonstrate acceptable calibration and poor discrimination of the UKPDS-OM2 for predicting death and CV events in this population-based German sample. Those limitations should be considered when using the UKPDS-OM2 for economic evaluations of healthcare strategies or using the risk equations for clinical decision-making.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Availability of Data and Materials

The data are subject to national data protection laws and restrictions were imposed by the Ethics Committee of the Bavarian Medical Association to ensure data privacy of the study participants and therefore data cannot be made freely available in a public repository. However, data can be requested through an individual project agreement with KORA via the online portal KORA.passt (https://epi.helmholtz-muenchen.de/).

References

  1. Jacobs E, Hoyer A, Brinks R, Kuss O, Rathmann W. Burden of mortality attributable to diagnosed diabetes: a nationwide analysis based on claims data from 65 million people in Germany. Diabetes Care. 2017;40(12):1703–9.

    Article  Google Scholar 

  2. Bommer C, Sagalova V, Heesemann E, Manne-Goehler J, Atun R, Barnighausen T, et al. Global economic burden of diabetes in adults: projections from 2015 to 2030. Diabetes Care. 2018;41(5):963–70.

    Article  Google Scholar 

  3. Hunger M, Thorand B, Schunk M, Doring A, Menn P, Peters A, et al. Multimorbidity and health-related quality of life in the older population: results from the German KORA-age study. Health Qual Life Outcomes. 2011;9:53.

    Article  Google Scholar 

  4. American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27(9):2262–5.

    Article  Google Scholar 

  5. Baena-Diez JM, Peñafiel J, Subirana I, Ramos R, Elosua R, Marín-Ibañez A, FRESCO Investigators, et al. Risk of cause-specific death in individuals with diabetes: a competing risks analysis. Diabetes Care. 2016;39(11):1987–95.

    Article  Google Scholar 

  6. National Institute for Health and Care Excellence. Lipid-modifying drugs; 2017. https://www.nice.org.uk/advice/ktt3/chapter/evidence-context#statins. Accessed 3 Sept 2018.

  7. Chou R, Dana T, Blazina I, Daeges M, Bougatsos C, Grusing S, et al. Statin use for the prevention of cardiovascular disease in adults: a systematic review for the U.S. Preventive Services Task Force. Rockville: Agency for Healthcare Research and Quality (US); 2016. (Evidence Syntheses, No. 139). https://www.ncbi.nlm.nih.gov/books/NBK396415/. Accessed 3 Sept 2018.

  8. Govan L, Wu O, Lindsay R, Briggs A. How do diabetes models measure up? A review of diabetes economic models and ADA guidelines. J Health Econ Outcomes Res. 2015;3(2):132–52.

    Google Scholar 

  9. UK Prospective Diabetes Study (UKPDS). VIII. Study design, progress and performance. Diabetologia. 1991;34(12):877–90.

    Article  Google Scholar 

  10. Hayes AJ, Leal J, Gray AM, Holman RR, Clarke PM. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56(9):1925–33.

    Article  CAS  Google Scholar 

  11. McEwan P, Bergenheim K, Currie CJ. External validation of the UKPDS Oucomes model equations (UKPDS 68), and the UKPDS risk engine equations (UKPDS 56 and 60) in forecasting cardiovascular outcomes in people with type 2 diabetes. Value Health. 2006;9(3):A37–8.

    Article  Google Scholar 

  12. Simmons RK, Coleman RL, Price HC, Holman RR, Khaw K-T, Wareham NJ, et al. Performance of the UK Prospective Diabetes Study Risk Engine and the Framingham risk equations in estimating cardiovascular disease in the EPIC-Norfolk cohort. Diabetes Care. 2009;32(4):708–13.

    Article  Google Scholar 

  13. Tao L, Wilson ECF, Griffin SJ, Simmons RK. Performance of the UKPDS outcomes model for prediction of myocardial infarction and stroke in the ADDITION-Europe trial cohort. Value Health. 2013;16(6):1074–80.

    Article  Google Scholar 

  14. van Dieren S, Peelen LM, Nöthlings U, van der Schouw YT, Rutten GE, Spijkerman AM, et al. External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes. Diabetologia. 2011;54(2):264–70.

    Article  CAS  Google Scholar 

  15. Kengne AP, Patel A, Colagiuri S, Heller S, Hamet P, Marre M, ADVANCE Collaborative Group, et al. The Framingham and UK Prospective Diabetes Study (UKPDS) risk equations do not reliably estimate the probability of cardiovascular events in a large ethnically diverse sample of patients with diabetes: the Action in Diabetes and Vascular Disease: Preterax and Diamicron-MR Controlled Evaluation (ADVANCE) study. Diabetologia. 2010;53(5):821–31.

    Article  CAS  Google Scholar 

  16. Diabetes Trials Unit, H.E.R.C. UKPDS Outcomes Model user manual. Version 2; 2015. https://www.dtu.ox.ac.uk/outcomesmodel/OM2Manual.pdf. Accessed 26 July 2017.

  17. Willis M, Johansen P, Nilsson A, Asseburg C. Validation of the economic and health outcomes model of type 2 diabetes mellitus (ECHO-T2DM). Pharmacoeconomics. 2017;35(3):375–96.

    Article  Google Scholar 

  18. Lundqvist A, Steen Carlsson K, Johansen P, Andersson E, Willis M. Validation of the IHE Cohort Model of Type 2 Diabetes and the impact of choice of macrovascular risk equations. PLoS One. 2014;9(10):e110235.

    Article  Google Scholar 

  19. McEwan P, Foos V, Palmer JL, Lamotte M, Lloyd A, Grant D. Validation of the IMS CORE Diabetes Model. Value Health. 2014;17(6):714–24.

    Article  Google Scholar 

  20. McEwan P, Ward T, Bennett H, Bergenheim K. Validation of the UKPDS 82 risk equations within the Cardiff Diabetes Model. Cost Effect Resour Alloc. 2015;13:12.

    Article  Google Scholar 

  21. Palmer AJ, Mount Hood 5 Modeling Group, Clarke P, Gray A, Leal J, Lloyd A, et al. Computer modeling of diabetes and its complications: a report on the Fifth Mount Hood challenge meeting. Value Health. 2013;16(4):670–85.

    Article  Google Scholar 

  22. Basu S, Sussman JB, Berkowitz SA, Hayward RA, Bertoni AG, Correa A, et al. Validation of Risk Equations for Complications of Type 2 Diabetes (RECODe) using individual participant data from diverse longitudinal cohorts in the U.S. Diabetes Care. 2018;41(3):586–95.

    Article  Google Scholar 

  23. Holle R, Happich M, Lowel H, Wichmann HE, MONICA/KORA Study Group. KORA—a research platform for population based health research. Gesundheitswesen. 2005;67(Suppl 1):S19–25.

    Article  Google Scholar 

  24. Laxy M, Knoll G, Schunk M, Meisinger C, Huth C, Holle R. Quality of diabetes care in Germany improved from 2000 to 2007 to 2014, but improvements diminished since 2007. Evidence from the population-based KORA studies. PLoS One. 2016;11(10):e0164704.

    Article  Google Scholar 

  25. Laxy M, Mielck A, Hunger M, Schunk M, Meisinger C, Ruckert IM, et al. The association between patient-reported self-management behavior, intermediate clinical outcomes, and mortality in patients with type 2 diabetes: results from the KORA-A study. Diabetes Care. 2014;37(6):1604–12.

    Article  Google Scholar 

  26. Yuan Y. Multiple imputation using SAS software. J Stat Softw. 2011;45(6):1–25.

    Article  Google Scholar 

  27. Kuhfeld W, So Y. Creating and customizing the Kaplan–Meier survival plot in PROC LIFETEST in SAS Global Forum Statistics and Data Analysis. Paper 427. 2013. https://support.sas.com/resources/papers/proceedings13/427-2013.pdf. Accessed 24 June 2019.

  28. Demler OV, Paynter NP, Cook NR. Tests of calibration and goodness-of-fit in the survival setting. Stat Med. 2015;34(10):1659–80.

    Article  Google Scholar 

  29. Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: validating a prognostic model. BMJ (Clin Res Ed). 2009;338:b605.

    Article  Google Scholar 

  30. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38.

    Article  Google Scholar 

  31. Pencina MJ, D’Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23(13):2109–23.

    Article  Google Scholar 

  32. Olchanski N, Cohen JT, Neumann PJ, Wong JB, Kent DM. Understanding the value of individualized information: the impact of poor calibration or discrimination in outcome prediction models. Med Decis Making. 2017;37(7):790–801.

    Article  Google Scholar 

  33. Kähm K, Laxy M, Schneider U, Rogowski WH, Lhachimi SK, Holle R. Health care costs associated with incident complications in patients with type 2 diabetes in Germany. Diabetes Care. 2018;41(5):971–8.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Health Economics and Health Care Management, Helmholtz Zentrum München (GmbH), Neuherberg, Germany

    Michael Laxy, Verena Maria Schöning, Christoph Kurz, Rolf Holle & Katharina Kähm

  2. German Center for Diabetes Research, DZD, Neuherberg-Munich, Germany

    Michael Laxy, Christoph Kurz, Rolf Holle & Katharina Kähm

  3. Institute for Medical Information Processing, Biometrics and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany

    Verena Maria Schöning

  4. Institute of Epidemiology II, Helmholtz Zentrum München (GmbH), Neuherberg, Germany

    Annette Peters & Christa Meisinger

  5. Institute for Biometrics and Epidemiology, German Diabetes Center, Düsseldorf, Germany

    Wolfgang Rathmann

  6. Department of Molecular Epidemiology, German Institute of Human Nutrition, Potsdam, Germany

    Kristin Mühlenbruch

Authors
  1. Michael Laxy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Verena Maria Schöning
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christoph Kurz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rolf Holle
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Annette Peters
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Christa Meisinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wolfgang Rathmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kristin Mühlenbruch
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Katharina Kähm
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ML planned the analysis, supervised analysis of the data, and drafted the manuscript. VMS analyzed the data and was a major contributor to the draft of the manuscript. CK consulted on the data analysis and commented on the manuscript draft. RH consulted on the data analysis, was involved in the data collection, and commented on the manuscript draft. AP, CM, WR, and KM were involved in the data collection and commented on the manuscript draft. KK was a major contributor to the draft of the manuscript.

Corresponding author

Correspondence to Michael Laxy.

Ethics declarations

Conflict of interest

Michael Laxy, Verena M. Schöning, Christoph Kurz, Rolf Holle, Annette Peters, Christa Meisinger, Wolfgang Rathmann, Kristin Mühlenbruch, and Katharina Kähm declare that they have no competing interests.

Funding

The KORA (Cooperative Health Research in the Region of Augsburg) research platform was initiated and financed by the Helmholtz Zentrum München (the German Research Center for Environmental Health), which is funded by the German Federal Ministry of Education and Research and by the State of Bavaria. Furthermore, KORA research was supported within the Munich Center of Health Sciences (MC Health), Ludwig-Maximilians-Universität (LMU), as part of the LMUinnovative.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOC 64 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Laxy, M., Schöning, V.M., Kurz, C. et al. Performance of the UKPDS Outcomes Model 2 for Predicting Death and Cardiovascular Events in Patients with Type 2 Diabetes Mellitus from a German Population-Based Cohort. PharmacoEconomics 37, 1485–1494 (2019). https://doi.org/10.1007/s40273-019-00822-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-019-00822-4

Search

Navigation