, Volume 36, Issue 9, pp 1125–1134 | Cite as

Novel Risk Engine for Diabetes Progression and Mortality in USA: Building, Relating, Assessing, and Validating Outcomes (BRAVO)

  • Hui Shao
  • Vivian Fonseca
  • Charles Stoecker
  • Shuqian Liu
  • Lizheng ShiEmail author
Original Research Article



There is an urgent need to update diabetes prediction, which has relied on the United Kingdom Prospective Diabetes Study (UKPDS) that dates back to 1970 s’ European populations.


The objective of this study was to develop a risk engine with multiple risk equations using a recent patient cohort with type 2 diabetes mellitus reflective of the US population.


A total of 17 risk equations for predicting diabetes-related microvascular and macrovascular events, hypoglycemia, mortality, and progression of diabetes risk factors were estimated using the data from the Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial (n = 10,251). Internal and external validation processes were used to assess performance of the Building, Relating, Assessing, and Validating Outcomes (BRAVO) risk engine. One-way sensitivity analysis was conducted to examine the impact of risk factors on mortality at the population level.


The BRAVO risk engine added several risk factors including severe hypoglycemia and common US racial/ethnicity categories compared with the UKPDS risk engine. The BRAVO risk engine also modeled mortality escalation associated with intensive glycemic control (i.e., glycosylated hemoglobin < 6.5%). External validation showed a good prediction power on 28 endpoints observed from other clinical trials (slope = 1.071, R2 = 0.86).


The BRAVO risk engine for the US diabetes cohort provides an alternative to the UKPDS risk engine. It can be applied to assist clinical and policy decision making such as cost-effective resource allocation in USA.


Author Contributions

Hui Shao analyzed the data for developing the BRAOVO risk engine with both internal and external validation and drafted the manuscript. Vivian Fonseca and Shuqian Liu provided clinical interpretation to support the BRAVO risk engine. Charles Stoecker reviewed the econometrics in the risk engine during the model development. Lizheng Shi as the principal investigator initiated the project and worked with Hui Shao in developing the BRAVO risk engine and the manuscript. All authors are extensively involved in writing the manuscript.

Compliance with Ethical Standards


No sources of funding were received for the preparation of this article.

Conflict of interest

Hui Shao, Vivian Fonseca, Charles Stoecker, Shuqian Liu, and Lizheng Shi have no conflicts of interest directly relevant to the content of this article.

Data availability

The datasets used for this study are publicly available and can be requested through the National Heart, Lung, and Blood Institute [42].

Supplementary material

40273_2018_662_MOESM1_ESM.docx (5.1 mb)
Supplementary material 1 (DOCX 5209 kb)


  1. 1.
    American Diabetes Association. Economic costs of diabetes in the U.S. in 2007. Diabetes Care. 2008;31(3):596–615.CrossRefGoogle Scholar
  2. 2.
    Menke A, Casagrande S, Geiss L, et al. Prevalence of and trends in diabetes among adults in the United States, 1988-2012. JAMA. 2015;314(10):1021–9.CrossRefPubMedGoogle Scholar
  3. 3.
    Boyle JP, Thompson TJ, Gregg EW, et al. Projection of the year 2050 burden of diabetes in the US adult population: dynamic modeling of incidence, mortality, and prediabetes prevalence. Popul Health Metr. 2010;8(1):29.CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    American Diabetes Association. Economic costs of diabetes in the US in 2012. Diabetes Care. 2013;36(4):1033–46.CrossRefPubMedCentralGoogle Scholar
  5. 5.
    Uusitupa M, Siitonen O, Aro A, et al. Prevalence of coronary heart disease, left ventricular failure and hypertension in middle-aged, newly diagnosed type 2 (non-insulin-dependent) diabetic subjects. Diabetologia. 1985;28(1):22–7.CrossRefPubMedGoogle Scholar
  6. 6.
    Fowler MJ. Microvascular and macrovascular complications of diabetes. Clin Diabetes. 2008;26(2):77–82.CrossRefGoogle Scholar
  7. 7.
    Palmer AJ, M.H.M. Group. Computer modeling of diabetes and its complications: a report on the Fifth Mount Hood challenge meeting. Value Health. 2013;16(4):670–85.CrossRefPubMedGoogle Scholar
  8. 8.
    McEwen LN, Karter AJ, Waitzfelder BE, et al. Predictors of mortality over 8 years in type 2 diabetic patients: Translating Research Into Action for Diabetes (TRIAD). Diabetes Care. 2012;35(6):1301–9.CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    De Cosmo S, Copetti M, Lamacchia O, et al. Development and validation of a predicting model of all-cause mortality in patients with type 2 diabetes. Diabetes Care. 2013;36(9):2830–5.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    McEwan P, Bennett H, Ward T, et al. Refitting of the UKPDS 68 risk equations to contemporary routine clinical practice data in the UK. Pharmacoeconomics. 2015;33(2):149–61.CrossRefPubMedGoogle Scholar
  11. 11.
    Erhardt W, Bergenheim K, Duprat-Lomon I, et al. Cost effectiveness of saxagliptin and metformin versus sulfonylurea and metformin in the treatment of type 2 diabetes mellitus in Germany: a Cardiff diabetes model analysis. Clin Drug Investig. 2012;32(3):189–202.CrossRefPubMedGoogle Scholar
  12. 12.
    Schwarz B, Gouveia M, Chen J, et al. Cost-effectiveness of sitagliptin-based treatment regimens in European patients with type 2 diabetes and haemoglobin A1c above target on metformin monotherapy. Diabetes Obes Metab. 2008;10(Suppl. 1):43–55.CrossRefPubMedGoogle Scholar
  13. 13.
    Shafie AA, Gupta V, Baabbad R, et al. An analysis of the short- and long-term cost-effectiveness of starting biphasic insulin aspart 30 in insulin-naive people with poorly controlled type 2 diabetes. Diabetes Res Clin Pract. 2014;106(2):319–27.CrossRefPubMedGoogle Scholar
  14. 14.
    Palmer AJ, Roze S, Valentine WJ, et al. Impact of changes in HbA1c, lipids and blood pressure on long-term outcomes in type 2 diabetes patients: an analysis using the CORE Diabetes Model. Curr Med Res Opin. 2004;20(S1):S53–8.CrossRefPubMedGoogle Scholar
  15. 15.
    Charokopou M, McEwan P, Lister S, et al. The cost-effectiveness of dapagliflozin versus sulfonylurea as an add-on to metformin in the treatment of type 2 diabetes mellitus. Diabet Med. 2015;32(7):890–8.CrossRefPubMedGoogle Scholar
  16. 16.
    Elgart JF, Caporale JE, Gonzalez L, et al. Treatment of type 2 diabetes with saxagliptin: a pharmacoeconomic evaluation in Argentina. Health Econ Rev. 2013;3(1):11.CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Hoerger TJ, Zhang P, Segel JE, et al. Cost-effectiveness of bariatric surgery for severely obese adults with diabetes. Diabetes Care. 2010;33(9):1933–9.CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    CDC, Diabetes Cost-Effectiveness Group. Cost-effectiveness of intensive glycemic control, intensified hypertension control, and serum cholesterol level reduction for type 2 diabetes. JAMA. 2002;287(19):2542–51.CrossRefGoogle Scholar
  19. 19.
    Hoerger TJ, Zhang P, Segel JE, et al. Improvements in risk factor control among persons with diabetes in the United States: evidence and implications for remaining life expectancy. Diabetes Res Clin Pract. 2009;86(3):225–32.CrossRefPubMedGoogle Scholar
  20. 20.
    Clarke P, Gray A, Briggs A, et al. A model to estimate the lifetime health outcomes of patients with type 2 diabetes: the United Kingdom Prospective Diabetes Study (UKPDS) Outcomes Model (UKPDS no. 68). Diabetologia. 2004;47(10):1747–59.CrossRefPubMedGoogle Scholar
  21. 21.
    Gerstein HC, Riddle MC, Kendall DM, et al. Glycemia treatment strategies in the Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial. Am J Cardiol. 2007;99(12A):34i–43i.CrossRefPubMedGoogle Scholar
  22. 22.
    Baser OHA, Li L, Wang L. Obese patients in the veteran population in the united states: a health care cost and utilization analysis. Value Health. 2013;16(3):A110.CrossRefGoogle Scholar
  23. 23.
    Buse JB. Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial: design and methods. Am J Cardiol. 2007;99(12):S21–33.CrossRefGoogle Scholar
  24. 24.
    Buse JB, A.S. Group. Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial: design and methods. Am J Cardiol. 2007;99(12):S21–33.CrossRefGoogle Scholar
  25. 25.
    Barhak J, Isaman DJ, Ye W, et al. Chronic disease modeling and simulation software. J Biomed Inform. 2010;43(5):791–9.CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    Zhou H, Isaman DJM, Messinger S, et al. A computer simulation model of diabetes progression, quality of life, and cost. Diabetes Care. 2005;28(12):2856–63.CrossRefPubMedGoogle Scholar
  27. 27.
    Fishman G. Discrete-event simulation: modeling, programming, and analysis. Berlin: Springer Science & Business Media; 2013.Google Scholar
  28. 28.
    Gerds TA, Schumacher M. Consistent estimation of the expected Brier score in general survival models with right-censored event times. Biom J. 2006;48(6):1029–40.CrossRefPubMedGoogle Scholar
  29. 29.
    Subramanian J, Simon R. Overfitting in prediction models: is it a problem only in high dimensions? Contemp Clin Trials. 2013;36(2):636–41.CrossRefPubMedGoogle Scholar
  30. 30.
    Mount H. Computer modeling of diabetes and its complications: a report on the Fourth Mount Hood Challenge Meeting. Diabetes Care. 2007;30(6):1638.CrossRefGoogle Scholar
  31. 31.
    Hayes A, Leal J, Gray A, et al. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56(9):1925–33.CrossRefPubMedGoogle Scholar
  32. 32.
    Caro JJ, Möller J, Getsios D. Discrete event simulation: the preferred technique for health economic evaluations? Value Health. 2010;13(8):1056–60.CrossRefPubMedGoogle Scholar
  33. 33.
    Gerstein HC, Miller ME, Ismail-Beigi F, et al. Effects of intensive glycaemic control on ischaemic heart disease: analysis of data from the randomised, controlled ACCORD trial. Lancet. 2014;384(9958):1936–41.CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Ismail-Beigi F, Craven T, Banerji MA, et al. Effect of intensive treatment of hyperglycaemia on microvascular outcomes in type 2 diabetes: an analysis of the ACCORD randomised trial. Lancet. 2010;376(9739):419–30.CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Zoungas S, Patel A, Chalmers J, et al. Severe hypoglycemia and risks of vascular events and death. N Engl J Med. 2010;363(15):1410–8.CrossRefPubMedGoogle Scholar
  36. 36.
    Zhao Y, Campbell CR, Fonseca V, et al. Impact of hypoglycemia associated with antihyperglycemic medications on vascular risks in veterans with type 2 diabetes. Diabetes Care. 2012;35(5):1126–32.CrossRefPubMedPubMedCentralGoogle Scholar
  37. 37.
    Shi L, Shao H, Zhao Y, et al. Is hypoglycemia fear independently associated with health-related quality of life? Health Qual Life Outcomes. 2014;12(1):167.CrossRefPubMedPubMedCentralGoogle Scholar
  38. 38.
    Karter AJ, Ferrara A, Liu JY, et al. Ethnic disparities in diabetic complications in an insured population. JAMA. 2002;287(19):2519–27.CrossRefPubMedGoogle Scholar
  39. 39.
    Riddle MC, Ambrosius WT, Brillon DJ, et al. Epidemiologic relationships between A1C and all-cause mortality during a median 3.4-year follow-up of glycemic treatment in the ACCORD trial. Diabetes Care. 2010;33(5):983–90.CrossRefPubMedPubMedCentralGoogle Scholar
  40. 40.
    Jackson CH. flexsurv: a platform for parametric survival modelling in R. J Stat Softw. 2016;70(8):1–33.CrossRefGoogle Scholar
  41. 41.
    Basu S, Sussman JB, Berkowitz SA, et al. Development and validation of Risk Equations for Complications Of type 2 Diabetes (RECODe) using individual participant data from randomised trials. Lancet Diabetes Endocrinol. 2017;5(10):788–98.CrossRefPubMedPubMedCentralGoogle Scholar
  42. 42.
    National Heart Lung and Blood Institute. Action to Control Cardiovascular Risk in Diabetes (ACCORD) data. Available from: Accessed 9 Apr 2018.

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Hui Shao
    • 1
  • Vivian Fonseca
    • 2
  • Charles Stoecker
    • 1
  • Shuqian Liu
    • 1
  • Lizheng Shi
    • 1
    Email author
  1. 1.Department of Global Health Management and Policy, School of Public Health and Tropical MedicineTulane UniversityNew OrleansUSA
  2. 2.School of MedicineTulane UniversityNew OrleansUSA

Personalised recommendations