Data mining for censored time-to-event data: a Bayesian network model for predicting cardiovascular risk from electronic health record data

Abstract

Models for predicting the risk of cardiovascular (CV) events based on individual patient characteristics are important tools for managing patient care. Most current and commonly used risk prediction models have been built from carefully selected epidemiological cohorts. However, the homogeneity and limited size of such cohorts restrict the predictive power and generalizability of these risk models to other populations. Electronic health data (EHD) from large health care systems provide access to data on large, heterogeneous, and contemporaneous patient populations. The unique features and challenges of EHD, including missing risk factor information, non-linear relationships between risk factors and CV event outcomes, and differing effects from different patient subgroups, demand novel machine learning approaches to risk model development. In this paper, we present a machine learning approach based on Bayesian networks trained on EHD to predict the probability of having a CV event within 5 years. In such data, event status may be unknown for some individuals, as the event time is right-censored due to disenrollment and incomplete follow-up. Since many traditional data mining methods are not well-suited for such data, we describe how to modify both modeling and assessment techniques to account for censored observation times. We show that our approach can lead to better predictive performance than the Cox proportional hazards model (i.e., a regression-based approach commonly used for censored, time-to-event data) or a Bayesian network with ad hoc approaches to right-censoring. Our techniques are motivated by and illustrated on data from a large US Midwestern health care system.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Agaku IT, King BA, Dube SR (2011) Vital signs: current cigarette smoking among adults aged 18 years-united states, 2005–2010. MMWR Morb Mortal Wkly Rep 60(35):1207

    Google Scholar 

  2. Andreassen S, Riekehr C, Kristensen B, Schonheyder H, Leibovici L (1999) Using probabilistic and decision-theoretic methods in treatment and prognosis modeling. Artif Intell Med 15(2):121–134

    Article  Google Scholar 

  3. Assmann G, Cullen P, Schulte H (2002) Simple scoring scheme for calculating the risk of acute coronary events based on the 10-year follow-up of the prospective cardiovascular Munster (PROCAM) study. Circulation 105(7):900–900

    Google Scholar 

  4. Bang H, Tsiatis AA (2000) Estimating medical costs with censored data. Biometrika 87(2):329–343

    MATH  MathSciNet  Article  Google Scholar 

  5. Bang H, Tsiatis AA (2002) Median regression with censored cost data. Biometrics 58(3):643–649

    MATH  MathSciNet  Article  Google Scholar 

  6. Bilmes JA (1998) A gentle tutorial of the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. Technical report. University of California, Berkeley

    Google Scholar 

  7. Blanco R, Inza M, Merino M, Quiroga J, Larranaga P (2005) Feature selection in Bayesian classifiers for the prognosis of survival of cirrhotic patients treated with TIPS. J Biomed Inform 38(5):376–388

    Article  Google Scholar 

  8. Buckley J, James I (1979) Linear-regression with censored data. Biometrika 66(3):429–436

    MATH  Article  Google Scholar 

  9. Clarke PM, Gray AM, Briggs A, Farmer AJ, Fenn P, Stevens RJ, Matthews DR, Stratton IM, Holman RR (2004) A model to estimate the lifetime health outcomes of patients with type 2 diabetes: the United Kingdom Prospective Diabetes Study (UKPDS) outcomes model. Diabetologia 47(10):1747–1759

    Article  Google Scholar 

  10. Collins GS, Altman DG (2009) An independent external validation and evaluation of QRISK cardiovascular risk prediction: a prospective open cohort study. Br Med J 339:b2584

    Article  Google Scholar 

  11. Colombet I, Ruelland A, Chatellier G, Gueyffier F, Degoulet P, Jaulent MC (2000) Models to predict cardiovascular risk: comparison of CART, multilayer perceptron and logistic regression. In: Proceedings of the AMIA symposium, American Medical Informatics Association, pp 156–160

  12. Conroy RM, Pyorala K, Fitzgerald AP, Sans S, Menotti A, DeBacker G, DeBacquer D, Ducimetiere P, Jousilahti P, Keil U, Njolstad I, Oganov RG, Thomsen T, Tunstall-Pedoe H, Tverdal A, Wedel H, Whincup P, Wilhelmsen L, Graham IM (2003) Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 24(11):987–1003

    Article  Google Scholar 

  13. Cook NR, Ridker PM (2009) The use and magnitude of reclassification measures for individual predictors of global cardiovascular risk. Ann Intern Med 150(11):795–802

    Article  Google Scholar 

  14. Cooney MT, Dudina AL, Graham IM (2009) Value and limitations of existing scores for the assessment of cardiovascular risk a review for clinicians. J Am Coll Cardiol 54(14):1209–1227

    Article  Google Scholar 

  15. Cooney MT, Dudina A, D’Agostino R, Graham IM (2010) Cardiovascular risk-estimation systems in primary prevention: do they differ? Do they make a difference? Can we see the future? Circulation 122(3):300–310

    Article  Google Scholar 

  16. Cox DR (1972) Regression models and life-tables. J R Stat Soc A Met 34(2):187–220

    MATH  Google Scholar 

  17. D’Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, Kannel WB (2008) General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 118(4):E86–E86

    Article  Google Scholar 

  18. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B Met 39(1):1–38

    MATH  MathSciNet  Google Scholar 

  19. Fraley C, Raftery AE, Murphy TB, Scrucca L (2012) MCLUST version 4 for R: normal mixture modeling for model-based clustering, classification, and density estimation. Technical report, University of Washington, Department of Statistics

  20. Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Blaha MJ, Dai S, Ford ES, Fox CS, Franco S, Fullerton HJ, Gillespie C, Hailpern SM, Heit JA, Howard VJ, Huffman MD, Judd SE, Kissela BM, Kittner SJ, Lackland DT, Lichtman JH, Lisabeth LD, Mackey RH, Magid DJ, Marcus GM, Marelli A, Matchar DB, McGuire DK, III MER, Moy CS, Mussolino ME, Neumar RW, Nichol G, Pandey DK, Paynter NP, Reeves MJ, Sorlie PD, Stein J, Towfighi A, Turan TN, Virani SS, Wong ND, Woo D, Turner MB (2014) Heart disease and stroke statistics 2014 update: a report from the American Heart Association. Circulation 129(3):E28–E292

  21. Greenwood M et al (1926) A report on the natural duration of cancer. Rep Public Health Med Subj Minist Health 33:1–26

  22. Harrell FE (2001) Regression modeling strategies. Springer, New York

    MATH  Book  Google Scholar 

  23. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P (2007) Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. Br Med J 335(7611):136–141

    Article  Google Scholar 

  24. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Brindle P (2008a) Performance of the qrisk cardiovascular risk prediction algorithm in an independent UK sample of patients from general practice: a validation study. Heart 94(1):34–39

  25. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Minhas R, Sheikh A, Brindle P (2008b) Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. Br Med J 336(7659):1475–1489

  26. Hosmer DW, Lemeshow S (1980) Goodness of fit tests for the multiple logistic regression-model. Commun Stat Theory Methods 9(10):1043–1069

    Article  Google Scholar 

  27. Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS (2008) Random survival forests. Ann Appl Stat 2:841–860

    MATH  MathSciNet  Article  Google Scholar 

  28. Jackson CH, Thompson SG, Sharples LD (2009) Accounting for uncertainty in health economic decision models by using model averaging. J R Stat Soc Ser A Stat Soc 172:383–404

    MathSciNet  Article  Google Scholar 

  29. Kalbfleisch JD, Prentice RL (2002) The statistical analysis of failure time data. Wiley, Hoboken, NJ

    MATH  Book  Google Scholar 

  30. Kattan MW (2003) Comparison of Cox regression with other methods for determining prediction models and nomograms. J Urol 170(6):S6–S9

    Article  Google Scholar 

  31. Kattan MW, Hess KR, Beck JR (1998) Experiments to determine whether recursive partitioning (CART) or an artificial neural network overcomes theoretical limitations of Cox proportional hazards regression. Comput Biomed Res 31(5):363–373

    Article  Google Scholar 

  32. Kazmierska J, Malicki J (2008) Application of the naive Bayesian classifier to optimize treatment decisions. Radiother Oncol 86(2):211–216

    Article  Google Scholar 

  33. Lamon-Fava S, Wilson PW, Schaefer EJ (1996) Impact of body mass index on coronary heart disease risk factors in men and women the framingham offspring study. Arterioscler Thromb Vasc Biol 16(12):1509–1515

    Article  Google Scholar 

  34. Lappenschaar M, Hommersom A, Lucas PJF, Lagro J, Visscher S (2013) Multilevel Bayesian networks for the analysis of hierarchical health care data. Artif Intell Med 57(3):171–183

    Article  Google Scholar 

  35. Larranaga P, Sierra B, Gallego MJ, Michelena MJ, Picaza JM (1997) Learning Bayesian networks by genetic algorithms: a case study in the prediction of survival in malignant skin melanoma. Artif Intell Med 1211:261–272

    Google Scholar 

  36. Law MR, Wald NJ, Rudnicka A (2003) Quantifying effect of statins on low density lipoprotein cholesterol, ischaemic heart disease, and stroke: systematic review and meta-analysis. BMJ 326(7404):1423

    Article  Google Scholar 

  37. Lemeshow S, Hosmer DW (1982) A review of goodness of fit statistics for use in the development of logistic-regression models. Am J Epidemiol 115(1):92–106

    Google Scholar 

  38. Lipsky AM, Lewis RJ (2005) Placing the Bayesian network approach to patient diagnosis in perspective. Ann Emerg Med 45(3):291–294

    Article  Google Scholar 

  39. Lloyd-Jones DM (2010) Cardiovascular risk prediction basic concepts, current status, and future directions. Circulation 121(15):1768–1777

    Article  Google Scholar 

  40. Lucas PJF, Boot H, Taal BG (1998) Computer-based decision support in the management of primary gastric non-Hodgkin lymphoma. Methods Inf Med 37(3):206–219

    Google Scholar 

  41. Lucas PJF, deBruijn NC, Schurink K, Hoepelman A (2000) A probabilistic and decision-theoretic approach to the management of infectious disease at the ICU. Artif Intell Med 19(3):251–279

    Article  Google Scholar 

  42. Lucas PJF, van der Gaag L, Abu-Hanna A (2004) Bayesian networks in biomedicine and health-care. Artif Intell Med 30(3):201–214

    Article  Google Scholar 

  43. Matheny M, McPheeters ML, Glasser A, Mercaldo N, Weaver RB, Jerome RN, Walden R, McKoy JN, Pritchett J, Tsai C (2011) Systematic review of cardiovascular disease risk assessment tools. Technical report, Agency for Healthcare Research and Quality (US)

  44. Murphy KP (1998) Inference and learning in hybrid Bayesian networks. Technical report, University of California, Berkeley, Computer Science Division

  45. Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS (2008) Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 27(2):157–172

    MathSciNet  Article  Google Scholar 

  46. Pencina MJ, D’Agostino RB Sr, Steyerberg EW (2011) Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med 30(1):11–21

    MathSciNet  Article  Google Scholar 

  47. Pepe MS (2011) Problems with risk reclassification methods for evaluating prediction models. Am J Epidemiol 173(11):1327–35

    Article  Google Scholar 

  48. Ridker PM, Buring JE, Rifai N, Cook NR (2007) Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: The Reynolds risk score. J Am Med Assoc 297(13):1433–1433

    Article  Google Scholar 

  49. Ridker PM, Paynter NP, Rifai N, Gaziano M, Cook NR (2008) C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds risk score for men. Circulation 118(18):S1145–S1145

    Google Scholar 

  50. Robins JM, Finkelstein DM (2000) Correcting for noncompliance and dependent censoring in an AIDS clinical trial with inverse probability of censoring weighted (IPCW) log-rank tests. Biometrics 56(3):779–788

    MATH  Article  Google Scholar 

  51. Rockwood MR, Howlett SE (2011) Blood pressure in relation to age and frailty. Can Geriatr J 14(1):2–7

    Google Scholar 

  52. Rotnitzky AG, Robins JM (2004) Inverse probability weighted estimation in survival analysis. In: Armitage P, Colton T (eds) The encyclopedia of biostatistics, 2nd edn. Wiley, Hoboken, NJ

    Google Scholar 

  53. Sarkar S, Koehler J (2013) A dynamic risk score to identify increased risk for heart failure decompensation. IEEE Trans Biomed Eng 60(1):147–150

    Article  Google Scholar 

  54. Sesen MB, Nicholson AE, Banares-Alcantara R, Kadir T, Brady M (2013) Bayesian networks for clinical decision support in lung cancer care. PLOS One 8(12):e82,349

    Article  Google Scholar 

  55. Sierra B, Larranaga P (1998) Predicting survival in malignant skill melanoma using Bayesian networks automatically induced by genetic algorithms: An empirical comparison between different approaches. Artif Intell Med 14(1–2):215–230

    Article  Google Scholar 

  56. Smith WP, Doctor J, Meyer J, Kalet IJ, Phillips MH (2009) A decision aid for intensity-modulated radiation-therapy plan selection in prostate cancer based on a prognostic Bayesian network and a Markov model. Artif Intell Med 46(2):119–130

    Article  Google Scholar 

  57. Song X, Mitnitski A, Cox J, Rockwood K (2004) Comparison of machine learning techniques with classical statistical models in predicting health outcomes. Medinfo 11(1):736–40

    Google Scholar 

  58. Stajduhar I, Dalbelo-Basic B (2010) Learning Bayesian networks from survival data using weighting censored instances. J Biomed Inform 43(4):613–622

    Article  Google Scholar 

  59. Stajduhar I, Dalbelo-Basic B (2012) Uncensoring censored data for machine learning: a likelihood-based approach. Expert Syst Appl 39(8):7226–7234

    Article  Google Scholar 

  60. Stajduhar I, Dalbelo-Basic B, Bogunovic N (2009) Impact of censoring on learning Bayesian networks in survival modelling. Artif Intell Med 47(3):199–217

    Article  Google Scholar 

  61. Stamler J, Stamler R, Neaton JD (1993) Blood pressure, systolic and diastolic, and cardiovascular risks: US population data. Arch Intern Med 153(5):598

    Article  Google Scholar 

  62. Stuart R, Peter N (2003) Artificial intelligence: a modern approach, vol 2. Prentice Hall, Upper Saddle River, NJ

    Google Scholar 

  63. Therneau TM, Grambsch PM, Fleming TR (1990) Martingale-based residuals for survival models. Biometrika 77(1):147–160

    MATH  MathSciNet  Article  Google Scholar 

  64. Tian J, He R, Ram L (2010) Bayesian model averaging using the k-best Bayesian network structures. In: Proceedings of the twenty-sixth conference annual conference on uncertainty in artificial intelligence (UAI-10). AUAI Press, Corvallis, Oregon, pp 589–597

  65. Tsiatis AA (2006) Semiparametric theory and missing data. Springer, New York

    MATH  Google Scholar 

  66. Velikova M, van Scheltinga JT, Lucas PJF, Spaanderman M (2014) Exploiting causal functional relationships in bayesian network modelling for personalised healthcare. Int J Approx Reason 55(1):59–73

    Article  Google Scholar 

  67. Verduijn M, Peek N, Rosseel PMJ, de Jonge E, de Mol BAJM (2007) Prognostic Bayesian networks I: rationale, learning procedure, and clinical use. J Biomed Inform 40(6):609–618

    Article  Google Scholar 

  68. Vila-Frances J, Sanchis J, Soria-Olivas E, Serrano AJ, Martinez-Sober M, Bonanad C, Ventura S (2013) Expert system for predicting unstable angina based on Bayesian networks. Expert Syst Appl 40(12):5004–5010

    Article  Google Scholar 

  69. Woodward M, Brindle P, Tunstall-Pedoe H (2007) Adding social deprivation and family history to cardiovascular risk assessment: the ASSIGN score from the Scottish Heart Health Extended Cohort (SHHEC). Heart 93(2):172–176

    Article  Google Scholar 

  70. Yet B, Bastani K, Raharjo H, Lifvergren S, Marsh W, Bergman B (2013) Decision support system for Warfarin therapy management using Bayesian networks. Decis Support Syst 55(2):488–498

    Article  Google Scholar 

  71. Zupan B, Demsar J, Kattan MW, Beck JR, Bratko I (1999) Machine learning for survival analysis: a case study on recurrence of prostate cancer. Artif Intell Med 1620:346–355

    Google Scholar 

Download references

Acknowledgments

This work was partially supported by NHLBI Grant R01HL102144-01 and AHRQ Grant R21HS017622-01.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sunayan Bandyopadhyay.

Additional information

Responsible editors: Fei Wang, Gregor Stiglic, Ian Davidson and Zoran Obradovic.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bandyopadhyay, S., Wolfson, J., Vock, D.M. et al. Data mining for censored time-to-event data: a Bayesian network model for predicting cardiovascular risk from electronic health record data. Data Min Knowl Disc 29, 1033–1069 (2015). https://doi.org/10.1007/s10618-014-0386-6

Download citation

Keywords

  • Bayesian networks
  • Electronic health data
  • Survival analysis
  • Mining censored data
  • Inverse probability of censoring weights
  • Risk prediction
  • Medical decision support