International Journal of Clinical Pharmacy

, Volume 38, Issue 4, pp 915–923 | Cite as

The development of an automated ward independent delirium risk prediction model

  • Hugo A. J. M. de WitEmail author
  • Bjorn Winkens
  • Carlota Mestres Gonzalvo
  • Kim P. G. M. Hurkens
  • Wubbo J. Mulder
  • Rob Janknegt
  • Frans R. Verhey
  • Paul-Hugo M. van der Kuy
  • Jos M. G. A. Schols
Research Article


Background A delirium is common in hospital settings resulting in increased mortality and costs. Prevention of a delirium is clearly preferred over treatment. A delirium risk prediction model can be helpful to identify patients at risk of a delirium, allowing the start of preventive treatment. Current risk prediction models rely on manual calculation of the individual patient risk. Objective The aim of this study was to develop an automated ward independent delirium riskprediction model. To show that such a model can be constructed exclusively from electronically available risk factors and thereby implemented into a clinical decision support system (CDSS) to optimally support the physician to initiate preventive treatment. Setting A Dutch teaching hospital. Methods A retrospective cohort study in which patients, 60 years or older, were selected when admitted to the hospital, with no delirium diagnosis when presenting, or during the first day of admission. We used logistic regression analysis to develop a delirium predictive model out of the electronically available predictive variables. Main outcome measure A delirium risk prediction model. Results A delirium risk prediction model was developed using predictive variables that were significant in the univariable regression analyses. The area under the receiver operating characteristics curve of the “medication model” model was 0.76 after internal validation. Conclusions CDSSs can be used to automatically predict the risk of a delirium in individual hospitalised patients’ by exclusively using electronically available predictive variables. To increase the use and improve the quality of predictive models, clinical risk factors should be documented ready for automated use.


Automation Decision support systems Decision support techniques Delirium Hospital Predicting 



We thank Eugene Hardy for aiding in the retrieval of the delirium diagnosis dates.


The SCREEN-study was supported by a grant from ZonMW, Grant Number 113101001.

Conflicts of interest

The authors declare no conflict of interest.


  1. 1.
    Inouye SK, Westendorp RG, Saczynski JS. Delirium in elderly people. Lancet. 2014;383(9920):911–22.CrossRefPubMedGoogle Scholar
  2. 2.
    Siddiqi N, House AO, Holmes JD. Occurrence and outcome of delirium in medical in-patients: a systematic literature review. Age Ageing. 2006;35(4):350–64.CrossRefPubMedGoogle Scholar
  3. 3.
    Leslie DL, Marcantonio ER, Zhang Y, Leo-Summers L, Inouye SK. One-year health care costs associated with delirium in the elderly population. Arch Intern Med. 2008;168(1):27–32.CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Salluh JI, Wang H, Schneider EB, Nagaraja N, Yenokyan G, Damluji A, et al. Outcome of delirium in critically ill patients: systematic review and meta-analysis. BMJ. 2015;350:h2538.CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Inouye SK, Bogardus ST Jr, Charpentier PA, Leo-Summers L, Acampora D, Holford TR, et al. A multicomponent intervention to prevent delirium in hospitalized older patients. N Engl J Med. 1999;340(9):669–76.CrossRefPubMedGoogle Scholar
  6. 6.
    Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941–8.CrossRefPubMedGoogle Scholar
  7. 7.
    Trzepacz PT, Mittal D, Torres R, Kanary K, Norton J, Jimerson N. Validation of the delirium rating scale-revised-98: comparison with the delirium rating scale and the cognitive test for delirium. J Neuropsychiatry Clin Neurosci. 2001;13(2):229–42.CrossRefPubMedGoogle Scholar
  8. 8.
    Schuurmans MJ, Shortridge-Baggett LM, Duursma SA. The Delirium Observation Screening Scale: a screening instrument for delirium. Res Theory Nurs Pract. 2003;17(1):31–50.CrossRefPubMedGoogle Scholar
  9. 9.
    Young J, Leentjens AF, George J, Olofsson B, Gustafson Y. Systematic approaches to the prevention and management of patients with delirium. J Psychosom Res. 2008;65(3):267–72.CrossRefPubMedGoogle Scholar
  10. 10.
    Inouye SK. Delirium in older persons. N Engl J Med. 2006;354(11):1157–65.CrossRefPubMedGoogle Scholar
  11. 11.
    Teslyar P, Stock VM, Wilk CM, Camsari U, Ehrenreich MJ, Himelhoch S. Prophylaxis with antipsychotic medication reduces the risk of post-operative delirium in elderly patients: a meta-analysis. Psychosomatics. 2013;54(2):124–31.CrossRefPubMedGoogle Scholar
  12. 12.
    Kalisvaart KJ, de Jonghe JF, Bogaards MJ, Vreeswijk R, Egberts TC, Burger BJ, et al. Haloperidol prophylaxis for elderly hip-surgery patients at risk for delirium: a randomized placebo-controlled study. J Am Geriatr Soc. 2005;53(10):1658–66.CrossRefPubMedGoogle Scholar
  13. 13.
    Young J, Murthy L, Westby M, Akunne A, O’Mahony R. Guideline development G. Diagnosis, prevention, and management of delirium: summary of nice guidance. BMJ. 2010;341:c3704.CrossRefPubMedGoogle Scholar
  14. 14.
    Laurila JV, Laakkonen ML, Tilvis RS, Pitkala KH. Predisposing and precipitating factors for delirium in a frail geriatric population. J Psychosom Res. 2008;65(3):249–54.CrossRefPubMedGoogle Scholar
  15. 15.
    Inouye SK. Predisposing and precipitating factors for delirium in hospitalized older patients. Dement Geriatr Cogn Disord. 1999;10(5):393–400.CrossRefPubMedGoogle Scholar
  16. 16.
    van den Boogaard M, Pickkers P, Slooter AJ, Kuiper MA, Spronk PE, van der Voort PH, et al. Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for intensive care patients: observational multicentre study. BMJ. 2012;344:e420.CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Inouye SK, Viscoli CM, Horwitz RI, Hurst LD, Tinetti ME. A predictive model for delirium in hospitalized elderly medical patients based on admission characteristics. Ann Intern Med. 1993;119(6):474–81.CrossRefPubMedGoogle Scholar
  18. 18.
    Carrasco MP, Villarroel L, Andrade M, Calderon J, Gonzalez M. Development and validation of a delirium predictive score in older people. Age Ageing. 2014;43(3):346–51.CrossRefPubMedGoogle Scholar
  19. 19.
    Douglas VC, Hessler CS, Dhaliwal G, Betjemann JP, Fukuda KA, Alameddine LR, et al. The AWOL tool: derivation and validation of a delirium prediction rule. J Hosp Med. 2013;8(9):493–9.CrossRefPubMedGoogle Scholar
  20. 20.
    de Wit HA, Mestres Gonzalvo C, Hurkens KP, Mulder WJ, Janknegt R, Verhey FR, et al. Development of a computer system to support medication reviews in nursing homes. Int J Clin Pharm. 2013;35(5):668–72.Google Scholar
  21. 21.
    Lin Y, Chen J, Wang Z. Meta-analysis of factors which influence delirium following cardiac surgery. J Card Surg. 2012;27(4):481–92.CrossRefPubMedGoogle Scholar
  22. 22.
    Smulter N, Lingehall HC, Gustafson Y, Olofsson B, Engstrom KG. Delirium after cardiac surgery: incidence and risk factors. Interact CardioVasc Thorac Surg. 2013;17(5):790–6.CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    World Health Organisation. [updated 2011-03-25; cited 2015-12-27].
  24. 24.
    Maher RL, Hanlon J, Hajjar ER. Clinical consequences of polypharmacy in elderly. Expert Opin Drug Saf. 2014;13(1):57–65.CrossRefPubMedGoogle Scholar
  25. 25.
    Hajjar ER, Cafiero AC, Hanlon JT. Polypharmacy in elderly patients. Am J Geriatr Pharmacother. 2007;5(4):345–51.CrossRefPubMedGoogle Scholar
  26. 26.
    Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–9.CrossRefPubMedGoogle Scholar
  27. 27.
    Fan J, Upadhye S, Worster A. Understanding Receiver Operating Characteristic (ROC) Curves. Cjem. 2006;8(1):19–20.PubMedGoogle Scholar
  28. 28.
    Harrell FE. Regression modeling strategies : with applications to linear models, logistic regression, and survival analysis. New York: Springer; 2001; xxii, 568 p.Google Scholar
  29. 29.
    Herr M, Robine JM, Pinot J, Arvieu JJ, Ankri J. Polypharmacy and frailty: prevalence, relationship, and impact on mortality in a French sample of 2350 old people. Pharmacoepidemiol Drug Saf. 2015;24(6):637–46.CrossRefPubMedGoogle Scholar
  30. 30.
    Clegg A, Young J, Iliffe S, Rikkert MO, Rockwood K. Frailty in elderly people. Lancet. 2013;381(9868):752–62.CrossRefPubMedGoogle Scholar
  31. 31.
    Hein C, Forgues A, Piau A, Sommet A, Vellas B, Nourhashémi F. Impact of polypharmacy on occurrence of delirium in elderly emergency patients. J Am Med Dir Assoc. 2014;15(11):850.e11–5.CrossRefGoogle Scholar
  32. 32.
    Stusser RJ, Dickey RA. Quality and cost improvement of healthcare via complementary measurement and diagnosis of patient general health outcome using electronic health record data: research rationale and design. J Med Syst. 2013;37(6):9977.CrossRefPubMedGoogle Scholar
  33. 33.
    Minne L, Ludikhuize J, de Rooij SE, Abu-Hanna A. Characterizing predictive models of mortality for older adults and their validation for use in clinical practice. J Am Geriatr Soc. 2011;59(6):1110–5.CrossRefPubMedGoogle Scholar
  34. 34.
    Soman S, Zasuwa G, Yee J. Automation, decision support, and expert systems in nephrology. Adv Chronic Kidney Dis. 2008;15(1):42–55.CrossRefPubMedGoogle Scholar
  35. 35.
    Maviglia SM, Zielstorff RD, Paterno M, Teich JM, Bates DW, Kuperman GJ. Automating complex guidelines for chronic disease: lessons learned. J Am Med Inform Assoc. 2003;10(2):154–65.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer International Publishing 2016

Authors and Affiliations

  • Hugo A. J. M. de Wit
    • 1
    Email author
  • Bjorn Winkens
    • 2
  • Carlota Mestres Gonzalvo
    • 3
  • Kim P. G. M. Hurkens
    • 4
  • Wubbo J. Mulder
    • 5
  • Rob Janknegt
    • 3
  • Frans R. Verhey
    • 6
  • Paul-Hugo M. van der Kuy
    • 3
  • Jos M. G. A. Schols
    • 7
  1. 1.Department of Clinical PharmacyZuyderland Medical CentreHeerlenThe Netherlands
  2. 2.Department of Methodology and Statistics, CAPHRI-School for Public Health and Primary CareMaastricht UniversityMaastrichtThe Netherlands
  3. 3.Department of Clinical PharmacyZuyderland Medical CentreSittard-GeleenThe Netherlands
  4. 4.Section of Geriatric Medicine, Department of Internal MedicineZuyderland Medical CentreHeerlenThe Netherlands
  5. 5.Department of Internal MedicineMaastricht University Medical CentreMaastrichtThe Netherlands
  6. 6.Department of Psychiatry and Neuropsychology, Alzheimer Centrum Limburg/School for Mental Health and NeurosciencesMaastricht UniversityMaastrichtThe Netherlands
  7. 7.Department of General Practice and Department of Health Services Research, CAPHRI-School for Public Health and Primary CareMaastricht UniversityMaastrichtThe Netherlands

Personalised recommendations