Intensive Care Medicine

, Volume 40, Issue 3, pp 361–369 | Cite as

Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study

  • M. van den BoogaardEmail author
  • L. Schoonhoven
  • E. Maseda
  • C. Plowright
  • C. Jones
  • A. Luetz
  • P. V. Sackey
  • P. G. Jorens
  • L. M. Aitken
  • F. M. P. van Haren
  • R. Donders
  • J. G. van der Hoeven
  • P. Pickkers



Recalibration and determining discriminative power, internationally, of the existing delirium prediction model (PRE-DELIRIC) for intensive care patients.


A prospective multicenter cohort study was performed in eight intensive care units (ICUs) in six countries. The ten predictors (age, APACHE-II, urgent and admission category, infection, coma, sedation, morphine use, urea level, metabolic acidosis) were collected within 24 h after ICU admission. The confusion assessment method for the intensive care unit (CAM-ICU) was used to identify ICU delirium. CAM-ICU screening compliance and inter-rater reliability measurements were used to secure the quality of the data.


A total of 2,852 adult ICU patients were screened of which 1,824 (64 %) were eligible for the study. Main reasons for exclusion were length of stay <1 day (19.1 %) and sustained coma (4.1 %). CAM-ICU compliance was mean (SD) 82 ± 16 % and inter-rater reliability 0.87 ± 0.17. The median delirium incidence was 22.5 % (IQR 12.8–36.6 %). Although the incidence of all ten predictors differed significantly between centers, the area under the receiver operating characteristic (AUROC) curve of the eight participating centers remained good: 0.77 (95 % CI 0.74–0.79). The linear predictor and intercept of the prediction rule were adjusted and resulted in improved re-calibration of the PRE-DELIRIC model.


In this multinational study, we recalibrated the PRE-DELIRIC model. Despite differences in the incidence of predictors between the centers in the different countries, the performance of the PRE-DELIRIC-model remained good. Following validation of the PRE-DELIRIC model, it may facilitate implementation of strategies to prevent delirium and aid improvements in delirium management of ICU patients.


Delirium Prediction model Recalibration Critical care 



The authors would like to thank Amanda McCairn and Sue Dowling (research nurses, Whiston Hospital), Anna Schandl (PhD student, Karolinska University Hospital, Stockholm) Lena James and Rod Hurford (Princess Alexandra Hospital, Brisbane, Australia) Walter Verbrugghe, Petra Vertongen (MD/staff member and data management, Antwerp University Hospital, Belgium) for their help in collecting the patient data.

Conflicts of interest

The authors declare that they have no competing interests.

Supplementary material

134_2013_3202_MOESM1_ESM.docx (18 kb)
Supplementary material 1 (DOCX 17 kb)


  1. 1.
    Association AP (2013) Diagnostic and statistical manual of mental disorders (DSM-V). American Psychiatric Publishing, ArlingtonGoogle Scholar
  2. 2.
    Dubois MJ, Bergeron N, Dumont M, Dial S, Skrobik Y (2001) Delirium in an intensive care unit: a study of risk factors. Intensive Care Med 27:1297–1304PubMedCrossRefGoogle Scholar
  3. 3.
    Ely EW, Gautam S, Margolin R, Francis J, May L, Speroff T, Truman B, Dittus R, Bernard R, Inouye SK (2001) The impact of delirium in the intensive care unit on hospital length of stay. Intensive Care Med 27:1892–1900PubMedCrossRefGoogle Scholar
  4. 4.
    Ouimet S, Kavanagh BP, Gottfried SB, Skrobik Y (2007) Incidence, risk factors and consequences of ICU delirium. Intensive Care Med 33:66–73PubMedCrossRefGoogle Scholar
  5. 5.
    Milbrandt EB, Deppen S, Harrison PL, Shintani AK, Speroff T, Stiles RA, Truman B, Bernard GR, Dittus RS, Ely EW (2004) Costs associated with delirium in mechanically ventilated patients. Crit Care Med 32:955–962PubMedCrossRefGoogle Scholar
  6. 6.
    Girard TD, Jackson JC, Pandharipande PP, Pun BT, Thompson JL, Shintani AK, Gordon SM, Canonico AE, Dittus RS, Bernard GR, Ely EW (2010) Delirium as a predictor of long-term cognitive impairment in survivors of critical illness. Crit Care Med 38:1513–1520PubMedCentralPubMedCrossRefGoogle Scholar
  7. 7.
    van den Boogaard M, Schoonhoven L, Evers AW, van der Hoeven JG, van Achterberg T, Pickkers P (2012) Delirium in critically ill patients: impact on long-term health-related quality of life and cognitive functioning. Crit Care Med 40:112–118PubMedCrossRefGoogle Scholar
  8. 8.
    van Eijk MM, van den Boogaard M, van Marum RJ, Benner P, Eikelenboom P, Honing ML, van der HB, Horn J, Izaks GJ, Kalf A, Karakus A, Klijn IA, Kuiper MA, De Leeuw FE, De MT, van der Mast RC, Osse RJ, de Rooij SE, Spronk PE, van dV, Van Gool WA, Slooter AJ (2011) Routine use of the confusion assessment method for the intensive care unit: a multicenter study. Am J Respir Crit Care Med 184:340–344PubMedCrossRefGoogle Scholar
  9. 9.
    Reade MC, Eastwood GM, Peck L, Bellomo R, Baldwin I (2011) Routine use of the confusion assessment method for the intensive care unit (CAM-ICU) by bedside nurses may underdiagnose delirium. Crit Care Resusc 13:217–224PubMedGoogle Scholar
  10. 10.
    Ely EW, Inouye SK, Bernard GR, Gordon S, Francis J, May L, Truman B, Speroff T, Gautam S, Margolin R, Hart RP, Dittus R (2001) Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). JAMA 286:2703–2710PubMedCrossRefGoogle Scholar
  11. 11.
    Ely EW, Margolin R, Francis J, May L, Truman B, Dittus R, Speroff T, Gautam S, Bernard GR, Inouye SK (2001) Evaluation of delirium in critically ill patients: validation of the confusion assessment method for the intensive care unit (CAM-ICU). Crit Care Med 29:1370–1379PubMedCrossRefGoogle Scholar
  12. 12.
    van den Boogaard M, Pickkers P, Schoonhoven L (2010) Assessment of delirium in ICU patients; a literature review. Neth J Crit Care 14:10–15Google Scholar
  13. 13.
    Gusmao-Flores D, Figueira Salluh JI, Chalhub RA, Quarantini LC (2012) The confusion assessment method for the intensive care unit (CAM-ICU) and intensive care delirium screening checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studies. Crit Care 16:R115PubMedCentralPubMedCrossRefGoogle Scholar
  14. 14.
    Spronk PE, Riekerk B, Hofhuis J, Rommes JH (2009) Occurrence of delirium is severely underestimated in the ICU during daily care. Intensive Care Med 35:1276–1280PubMedCentralPubMedCrossRefGoogle Scholar
  15. 15.
    Heymann A, Radtke F, Schiemann A, Lutz A, MacGuill M, Wernecke KD, Spies C (2010) Delayed treatment of delirium increases mortality rate in intensive care unit patients. J Int Med Res 38:1584–1595PubMedCrossRefGoogle Scholar
  16. 16.
    van den Boogaard M, Pickkers P, van der Hoeven JG, Roodbol G, van Achterberg T, Schoonhoven L (2009) Implementation of a delirium assessment tool in the ICU can influence haloperidol use. Crit Care 13:R131PubMedCentralPubMedCrossRefGoogle Scholar
  17. 17.
    Inouye SK, Bogardus ST Jr, Charpentier PA, Leo-Summers L, Acampora D, Holford TR, Cooney LM Jr (1999) A multicomponent intervention to prevent delirium in hospitalized older patients. N Engl J Med 340:669–676PubMedCrossRefGoogle Scholar
  18. 18.
    Kalisvaart KJ, de Jonghe JF, Bogaards MJ, Vreeswijk R, Egberts TC, Burger BJ, Eikelenboom P, van Gool WA (2005) Haloperidol prophylaxis for elderly hip-surgery patients at risk for delirium: a randomized placebo-controlled study. J Am Geriatr Soc 53:1658–1666PubMedCrossRefGoogle Scholar
  19. 19.
    Wang W, Li HL, Wang DX, Zhu X, Li SL, Yao GQ, Chen KS, Gu XE, Zhu SN (2012) Haloperidol prophylaxis decreases delirium incidence in elderly patients after noncardiac surgery: a randomized controlled trial. Crit Care Med 40:1–9Google Scholar
  20. 20.
    van den Boogaard M, Pickkers P, Slooter AJ, Kuiper MA, Spronk PE, van der Voort PHJ, van der Hoeven JG, Donders R, van AT, Schoonhoven L (2012) Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICU patients) delirium prediction model for intensive care patients: observational multicentre study. BMJ 344:e420PubMedCentralPubMedCrossRefGoogle Scholar
  21. 21.
    Van Rompaey B, Elseviers MM, Schuurmans MJ, Shortridge-Baggett LM, Truijen S, Bossaert L (2009) Risk factors for delirium in intensive care patients: a prospective cohort study. Crit Care 13:R77PubMedCentralPubMedCrossRefGoogle Scholar
  22. 22.
    van den Boogaard M, Schoonhoven L, Van Achterberg T, Van der Hoeven JG, Pickkers P (2013) Haloperidol prophylaxis in critically ill patients with a high risk for delirium. Crit Care 17:R9PubMedCrossRefGoogle Scholar
  23. 23.
    Harrell Jr FE (2001) Regression modeling strategies. With applications to linear models, logistic regression, and survival analysis. Springer, New YorkGoogle Scholar
  24. 24.
    Steyerberg EW, Eijkemans MJ, Harrell FE Jr, Habbema JD (2001) Prognostic modeling with logistic regression analysis: in search of a sensible strategy in small data sets. Med Decis Mak Int J Soc Med Decis Mak 21:45–56CrossRefGoogle Scholar
  25. 25.
    Lemeshow S, Hosmer DW Jr (1982) A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 115:92–106PubMedGoogle Scholar
  26. 26.
    Finazzi S, Poole D, Luciani D, Cogo PE, Bertolini G (2011) Calibration belt for quality-of-care assessment based on dichotomous outcomes. PLoS ONE 6:e16110PubMedCentralPubMedCrossRefGoogle Scholar
  27. 27.
    Team RDC (2009) R: a language and environment for statistical computing. R Foundation for Statistical Computing, ViennaGoogle Scholar
  28. 28.
    Harrell FE, Jr (2012) Hmisc: Harrell Miscellaneous (R package version 3.9–3). Contributions from many other users. Available at
  29. 29.
    Pandharipande P, Ely EW (2006) Sedative and analgesic medications: risk factors for delirium and sleep disturbances in the critically ill. Crit Care Clin 22:313–327PubMedCrossRefGoogle Scholar
  30. 30.
    Roberts DJ, Haroon B, Hall RI (2012) Sedation for critically ill or injured adults in the intensive care unit: a shifting paradigm. Drugs 72:1881–1916PubMedCrossRefGoogle Scholar
  31. 31.
    Awissi DK, Lebrun G, Coursin DB, Riker RR, Skrobik Y (2013) Alcohol withdrawal and delirium tremens in the critically ill: a systematic review and commentary. Intensive Care Med 39:16–30PubMedCrossRefGoogle Scholar
  32. 32.
    Reynolds T, Cooke F, Murch N (2012) Problem based review: alcohol-use disorders on the acute medical unit. Acute Med 11:101–106PubMedGoogle Scholar
  33. 33.
    Riker RR, Shehabi Y, Bokesch PM, Ceraso D, Wisemandle W, Koura F, Whitten P, Margolis BD, Byrne DW, Ely EW, Rocha MG (2009) Dexmedetomidine vs midazolam for sedation of critically ill patients: a randomized trial. JAMA 301:489–499PubMedCrossRefGoogle Scholar
  34. 34.
    Pandharipande PP, Pun BT, Herr DL, Maze M, Girard TD, Miller RR, Shintani AK, Thompson JL, Jackson JC, Deppen SA, Stiles RA, Dittus RS, Bernard GR, Ely EW (2007) Effect of sedation with dexmedetomidine vs lorazepam on acute brain dysfunction in mechanically ventilated patients: the MENDS randomized controlled trial. JAMA 298:2644–2653PubMedCrossRefGoogle Scholar
  35. 35.
    Hughes CG, McGrane S, Pandharipande PP (2012) Sedation in the intensive care setting. Clin Pharmacol: Adv Appl 4:53–63Google Scholar
  36. 36.
    Riker R, Shehabi Y, Wisemandle W, Rocha M (2012) Relationship between delirium iIncidence assessed with CAM-ICU and level of sedation assessed by RASS in adult ICU Patients. Crit Care Med. doi:  10.1097/01.ccm.0000425304.19686.5d
  37. 37.
    Donders AR, van der Heijden GJ, Stijnen T, Moons KG (2006) Review: a gentle introduction to imputation of missing values. J Clin Epidemiol 59:1087–1091PubMedCrossRefGoogle Scholar
  38. 38.
    Haenggi M, Blum S, Brechbuehl R, Brunello A, Jakob SM, Takala J (2013) Effect of sedation level on the prevalence of delirium when assessed with CAM-ICU and ICDSC. Intensive Care Med 39:2171–2179PubMedCrossRefGoogle Scholar
  39. 39.
    Devlin JW, Fraser GL, Joffe AM, Riker RR, Skrobik Y (2013) The accurate recognition of delirium in the ICU: the emperor’s new clothes? Intensive Care Med 39:2196–2199Google Scholar
  40. 40.
    Vasilevskis EE, Morandi A, Boehm L, Pandharipande PP, Girard TD, Jackson JC, Thompson JL, Shintani A, Gordon SM, Pun BT, Wesley EE (2011) Delirium and sedation recognition using validated instruments: reliability of bedside intensive care unit nursing assessments from 2007 to 2010. J Am Geriatr Soc 59(Suppl 2):S249–S255PubMedCentralPubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and ESICM 2014

Authors and Affiliations

  • M. van den Boogaard
    • 1
    Email author
  • L. Schoonhoven
    • 2
    • 3
  • E. Maseda
    • 4
  • C. Plowright
    • 5
  • C. Jones
    • 6
  • A. Luetz
    • 7
  • P. V. Sackey
    • 8
  • P. G. Jorens
    • 9
  • L. M. Aitken
    • 10
    • 11
  • F. M. P. van Haren
    • 12
  • R. Donders
    • 13
  • J. G. van der Hoeven
    • 1
  • P. Pickkers
    • 1
  1. 1.Department of Intensive Care MedicineRadboud University Medical CenterNijmegenThe Netherlands
  2. 2.Scientific Institute for Quality of HealthcareRadboud University Medical CenterNijmegenThe Netherlands
  3. 3.Faculty of Health SciencesUniversity of SouthamptonSouthamptonUK
  4. 4.Department of Intensive Care MedicineHospital Universitario La PazMadridSpain
  5. 5.Anaesthetic DepartmentMedway Maritime HospitalKentUK
  6. 6.Ward 4E (Critical Care)Whiston HospitalPrescotUK
  7. 7.Department of Anesthesiology and Intensive Care MedicineCharité—Universitaetsmedizin BerlinBerlinGermany
  8. 8.Departments of Anesthesiology, Surgical Services and Intensive Care Medicine, Physiology and PharmacologyKarolinska Institute, Karolinska University Hospital SolnaStockholmSweden
  9. 9.Department of Critical Care MedicineAntwerp University Hospital, University of AntwerpEdegem (Antwerp)Belgium
  10. 10.Intensive Care UnitPrincess Alexandra HospitalBrisbaneAustralia
  11. 11.NHMRC Center for Research Excellence, Center for Health Practice InnovationGriffith Health Institute, Griffith UniversityBrisbaneAustralia
  12. 12.Department of Intensive CareCanberra HospitalCanberraAustralia
  13. 13.Department for Health EvidenceRadboud University Medical CenterNijmegenThe Netherlands

Personalised recommendations