Intensive Care Medicine

, Volume 41, Issue 6, pp 1048–1056 | Cite as

Multinational development and validation of an early prediction model for delirium in ICU patients

  • A. Wassenaar
  • M. van den BoogaardEmail author
  • T. van Achterberg
  • A. J. C. Slooter
  • M. A. Kuiper
  • M. E. Hoogendoorn
  • K. S. Simons
  • E. Maseda
  • N. Pinto
  • C. Jones
  • A. Luetz
  • A. Schandl
  • W. Verbrugghe
  • L. M. Aitken
  • F. M. P. van Haren
  • A. R. T. Donders
  • L. Schoonhoven
  • P. Pickkers
Open Access



Delirium incidence in intensive care unit (ICU) patients is high and associated with poor outcome. Identification of high-risk patients may facilitate its prevention.


To develop and validate a model based on data available at ICU admission to predict delirium development during a patient’s complete ICU stay and to determine the predictive value of this model in relation to the time of delirium development.


Prospective cohort study in 13 ICUs from seven countries. Multiple logistic regression analysis was used to develop the early prediction (E-PRE-DELIRIC) model on data of the first two-thirds and validated on data of the last one-third of the patients from every participating ICU.


In total, 2914 patients were included. Delirium incidence was 23.6 %. The E-PRE-DELIRIC model consists of nine predictors assessed at ICU admission: age, history of cognitive impairment, history of alcohol abuse, blood urea nitrogen, admission category, urgent admission, mean arterial blood pressure, use of corticosteroids, and respiratory failure. The area under the receiver operating characteristic curve (AUROC) was 0.76 [95 % confidence interval (CI) 0.73–0.77] in the development dataset and 0.75 (95 % CI 0.71–0.79) in the validation dataset. The model was well calibrated. AUROC increased from 0.70 (95 % CI 0.67–0.74), for delirium that developed <2 days, to 0.81 (95 % CI 0.78–0.84), for delirium that developed >6 days.


Patients’ delirium risk for the complete ICU length of stay can be predicted at admission using the E-PRE-DELIRIC model, allowing early preventive interventions aimed to reduce incidence and severity of ICU delirium.


Intensive care unit Delirium Clinical prediction Adult 


Delirium is an acute organic brain dysfunction characterised by disturbances of attention and cognition with a fluctuating course, as a direct consequence of an underlying medical condition [1]. Delirium occurs frequently in intensive care unit (ICU) patients [2, 3], and is associated with poor outcome [3, 4, 5, 6].

Systematic delirium assessment in ICU patients is important to deliver adequate patient care by allowing clinicians to detect and treat delirium at an early stage [7, 8, 9]. Several promising interventions to prevent delirium are available that target cognitive impairment, sleep deprivation, immobility, and visual and hearing impairment [10, 11, 12, 13, 14, 15]. Although one study found no beneficial effects of haloperidol as prophylaxis and treatment for ICU delirium [16, 17], other studies have shown that prophylactic haloperidol may exert beneficial effects in critically ill patients [18, 19].

As part of a process to improve delirium care and to further facilitate prevention of ICU delirium, identification of high-risk patients is paramount [20], since these are the most fragile patients who require maximum preventive efforts [21]. Also, when preventive measures can be restricted to high-risk patients, both the number of patients who will be unnecessarily exposed to potential harmful side effects [22] and costs are likely less. Moreover, the effectiveness of prevention seems more pronounced in patients with a higher risk for delirium [18]. Finally, prediction of the ICU delirium risk provides relevant information for family members and caregivers and will facilitate stratification of patients in future delirium prevention studies. This stratification will positively influence the efficient use of research resources.

Currently, an ICU delirium prediction model is available [23, 24]. However, the usefulness of this model is limited by the fact that it requires predictors obtained during the first 24 h of ICU admission. A relevant number of ICU patients develop delirium within 24 h following admission [25, 26], and preventive measures should ideally be applied as early as possible. This calls for a prediction model that can reliably predict delirium at the time of ICU admission. Therefore, the objective of this multinational study was to develop and validate a model based on data available at ICU admission to predict the development of delirium during a patient’s complete ICU stay, and to determine the predictive value of this model in relation to the time of development of delirium.

Some results of this study have been previously presented at the ISICEM congress in Brussels, March 2014, the ESICM congress in Barcelona, October 2014 [27], and the EDA congress in Cremona, November 2014 [28].


Design and study population

A multinational prospective cohort study was carried out in 13 ICUs from seven countries (Australia, Belgium, England, Germany, Spain, Sweden, and the Netherlands) (Table E1 online data supplement). Each participating ICU included all consecutive patients aged ≥18 years (surgical, medical, neurology/neurosurgical, or trauma patients) during a continuous period of approximately three months between October 2011 and June 2012. Patients were excluded if: they were delirious at ICU admission; had an ICU stay shorter than one day; were unable to reliably assess for delirium (e.g., coma during entire ICU stay; unable to understand the language spoken; severely mentally disabled; serious receptive aphasia; serious auditory or visual disorders); or if the compliance rate of the CAM-ICU for delirium screening was <80 % during an individual patient’s ICU stay (Fig. 1).
Fig. 1

Study flow chart

This study was approved by the medical ethical research committee Arnhem-Nijmegen region, The Netherlands (CMO Region Arnhem-Nijmegen, no. 2010/365), and conducted in accordance with the applicable rules concerning the review of research ethics committees and informed consent. All participating centres obtained ethics approval from their committees to conduct the study.

Data collection

Data of 18 candidate delirium predictors were collected at ICU admission in an electronic clinical report form using a secured website. Candidate predictors were selected based on the PRE-DELIRIC model [23, 24], a systematic review about risk factors for delirium [29] and on expert opinion [gathered during the annual meeting of the working group Postoperative Delirium and Cognitive Dysfunction (PoDeCoD) of the European Society of Intensive Care Medicine (ESICM)]. A proposal with candidate predictors, based on the PRE-DELIRIC model and the systematic review, was presented and discussed. Candidate predictors were removed from/added to the final list based on consensus of the members of this working group.

Candidate predictors available at time of ICU admission were: age, gender, history of cognitive impairment (history of dementia, mild cognitive impairment or delirium), history of alcohol, nicotine and drugs abuse, history of vascular disease, Glasgow Coma Scale (GCS) score, diabetes, blood urea nitrogen (BUN), use of opiates in the 24 h before ICU admission, use of anti-psychotics before ICU admission, admission category (surgery, medical, trauma, neurology/neurosurgery), urgent admission, mean arterial blood pressure (MAP), (strong suspicion of) infection, use of corticosteroids, or respiratory failure (see Table E2 online data supplement for exact definitions). Data on sedation and illness severity (APACHE) were not collected, as these are not available at ICU admission.

Delirium assessment

All consecutive patients were screened for delirium by trained ICU nurses at least once every 8- or 12-h shift using the CAM-ICU. In accordance with previous prediction studies [23, 24], delirium was diagnosed when the CAM-ICU was positive [7, 30], and/or when a patient was treated with haloperidol or other anti-psychotics because of delirium (see Table E2). Patients were screened using the CAM-ICU as soon as delirium assessment was possible after awakening from sedation. To exclude a potential source of bias, assessors of the CAM-ICU were not made aware of the fact that they were collecting data for this study [31].

In order to check the quality of the delirium assessments, compliance with the CAM-ICU was per participating centre calculated monthly as the percentage of assessments performed per day in relation to the total number of assessments that should have been performed. Also, inter-rater reliability (IRR) measurements of the CAM-ICU were performed by duplicate measurements of the CAM-ICU for all admitted ICU patients, on one given day per month per centre, by comparing CAM-ICU scores assessed by the ICU nurse with the scores assessed by a CAM-ICU expert. CAM-ICU experts and ICU nurses were blinded to each others delirium assessments. If per centre, the CAM-ICU screening compliance was above 80 % and the Cohen’s kappa was above 0.80, their data were considered reliable. Centres that did not meet these criteria were per centre excluded from the analysis, after which re-analysis was performed. If exclusion of a centre did not significantly affect the performance of the E-PRE-DELIRIC model, the centre was included in the final analysis, as the data were considered to be of sufficient quality [23, 24].

Sensitivity and specificity of the delirium assessments were calculated using the CAM-ICU outcomes obtained by CAM-ICU experts compared with those of the ICU nurses.

Outcome definition

The primary outcome of this study was the development of ICU delirium, defined as a positive assessment for delirium during a patient’s complete ICU stay.

Statistical analysis

Statistical analyses, including information regarding missing values, are described in Supplemental file 1 online data supplement. The model was developed using multiple logistic regression analysis on data of the first two-thirds of the patients of all participating hospitals. Selection of the final predictors of the E-PRE-DELIRIC model took place through manual backward selection of the candidate predictors using a P value of >0.10 for exclusion. The model was validated on data of the last one-third of the patients. Discriminative power was assessed using the area under the receiver operating characteristic curve (AUROC) [31]. Calibration was assessed graphically by plotting the observed outcome frequencies against the mean predicted outcome probabilities or risks, within subgroups of patients that were ranked by increasing estimated probability [32]. To evaluate the predictive value of the E-PRE-DELIRIC model related to the moment at which delirium first occurred, the database was divided into four groups based on the quartiles of the time to development of delirium at: days 0–1; day 2; days 3–6; >day 6.

Data were analyzed using IBM® SPSS® Statistics 20.01 and R statistics R 3.0.1 [33]. In this study, we used the Transparent Reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD), a strict methodology for data reporting, to check our adherence to suggested optimal levels of transparency and completeness of reporting of a prediction model [34].

Sample size calculation

The E-PRE-DELIRIC model was compiled from 18 candidate predictors. For each candidate predictor, at least ten patients with delirium were needed for the validation and calibration of the model [35]. With an anticipated delirium incidence of 15–30 %, an expected attrition of 25 %, we aimed to enrol at least (18 × 10/0.15)/0.75 = 1600 patients.


In total, 5352 patients were screened, of whom 2438 patients did not fulfil the inclusion criteria (Fig. 1). The cohort consisted of 2914 patients of whom 1962 patients were included in the development dataset. The ICU delirium incidence was 24.5 %. The remaining 952 patients were included in the validation dataset with an ICU delirium incidence of 21.8 %. The demographic characteristics of both groups were comparable (Table 1).
Table 1

Patient characteristics


Development dataset (n = 1962)

Validation dataset (n = 952)

Whole study population (n = 2914)

Age in years, mean (Q1–Q3, min/max)

61.7 (53–74, 18/95)

60.6 (51–73, 18/94)

61.3 (52–73, 18/95)

Male, n (%)

1166 (59.4)

550 (57.8)

1716 (58.9)

Admission category, n (%)


1019 (51.9)

476 (50.0)

1495 (51.3)


683 (34.8)

338 (35.5)

1021 (35.0)


90 (4.6)

44 (4.6)

134 (4.6)


170 (8.7)

94 (9.9)

264 (9.1)

Urgent admission, n (%)

1163 (59.3)

570 (59.9)

1733 (59.5)

LOS-ICU in days, median (Q1–Q3, min/max)

2.0 (1–6, 1/133)

2.0 (1–5, 1/125)

2.0 (1–5, 1/133)

Delirium, n (%)

481 (24.5)

208 (21.8)

689 (23.6)

CAM-ICU compliance and inter-rater reliability

As exclusion of the centres with a CAM-ICU screening compliance <80 % and/or a Cohen’s kappa <0.80 did not significantly affect the performance of the E-PRE-DELIRIC model, all centres were subsequently included in the final analysis. The overall CAM-ICU compliance was 83 % (Q1–Q3: 78–93). In total, 648 inter-rater CAM-ICU measurements were performed. The mean IRR measurements were 0.83 [95 % confidence interval (CI) 0.78–0.88] Cohen’s kappa (Table E3 online data supplement). The CAM-ICU had a sensitivity of 0.82 (95 % CI 0.76–0.87) and a specificity of 0.98 (95 % CI 0.96–0.99).

Discrimination and calibration of the E-PRE-DELIRIC model

The developed E-PRE-DELIRIC model consists of nine predictors: age, history of cognitive impairment, history of alcohol abuse, BUN at time of ICU admission, admission category, urgent admission, MAP at the time of ICU admission, use of corticosteroids, and respiratory failure (Table 2). The discriminative power (AUROC) was 0.76 (95 % CI 0.73–0.78) in the development dataset. This database was divided in groups based on the quartiles of the predicted probabilities for delirium development: very low (0–10 %), low (10–20 %), moderate (20–35 %), and high risk for delirium (>35 %). For each of these groups, the sensitivity, specificity and likelihood ratios were calculated (Fig. 2). At a cut-off of 24.5 %, which was the ICU delirium incidence in the development dataset, the sensitivity and specificity were 71 and 69 %, respectively. The AUROC was 0.75 (95 % CI 0.71–0.79) in the validation dataset (see Figure E1 online data supplement for the calibration plot of the validation dataset). Fitting the model Log(Odds) = b 0 + b 1 × LP on the observed data showed that recalibration was not necessary. The slope of the model was 0.961 with an intercept of −0.094.
Table 2

Variables of the E-PRE-DELIRIC model and regression coefficients and formula for the E-PRE-DELIRIC model


Regression coefficient

Odds ratio (95 % CI)



1.025 (1.017–1.033)

History of cognitive impairment


2.406 (1.700–3.404)

History of alcohol abuse


1.657 (1.170–2.347)

Admission category






1.448 (1.058–1.982)



3.384 (1.997–5.735)



1.655 (1.064–2.575)

Urgent admission


1.843 (1.326–2.563)

MAP at the time of ICU admission


0.994 (0.988–1.001)

Use of corticosteroids


1.327 (0.996–1.768)

Respiratory failure


2.670 (2.099–3.396)

BUN at time of ICU admission


1.018 (1.006–1.031)

Data development set. Formula for the E-PRE-DELIRIC model: Risk of delirium = 1/[1 + exp − (−3.907 + 0.025 × age + 0.878 for history of cognitive impairment + 0.505 for history of alcohol abuse + 0 for surgery or 0.370 for medical or 1.219 for trauma or 0.504 for neurology/neurosurgery + 0.612 for urgent admission − 0.006 × MAP at the time of ICU admission + 0.283 for use of corticosteroids + 0.982 for respiratory failure + 0.018 x BUN in mmol/l at time of ICU admission)]

RC reverence category

Fig. 2

Area under the receiver operating characteristic curve (AUROC) of development dataset (AUROC = 0.76)

Prediction of early and late development of delirium

In patients who developed delirium at days 0–1 the AUROC was 0.70 (95 % CI 0.67–0.74) with a sensitivity and specificity of 62 and 67 %, respectively. This increased to 0.76 (95 % CI 0.72–0.80) (development of delirium at day 2), 68 and 67 %; 0.77 (95 % CI 0.74–0.80) (development of delirium at days 3–6), 72 and 68 %; and 0.81 (95 % CI 0.78–0.84) (development of delirium after day 6), 78 and 68 %, respectively. The predictive value of the model improved significantly over time (Fig. 3).
Fig. 3

Area under the receiver operating characteristic curve (AUROC) during ICU stay. Error bars 95 % CIs


In this prospective, international multicentre study, we developed and validated a model allowing early prediction of patients likely to develop delirium during ICU admission. Using nine predictors that are readily available at ICU admission, it is now possible to stratify ICU patients for delirium risk for the complete ICU length of stay with a high discriminative power. In contrast to the earlier prediction model [23, 24], which reliably predicts delirium using data available only after 24 h in the ICU, our E-PRE-DELIRIC model has a moderate-to-good performance using data available immediately at ICU admission. Of interest, the predictive accuracy of the model is higher for delirium that develops later compared to delirium that develops early following ICU admission.

In view of the high prevalence of delirium in ICU patients [2, 3] and its serious consequences [3, 4, 5, 6], prediction of ICU delirium is clearly of clinical relevance as early risk identification may serve different purposes. Both caregivers and patients families can be informed early about the risk of developing delirium in the ICU, and this may, for example, assist them in taking decisions related to the use of preventive measures. In addition, the model enables stratification for the risk to develop delirium, which may be helpful in future trials that aim to investigate preventive interventions. Our group previously reported the effects of prevention with low-dose haloperidol in critically ill patients using a prediction model and demonstrated that patients with the highest risk appear to benefit most from prevention [18]. With the E-PRE-DELIRIC model, stratification is possible immediately after ICU admission. Various other preventive measures that target modifiable delirium risk factors have been found beneficial in non-critically ill patients [8, 10, 11] and need confirmation in ICU patients. In these confirmation trials, stratification of the a priori risk to develop delirium should be determined to demonstrate that treatment groups are adequately randomized. As a consequence, possible interventions that may be expensive and/or labour intensive can then be provided only to those patients with highest risk. In this way, improved quality and efficiency of care may be achieved in a cost-effective manner [36].

The predictors that build the E-PRE-DELIRIC-model are largely consistent with previously reported risk factors for delirium, including age, pre-existing dementia, history of alcoholism, and a high severity of illness at admission [8, 37, 38]. Other factors that confirm previously reported predictors include admission category, urgent admission, BUN [23, 24], and respiratory failure [37, 38, 39]. While use of corticosteroids was not reported as a relevant factor in another study [2], in our study the use of corticosteroids was a significant predictor for delirium. We only collected data about corticosteroids use at ICU admission, in contrast, the authors of [2] collected daily data about corticosteroids use during the whole study period. Of importance, as relevant differences between countries may exist, the current study was a multinational study using data of ICUs from different countries for the development of a delirium prediction model, which increases its generalisability. Interestingly, while the delirium incidence remains stable over time in the four groups in this study, the predictive value of the model increases over time. However, examining differences between early and late delirium did not fall within the scope of this study, and therefore should be studied in future research.

Several limitations of this study need to be addressed. First, despite the fact that availability of candidate predictors at ICU admission and the effort it would take to collect the data in daily clinical practice were leading in determining candidate predictors, three of the predictors from the E-PRE-DELIRIC model (BUN, history of alcohol abuse, and cognitive impairment) had a fairly high rate of missing values. Probably due to unawareness of the importance of these predictors for delirium prediction, currently these predictors are not collected in a standardised manner in daily practice. When health professionals need the information about these predictors to be able to use the E-PRE-DELIRIC model, they will (most often) be able to collect the information as they are readily available. We used imputation techniques to handle missing data. Multiple imputation helps to reduce bias and increases precision by including variation of data [31]. Second, the performance of the CAM-ICU in daily practice is not optimal [40, 41, 42]. However, in contrast to the evaluation study of Van Eijk et. al [40], in the current study ICU patients were screened for delirium at least two times daily and during the whole ICU length of stay. This increases the chance of correct delirium scores, because of the fluctuating course of delirium. The scores were determined in the same way in all participating ICUs, positively influencing possible measurement bias. In addition, data on the use of haloperidol and other anti-psychotics for delirium were used [23]. This usage was consistent among the different centres. Therefore, we may assume that sensitivity of the bedside nurse to recognize delirium anywhere during ICU admission is high. The quality of data of the CAM-ICU screening was further improved by determination of compliance and inter-rater reliability in each ICU. Third, although the full model approach to include all 18 candidate predictors in the predictive model is likely to avoid overfitting and selection data [43], we chose to perform manual backward selection of predictors to improve the clinical feasibility. Fourth, based on the predetermined exclusion criteria, a large number of patients had to be excluded. One of these exclusion criteria was an ICU stay shorter than one day. Since the E-PRE-DELIRIC model will be used prospectively following ICU admission, one may not be able to anticipate a patient’s ICU length of stay (LOS). Therefore, for each patient admitted to the ICU, the E-PRE-DELIRIC score will be calculated. Predictions for patients with an ICU LOS shorter than one day might be different than those for patients with an ICU LOS longer than one day. Therefore, in future research, it is recommended to include these patients to determine the influence on the performance of the E-PRE-DELIRIC model. Finally, it should be taken into account that selection of candidate predictors in the E-PRE-DELIRIC model might be biased. However, the candidate predictors were based on multiple scientific sources, including the previously published PRE-DELIRIC model [23, 24], a systematic review about delirium risk factors [29], and expert opinion. Although external validation of the E-PRE-DELIRIC model in new centres was not performed, this prediction model was internationally developed and prospectively validated in 13 ICUs from seven different countries. The E-PRE-DELIRIC model is a result of a relevant sample from these ICUs, and therefore widely applicable and probably generalisable to other countries. ICUs should consider, however, that the prediction of delirium derived from the E-PRE-DELIRIC model is an approximation; there can always be some over- or underestimation in the predicted delirium risk. Therefore, for future research, it is recommended to test the external validity of the E-PRE-DELIRIC model in other centres to further improve its generalisability.

In conclusion, in this multinational study, we developed and validated an early ICU delirium prediction model that revealed sufficient validity. It enables the clinician to identify those patients likely to develop delirium following ICU admission using only nine predictors. The model allows early delirium preventive interventions in ICU patients with a high risk of delirium.

An automatic version of the E-PRE-DELIRIC model (Excel and web-based) will soon be available at

(In multiple languages available). Furthermore the E-PRE-DELIRIC model will soon be integrated in the DeliriumICU app which makes it possible to use it bedside ( and



The authors would like to thank all co-investigators, their research nurses, and their intensive care teams at each of the participating sites for their help in collecting the patient data. There was no external funding for this study.

Conflicts of interest

The authors declare that they have no conflict of interest.

Supplementary material

134_2015_3777_MOESM1_ESM.docx (39 kb)
Supplementary material 1 (DOCX 38 kb)


  1. 1.
    APA (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Association (APA), WashingtonGoogle Scholar
  2. 2.
    Ouimet S, Kavanagh BP, Gottfried SB, Skrobik Y (2007) Incidence, risk factors and consequences of ICU delirium. Intensive Care Med 33:66–73PubMedCrossRefGoogle Scholar
  3. 3.
    van den Boogaard M, Schoonhoven L, van der Hoeven JG, van Achterberg T, Pickkers P (2012) Incidence and short-term consequences of delirium in critically ill patients: a prospective observational cohort study. Int J Nurs Stud 49:775–783PubMedCrossRefGoogle Scholar
  4. 4.
    Mehta S, Cook D, Devlin JW, Skrobik Y, Meade M, Fergusson D, Herridge M, Steinberg M, Granton J, Ferguson N, Tanios M, Dodek P, Fowler R, Burns K, Jacka M, Olafson K, Mallick R, Reynolds S, Keenan S, Burry L, Investigators S, the Canadian Critical Care Trials G (2015) Prevalence, risk factors, and outcomes of delirium in mechanically ventilated adults*. Crit Care Med 43:557–566PubMedCrossRefGoogle Scholar
  5. 5.
    Pandharipande PP, Girard TD, Jackson JC, Morandi A, Thompson JL, Pun BT, Brummel NE, Hughes CG, Vasilevskis EE, Shintani AK, Moons KG, Geevarghese SK, Canonico A, Hopkins RO, Bernard GR, Dittus RS, Ely EW, Investigators B-IS (2013) Long-term cognitive impairment after critical illness. N Engl J Med 369:1306–1316PubMedCentralPubMedCrossRefGoogle Scholar
  6. 6.
    Zhang Z, Pan L, Ni H (2013) Impact of delirium on clinical outcome in critically ill patients: a meta-analysis. Gen Hosp Psychiatry 35:105–111PubMedCrossRefGoogle Scholar
  7. 7.
    Ely EW, Inouye SK, Bernard GR, Gordon S, Francis J, May L, Truman B, Speroff T, Gautam S, Margolin R, Hart RP, Dittus R (2001) Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). JAMA 286:2703–2710PubMedCrossRefGoogle Scholar
  8. 8.
    Barr J, Fraser GL, Puntillo K, Ely EW, Gelinas C, Dasta JF, Davidson JE, Devlin JW, Kress JP, Joffe AM, Coursin DB, Herr DL, Tung A, Robinson BR, Fontaine DK, Ramsay MA, Riker RR, Sessler CN, Pun B, Skrobik Y, Jaeschke R, American College of Critical Care M (2013) Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit Care Med 41:263–306PubMedCrossRefGoogle Scholar
  9. 9.
    Antonelli M, Bonten M, Cecconi M, Chastre J, Citerio G, Conti G, Curtis JR, Hedenstierna G, Joannidis M, Macrae D, Maggiore SM, Mancebo J, Mebazaa A, Preiser JC, Rocco P, Timsit JF, Wernerman J, Zhang H (2013) Year in review in Intensive Care Medicine 2012: I. Neurology and neurointensive care, epidemiology and nephrology, biomarkers and inflammation, nutrition, experimentals. Intensive Care Med 39:232–246PubMedCentralPubMedCrossRefGoogle Scholar
  10. 10.
    Inouye SK, Bogardus ST Jr, Charpentier PA, Leo-Summers L, Acampora D, Holford TR, Cooney LM Jr (1999) A multicomponent intervention to prevent delirium in hospitalized older patients. N Engl J Med 340:669–676PubMedCrossRefGoogle Scholar
  11. 11.
    Zaubler TS, Murphy K, Rizzuto L, Santos R, Skotzko C, Giordano J, Bustami R, Inouye SK (2013) Quality improvement and cost savings with multicomponent delirium interventions: replication of the Hospital Elder Life Program in a community hospital. Psychosomatics 54:219–226PubMedCrossRefGoogle Scholar
  12. 12.
    Colombo R, Corona A, Praga F, Minari C, Giannotti C, Castelli A, Raimondi F (2012) A reorientation strategy for reducing delirium in the critically ill. Results of an interventional study. Minerva Anestesiol 78:1026–1033PubMedGoogle Scholar
  13. 13.
    Needham DM, Korupolu R, Zanni JM, Pradhan P, Colantuoni E, Palmer JB, Brower RG, Fan E (2010) Early physical medicine and rehabilitation for patients with acute respiratory failure: a quality improvement project. Arch Phys Med Rehabil 91:536–542PubMedCrossRefGoogle Scholar
  14. 14.
    Patel J, Baldwin J, Bunting P, Laha S (2014) The effect of a multicomponent multidisciplinary bundle of interventions on sleep and delirium in medical and surgical intensive care patients. Anaesthesia 69:540–549PubMedCrossRefGoogle Scholar
  15. 15.
    Kamdar BB, King LM, Collop NA, Sakamuri S, Colantuoni E, Neufeld KJ, Bienvenu OJ, Rowden AM, Touradji P, Brower RG, Needham DM (2013) The effect of a quality improvement intervention on perceived sleep quality and cognition in a medical ICU. Crit Care Med 41:800–809PubMedCentralPubMedCrossRefGoogle Scholar
  16. 16.
    Page VJ, Ely EW, Gates S, Zhao XB, Alce T, Shintani A, Jackson J, Perkins GD, McAuley DF (2013) Effect of intravenous haloperidol on the duration of delirium and coma in critically ill patients (Hope-ICU): a randomised, double-blind, placebo-controlled trial. Lancet Resp Med 1:515–523CrossRefGoogle Scholar
  17. 17.
    van den Boogaard M, Pickkers P (2013) Hope for haloperidol in delirium. Lancet Resp Med 1:e27CrossRefGoogle Scholar
  18. 18.
    van den Boogaard M, Schoonhoven L, van Achterberg T, van der Hoeven JG, Pickkers P (2013) Haloperidol prophylaxis in critically ill patients with a high risk for delirium. Crit Care 17:R9PubMedCentralPubMedCrossRefGoogle Scholar
  19. 19.
    Wang W, Li HL, Wang DX, Zhu X, Li SL, Yao GQ, Chen KS, Gu XE, Zhu SN (2012) Haloperidol prophylaxis decreases delirium incidence in elderly patients after noncardiac surgery: a randomized controlled trial. Crit Care Med 40:731–739PubMedCrossRefGoogle Scholar
  20. 20.
    Inouye SK, Westendorp RG, Saczynski JS (2014) Delirium in elderly people. Lancet 383:911–922PubMedCentralPubMedCrossRefGoogle Scholar
  21. 21.
    Mistraletti G, Pelosi P, Mantovani ES, Berardino M, Gregoretti C (2012) Delirium: clinical approach and prevention. Best Prac Res Clin Anaesthesiol 26:311–326CrossRefGoogle Scholar
  22. 22.
    Huybrechts KF, Gerhard T, Crystal S, Olfson M, Avorn J, Levin R, Lucas JA, Schneeweiss S (2012) Differential risk of death in older residents in nursing homes prescribed specific antipsychotic drugs: population based cohort study. BMJ 344:e977PubMedCentralPubMedCrossRefGoogle Scholar
  23. 23.
    van den Boogaard M, Pickkers P, Slooter AJ, Kuiper MA, Spronk PE, van der Voort PH, van der Hoeven JG, Donders R, van Achterberg T, Schoonhoven L (2012) Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for intensive care patients: observational multicentre study. BMJ 344:e420PubMedCentralPubMedCrossRefGoogle Scholar
  24. 24.
    van den Boogaard M, Schoonhoven L, Maseda E, Plowright C, Jones C, Luetz A, Sackey PV, Jorens PG, Aitken LM, van Haren FM, Donders R, van der Hoeven JG, Pickkers P (2014) Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study. Intensive Care Med 40:361–369PubMedCrossRefGoogle Scholar
  25. 25.
    Ely EW, Shintani A, Truman B, Speroff T, Gordon SM, Harrell FE Jr, Inouye SK, Bernard GR, Dittus RS (2004) Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA 291:1753–1762PubMedCrossRefGoogle Scholar
  26. 26.
    Serafim RB, Dutra MF, Saddy F, Tura B, de Castro JE, Villarinho LC, da Gloria Santos M, Bozza FA, Rocco JR (2012) Delirium in postoperative nonventilated intensive care patients: risk factors and outcomes. Ann Intensive Care 2:51PubMedCentralPubMedCrossRefGoogle Scholar
  27. 27.
    Wassenaar A, Van den Boogaard M, Van Achterberg T, Slooter AJC, Kuiper MA, Hoogendoorn ME, K.S. S, Maseda E, Pinto N, Jones C, Luetz A, Schandl A, Verbrugghe W, Aitken LM, Van Haren FMP, Donders ART, Schoonhoven L, Pickkers P (2014) Early PREdiction of DELIRium in ICu patients (E-PRE-DELIRIC): Multinational development and validation of an early delirium prediction model for intensive care patients. In: 27th Annual Congress of the European Society of Intensive Care Medicine, Barcelona, Spain, September 27–October 1, 40: 291Google Scholar
  28. 28.
    Van den Boogaard M, Wassenaar A, Van Achterberg T, Slooter AJC, Kuiper MA, Hoogendoorn ME, K.S. S, Maseda E, Pinto N, Jones C, Luetz A, Schandl A, Verbrugghe W, Aitken LM, Van Haren FMP, Donders ART, Schoonhoven L, Pickkers P (2014) Early PREdiction of DELIRium in ICu patients (E-PRE-DELIRIC): Multinational development and validation of an early delirium prediction model for intensive care patients. In: 9th Annual meeting European Delirium Association, Cremona, Italy, November 6–7, Session 1, nr 4Google Scholar
  29. 29.
    Van Rompaey B, Schuurmans MJ, Shortridge-Baggett LM, Truijen S, Bossaert L (2008) Risk factors for intensive care delirium: a systematic review. Intensive Crit Care Nurs 24:98–107PubMedCrossRefGoogle Scholar
  30. 30.
    Ely EW, Margolin R, Francis J, May L, Truman B, Dittus R, Speroff T, Gautam S, Bernard GR, Inouye SK (2001) Evaluation of delirium in critically ill patients: validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU). Crit Care Med 29:1370–1379PubMedCrossRefGoogle Scholar
  31. 31.
    Moons KG, Kengne AP, Woodward M, Royston P, Vergouwe Y, Altman DG, Grobbee DE (2012) Risk prediction models: I. development, internal validation, and assessing the incremental value of a new (bio)marker. Heart 98:683–690PubMedCrossRefGoogle Scholar
  32. 32.
    Steyerberg EW (2009) Clinical prediction models: a practical approach to development, validation, and updating. Springer, New YorkCrossRefGoogle Scholar
  33. 33.
    R Core Team (2013) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Available at
  34. 34.
    Localio AR, Stack CB (2015) TRIPOD: a new reporting baseline for developing and interpreting prediction models. Ann Intern Med 162:73–74PubMedCrossRefGoogle Scholar
  35. 35.
    Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG (2009) Prognosis and prognostic research: what, why, and how? BMJ 338:b375PubMedCrossRefGoogle Scholar
  36. 36.
    Milbrandt EB, Deppen S, Harrison PL, Shintani AK, Speroff T, Stiles RA, Truman B, Bernard GR, Dittus RS, Ely EW (2004) Costs associated with delirium in mechanically ventilated patients. Crit Care Med 32:955–962PubMedCrossRefGoogle Scholar
  37. 37.
    Huai J, Ye X (2014) A meta-analysis of critically ill patients reveals several potential risk factors for delirium. Gen Hosp Psychiatry 36:488–496PubMedCrossRefGoogle Scholar
  38. 38.
    Zaal IJ, Devlin JW, Peelen LM, Slooter AJ (2015) A systematic review of risk factors for delirium in the ICU. Crit Care Med 43:40–47PubMedCrossRefGoogle Scholar
  39. 39.
    Van Rompaey B, Elseviers MM, Schuurmans MJ, Shortridge-Baggett LM, Truijen S, Bossaert L (2009) Risk factors for delirium in intensive care patients: a prospective cohort study. Crit Care 13:R77PubMedCentralPubMedCrossRefGoogle Scholar
  40. 40.
    van Eijk MM, van den Boogaard M, van Marum RJ, Benner P, Eikelenboom P, Honing ML, van der Hoven B, Horn J, Izaks GJ, Kalf A, Karakus A, Klijn IA, Kuiper MA, de Leeuw FE, de Man T, van der Mast RC, Osse RJ, de Rooij SE, Spronk PE, van der Voort PH, van Gool WA, Slooter AJ (2011) Routine use of the confusion assessment method for the intensive care unit: a multicenter study. Am J Respir Crit Care Med 184:340–344PubMedCrossRefGoogle Scholar
  41. 41.
    Haenggi M, Blum S, Brechbuehl R, Brunello A, Jakob SM, Takala J (2013) Effect of sedation level on the prevalence of delirium when assessed with CAM-ICU and ICDSC. Intensive Care Med 39:2171–2179PubMedCrossRefGoogle Scholar
  42. 42.
    Patel SB, Poston JT, Pohlman A, Hall JB, Kress JP (2014) Rapidly reversible, sedation-related delirium versus persistent delirium in the intensive care unit. Am J Respir Crit Care Med 189:658–665PubMedCrossRefGoogle Scholar
  43. 43.
    Royston P, Moons KG, Altman DG, Vergouwe Y (2009) Prognosis and prognostic research: developing a prognostic model. BMJ 338:b604PubMedCrossRefGoogle Scholar

Copyright information

© The Author(s) 2015

Open AccessThis article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • A. Wassenaar
    • 1
  • M. van den Boogaard
    • 2
    Email author
  • T. van Achterberg
    • 1
    • 3
  • A. J. C. Slooter
    • 4
  • M. A. Kuiper
    • 5
  • M. E. Hoogendoorn
    • 6
  • K. S. Simons
    • 7
  • E. Maseda
    • 8
  • N. Pinto
    • 9
  • C. Jones
    • 10
  • A. Luetz
    • 11
  • A. Schandl
    • 12
  • W. Verbrugghe
    • 13
  • L. M. Aitken
    • 14
    • 15
    • 16
  • F. M. P. van Haren
    • 17
  • A. R. T. Donders
    • 18
  • L. Schoonhoven
    • 1
    • 19
  • P. Pickkers
    • 2
  1. 1.Scientific Institute for Quality of Healthcare, Radboud Institute for Health SciencesRadboud university medical centerNijmegenThe Netherlands
  2. 2.Department of Intensive Care Medicine, Radboud Institute for Health SciencesRadboud university medical centerNijmegenThe Netherlands
  3. 3.Centre for Health Services and Nursing ResearchKU LeuvenLouvainBelgium
  4. 4.Department of Intensive Care MedicineUniversity Medical Centre UtrechtUtrechtThe Netherlands
  5. 5.Department of Intensive Care MedicineMedical Centre LeeuwardenLeeuwardenThe Netherlands
  6. 6.Research Department Anesthesiology and Intensive CareISALA KliniekenZwolleThe Netherlands
  7. 7.Department of Intensive Care MedicineJeroen Bosch Ziekenhuis’s-HertogenboschThe Netherlands
  8. 8.Department of Anesthesiology and Surgical Intensive Care MedicineHospital Universitario La PazMadridSpain
  9. 9.Anaesthetic DepartmentMedway Maritime HospitalGillingham, KentUK
  10. 10.Ward 4E (Critical Care)Whiston HospitalPrescotUK
  11. 11.Department of Anesthesiology and Intensive Care MedicineCharité-Universitaetsmedizin BerlinBerlinGermany
  12. 12.Department of Anesthesiology, Surgical Services and Intensive Care Medicine and Department of Neurobiology, Care Science and Society, Karolinska InstitutetKarolinska University Hospital SolnaStockholmSweden
  13. 13.Department of Critical Care MedicineAntwerp University Hospital, University of AntwerpEdegem (Antwerp)Belgium
  14. 14.Intensive Care UnitPrincess Alexandra HospitalWoolloongabba (Brisbane)Australia
  15. 15.NHMRC Centre for Research Excellence in Nursing, Centre for Health Practice InnovationGriffith UniversityGold Coast (Brisbane)Australia
  16. 16.School of Health SciencesCity University LondonLondonUK
  17. 17.Intensive Care UnitThe Canberra HospitalWoden (Canberra)Australia
  18. 18.Department for Health Evidence, Radboud Institute for Health SciencesRadboud university medical centerNijmegenThe Netherlands
  19. 19.Faculty of Health SciencesUniversity of SouthamptonSouthamptonUK

Personalised recommendations