Acutely hospitalized older patients have an increased risk of mortality, but at the moment of presentation this risk is difficult to assess. Early identification of patients at high risk might increase the awareness of the physician, and enable tailored decision-making. Existing screening instruments mainly use either geriatric factors or severity of disease for prognostication. Predictive performance of these instruments is moderate, which hampers successive interventions. We conducted a retrospective cohort study among all patients aged 70 years and over who were acutely hospitalized in the Acute Medical Unit of the Leiden University Medical Center, the Netherlands in 2012. We developed a prediction model for 90-day mortality that combines vital signs and laboratory test results reflecting severity of disease with geriatric factors, represented by comorbidities and number of medications. Among 517 patients, 94 patients (18.2 %) died within 90 days after admission. Six predictors of mortality were included in a model for mortality: oxygen saturation, Charlson comorbidity index, thrombocytes, urea, C-reactive protein and non-fasting glucose. The prediction model performs satisfactorily with an 0.738 (0.667–0.798). Using this model, 53 % of the patients in the highest risk decile (N = 51) were deceased within 90 days. In conclusion, we are able to predict 90-day mortality in acutely hospitalized older patients using a model with directly available clinical data describing disease severity and geriatric factors. After further validation, such a model might be used in clinical decision making in older patients.
Acute medical illness in older adults is a serious contributor to deterioration . Within 90 days after hospitalization, approximately 20 % will die [2, 3]. At the time of admission it is difficult to determine who is at highest risk. Visualizing the individual risk in an early phase of hospitalization might increase the awareness of the physician, and enable tailored decision-making for the older patient, although these interventions may not primarily be aimed at reducing mortality. A high risk of mortality may reflect overall vulnerability, which preventive interventions may be aimed at, or conversely by usefully initiating palliative care.
Screening models to identify older patients at risk of mortality have been developed and evaluated [4, 5]. These models mainly use either geriatric factors [5, 6] or severity of disease [7, 8]. In these models, scores are assigned to the predictors, which lead to a total score with a cut-off point for high-risk patients. Predictive performance using the cut-off point shows relatively high sensitivity and low specificity, resulting in high numbers of false positives. As a consequence, large-scale implementation of these models in daily care hampers successive interventions . A combination of routine clinical parameters, which reflect the severity of disease, in combination with geriatric factors might improve accuracy and feasibility in daily care.
In the present retrospective cohort study we developed a prediction model for 90-day mortality. We collected clinical parameters of all hospitalized older patients of the acute medical unit in 2012. Vital signs and laboratory results reflect severity of disease with geriatric factors, represented by comorbidities and number of medications. We selected variables that are available in a very early phase of hospitalization to enable in-hospital interventions.
Study design and setting
We performed a retrospective follow-up study among all patients aged 70 years and over who were acutely hospitalized on the acute medical unit (AMU) of the Leiden University Medical Center (LUMC), the Netherlands in 2012. Any following individual admission in the study period, independent of the reason, and patients with palliative care who were expected to die in a few days were excluded. The AMU is a 13-bed ward particularly focussed on acute admissions, mainly from the Emergency Department. The population is characterized by hemodynamically stable patients in the fields of internal medicine, surgery, pulmonary diseases and gastroenterology. The medical ethics committee of the LUMC waived the necessity for formal approval of the present study, as all data were available from standard care.
We selected potential predictors of 90-day mortality from the clinical parameters available at the moment of hospitalization on the AMU. These parameters reflect severity of disease, including vital signs and laboratory results, and underlying level of vulnerability, including comorbidity and number of medications used at home. A predictor was eligible if it fulfilled the following criteria: (1) it was available in the medical records for retrospective analysis; (2) available to the physician within 24 h after admission and (3) assumed to have a relationship to the outcome based on clinical reasoning by three medical doctors and (4) was already being measured routinely to enhance in future implementation, with a maximum of 15 % missing values of each predictor. Multiple imputation techniques were used to compute the missing predictors . First measured vital signs after hospitalization and first known in-hospital laboratory results were extracted from the electronic patient records (Chipsoft-EZIS®, version 5.2, 2006–2014). Existing comorbidities and medications used at home were obtained manually from the patient records, where medication was reported as part of routine clinical practice. Usually, the physician will first ask the patient at the moment of hospitalization for comorbidities and medication use. If necessary, the information will be verified with the general practitioner or pharmacy.
Vital signs were assessed by the nurse directly after admission and consisted of systolic and diastolic blood pressure, heart rate, respiratory rate, oxygen saturation and body temperature. First known in-hospital laboratory results within 24 h after presentation were extracted and consisted of: sodium (mmol/L), potassium (mmol/L), urea (mmol/L), eGFR (estimated glomular filtration rate, calculated by the modification of diet in renal disease (MDRD) equation, ml/min/1.73 m2), leukocytes (×109/L), thrombocytes (×109/L), C-reactive protein (mg/L), non-fasted glucose (mmol/L) and haemoglobin (mmol/L). Comorbidity was evaluated with the Charlson comorbidity index (CCI). It incorporates weighted scores for 19 medical conditions, increasing from 1 to 6 with severity. The CCI is a frequently used instrument by researchers, and has been validated to predict 1-year mortality [10, 11]. The number of different medications at home was recorded according to their pharmacological sub classification. Medications of the same subgroup count as one drug, and combined medications of two different pharmacological sub-classifications were considered as two different drugs. Topical and ‘as required’ medications were excluded because of the unreliable registration rate of the physicians and the absence of information whether the patient actually used it. If recorded in the medical records, over-the-counter medications were included when patients used it on regular base.
The primary endpoint was mortality within 90 days after hospital admission. Mortality dates were assessed from the Dutch municipality records.
We aimed to develop a model with a high positive predictive value (PPV) to enable targeted interventions, and therefore the model should have high specificity. Because a model based on a risk score derived from clinical cut-off values is easier to implement in clinical practice than a model based on computations with continuous variables, we started to dichotomize continuous predictors by using the ranges of clinical reference categories. A stricter clinically relevant cut-off point was chosen in cases when specificity on 90-day mortality was lower than fifty percent. However, dichotomizing may lead to loss of information, reduction in power and uncertainly in defining the optimal cutpoint . In a sensitivity analysis, we repeated the same analyses with preservation of continuous predictors to compare discriminative performance.
Descriptive baseline characteristics were expressed in percentages, means with standard deviations and medians with interquartile ranges. A Kaplan–Meier curve was used to show cumulative mortality of patients after admission. Binary logistic regression techniques for both dichotomised and continuous data were used for uni- and multivariable analysis on mortality (no censoring occurred). The prediction model was derived via backward elimination with Akaike’s Information Criterion for candidate predictors. With this technique the most significant predictors remain in the model, while “noise” is reduced by eliminating predictors that are not statistically significant. Discriminative performance of the different models was assessed with the area under the receiver operating characteristic curve (AUC).The model was internal validated with 500 bootstrap samples, where we repeated the backward elimination procedure. With this method internal validity is tested by drawing 500 different population samples from the original sample. The test reflects how robust the findings are when slight changes are made to the population, and was presented with the internal validated AUC. The formula 1/(1 + exp(−linear predictor))was applied to determine the individual risk on 90-day mortality . Performance of the final model is shown with the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio and negative likelihood ratio.
In clinical practice, cut-off points are often used, e.g., to interpret laboratory results. In prediction dichotomizing continuous predictors is arguable, because of a loss of information . When using cut-off points, laboratory results just outside the reference range are considered the same risk as more extreme results, which diminishes the power of the model. In a sensitivity analyses we treated predictors as continuous variables to compare discrimination. First outlying observations were truncated to 5 and 95 % by means of winsorization . Second restricted cubic spline techniques with three knots were applied to continuous predictors in a binary regression model, and discrimination was analysed by calculation the AUC. The level of significance was set at P < 0.05. Statistical analyses were performed using IBM SPSS Statistics package (version 20) and R version 3.1.1.
In 2012, 606 older patients were acutely hospitalized to the acute medical unit (AMU) of our hospital. By excluding 86 subsequent admissions and 3 moribund patients, a final cohort of 517 patients was available for final analysis.
The baseline characteristics of the cohort are described in Table 1. The mean age was 78.3 years, 269 (52.0 %) patients were male, 467 (90.3 %) were admitted via the Emergency Department and 367 (71.0 %) were primary treated under the responsibility of one of the medical specialities (internal medicine, surgery or pulmonary diseases). Mean laboratory results were within the normal range or slightly below or above these thresholds. The median number of comorbidities was 2 (IQR 1-4), and median number of medications used at home was 7 (IQR 4-11).
Supplemental Table 1 gives an overview of categories of the dichotomization process. Missing values, up to 11 % in thrombocytes, were imputed. Most reference ranges reflect clinical normal ranges, except that we chose different rounded cut-offs for systolic blood pressure (<200 mmHg), urea (<15.0 mmol/L) and c-reactive protein (CRP, <100 mg/L), leukocytes (<13 × 109/L), eGFR (>30 ml/min/1.73 m2) and haemoglobin (>6.5 mmol/L for females and >7.5 mmol/L for males) to reflect more specific measures of disease severity.
After 90 days, 94 patients (18.2 %) had died (supplemental Fig. 1). In Table 2, results of the univariate analyses and the performance of all individual predictors are shown. Oxygen saturation, heart rate, Charlson comorbidity index (CCI), thrombocytes, urea, potassium and CRP outside the reference range are statistically significantly associated with 90-day mortality. In contrast, non-fasted glucose and creatinine clearance outside the reference range prove to have a protective effect. Age and gender show no association with 90-day mortality. Best performance of a single variable is the CCI. A score of 5 or higher (N = 91) yields a positive predictive value (PPV) of 0.37 and area under the curve (AUC) of 0.61.
Results of the multivariable and final model are displayed in Table 3. A backward selection procedure results in a model of six predictors including oxygen saturation, CCI, thrombocytes, urea, CRP and non-fasted glucose. The area under the curve (AUC) is 0.738 (95 %CI 0.967–0.798) and decreases to 0.724 after internal validation.
By applying the formula 1/(1 + exp(−(−2.127 + 0.862 × ‘saturation’ + 1.201 × ‘CCI’ + 0.774 × ‘thrombocytes’ + 0.706 × ‘urea’ + 0.588 × ‘CRP’ + −0.791 × ‘non-fasted glucose’ ))) the individual risk of 90-day mortality in acutely hospitalized older patients was calculated. Figure 1 shows the calibration plot of the final model. Over the whole range predicted probabilities are in line with the observed, with more spread in the higher risk groups. In Table 4 we calculated predictive performance of the 30, 20 and 10 % patients at highest risk to provide information about clinical usefulness. Positive likelihood ratio’s range from 2.70 to 5.06. The positive predicting value of the 51 patients (10 %) at highest risk is 0.53 and implies that 53 % die within 90 days after admission.
As a sensitivity analysis we repeated analyses for the multivariate and final model with continuous data. Accuracy is comparable in both multivariate and final model. The AUC for continuous data is 0.771 (95 %CI 0.717–0.825) and after dichotomization 0.758 (95 %CI 0.702–0.815) in the multivariate model and 0.736 (95 %CI 0.677–0.795) and 0.738 (95 %CI 0.678–0.798) in the final model (data not shown).
In the present study, we developed a prediction model for 90-day mortality in acutely hospitalized older patients using routinely collected clinical parameters describing disease severity and geriatric factors. With this model we are able to identify a high-risk group with an average 53 % risk of mortality within 90 days after admission compared to the baseline risk of 18.2 %.
We developed and internally validated a prediction model for 90-day mortality in acutely hospitalized older patients using six routinely collected clinical parameters and with adding age and gender. Underlying vulnerability of older patients is reflected in the Charlson comorbidity index (CCI) and severity of disease in oxygen saturation, thrombocytes, urea, C-reactive protein and non-fasted glucose. CCI was developed to predict mortality in medical patients , and was recently validated in acutely hospitalized older adults to predict both short- and long-term mortality . Models using vital signs have also been previously studied to predict mortality. The acute physiology and chronic health evaluation (APACHE II) is a severity of disease classification system developed to predict in-hospital mortality in intensive care unit patients of all ages . The APACHE II comprises a combination of vital parameters and different laboratory results. The simple clinical score (SCS) is a prediction model for 30-day mortality in acutely admitted patients , and consists of 16 parameters, including vital parameters and presentation signs such as new stroke, coma and abnormal ECG. The Modified Early Warning Score (MEWS) was originally designed for the ED setting to identify medical patients at risk of catastrophic deterioration, and was subsequently validated for prediction of in-hospital mortality in hospitalized patients . The MEWS incorporates five vital signs: systolic blood pressure, pulse rate, respiratory rate, body temperature and level of consciousness. The aforementioned models are well validated and are used in practice, but share the disadvantage that prognostic accuracy among older patients is modest with relatively low positive predicting values. An explanation might be the use of (bed-side) scores with a cut-off point, instead of using individual risk scores. Or it could be the use of either severity of disease characteristics or geriatric factors in the prediction model. Another explanation could be that prediction models were developed in a more severely ill population of all ages, with the consequence that results were neither representative nor tailored for these older patients [16, 17]. Unexpected findings is the positiveness of abnormal thrombocytes and urea. To our knowledge, these measurements are not used in other comparable prediction models. Validity of this might be explained by the possible over-representation of patients with low thrombocytes being treated with chemotherapy or high urea caused by dehydration or kidney failure. Another unexpected finding is the protective value of creatinine clearance <30 (ml/min/1.73m2) on 90-day mortality (OR 0.48, 95 % CI 0.28–0.84) in the univariate analysis. A possible explanation could be that the hospital is a centre for patients requiring dialysis and kidney transplantation. These patient groups are hospitalized more readily, with possible less severe acute medical conditions. However, in the multivariate model and by using creatinine clearance as a continuous variable the association is lost, indicating that it could also be caused by outliers. Taken together, we show that combining parameters reflecting severity of disease and geriatric factors results in an prediction model capable of predicting 90-day mortality in acutely hospitalized older patients.
We made several choices in developing our model, in order to be ready for clinical implementation. First, we used routinely available clinical parameters. Candidate predictors were known within 24 h after admission. Second, we used a formula instead of a bed-side scorecard. Health care professionals do not have to calculate risks by hand, preferable the formula should be integrated in the electronic patients records or be available in a smartphone application. By providing an individual risk for each patient, the consequences of the screening can vary. Depending on the local resource availability a hospital can implement interventions. As an example, comprehensive geriatric assessment could be performed in all older patients with a risk of 30 % or higher, which includes extra attention for patient preferences, treatment goals and possible palliative trajectories. In another hospital the advice for the treating physician could be to take into account both the individual risk score and the condition of the patient in decision making, without standardized interventions. Third, we aimed for a model with a high specificity, resulting in a high positive predictive value (PPV) in relation with the baseline risk. The PPV will give the clinician a robust feeling and may therefore be more relevant than a high AUC with modest predictive values. The identification senior at risk (ISAR) is also a prediction tool, originally developed to identify older patients in the ED at increased risk of adverse health outcomes, a composite outcome of functional decline and mortality . The ISAR is a widely used tool in the ED , and validated among 667 acute hospitalized older adults for prediction of adverse outcomes, including mortality. After 90 days of follow-up 5 % had died, with 6 % of the patients assigned high risk deceased within 90 days, indicating a low positive predictive value. The negative predictive value (NPV) for 90-day mortality was 0.97, which means that 97 % of the patients not at risk were still alive after 90 days. These results imply that the ISAR in this setting is more suitable to rule out patients at high risk, whereas our model is tailored to identify older patients at high risk for mortality with a PPV of 0.53 in the highest risk group. Identifying of patients at low risk (“rule-out”) may be a very sensible strategy in its own right. However, our aim is to specifically identify patients at the highest risk because these are the patients we want to follow-up with intervention, and we want to aim our limited clinical resources to only those at the highest risk.
Our study has several limitations. First, we studied retrospective data, and therefore the number of available predictors and related outcomes were limited. Ideally, predictors such as cognition, functional status and outcomes such as functional decline, and readmissions should also be used, but these were not available in this retrospective study. Second, the fact that we found some unexpected results further stresses the need for external validation, as it is impossible to distinguish whether these findings are specific to our cohort, chance finding or reproducible in other cohorts. Strengths of the present study are that our prediction model is based on routinely measured and directly available candidate predictors. This enhances convenient future implementation in an early phase of presentation. We used clinical cut-off points to reflect clinical practice and relate to the awareness of the physician. From a methodological point of view using continuous variables is preferable, but is harder to relate to clinical practice. Nevertheless, accuracy of our model is equally well when dichotomized or with continuous variables. Another strength is the high specificity of the developed model. This specificity ensures the development of interventions that are aimed at a relatively small group of patients at high risk of a negative event. Such tools are of importance in the emergency medicine setting, allowing physicians in EDs and Acute Wards to make informed decisions on diagnostic and therapeutic strategies in older patients and the implementation of measures to prevent poor outcome.
In conclusion, we developed a prediction model on 90-day mortality in acutely hospitalized older patients. We used a combination of predictors containing information about severity of disease and geriatric factors and calculated individual risk scores. Currently, we are conducting a large multicentre prospective follow-up study among acutely presenting older patients in both the ED and wards, including more candidate predictors and outcomes.
Clegg A, Young J, Iliffe S, Rikkert MO, Rockwood K (2013) Frailty in elderly people. Lancet 381(9868):752–762
Covinsky KE, Palmer RM, Fortinsky RH, Counsell SR, Stewart AL, Kresevic D, Burant CJ, Landefeld CS (2003) Loss of independence in activities of daily living in older adults hospitalized with medical illnesses: increased vulnerability with age. J Am Geriatr Soc 51(4):451–458
Smolin B, Levy Y, Sabbach-Cohen E, Levi L, Mashiach T (2015) Predicting mortality of elderly patients acutely admitted to the Department of Internal Medicine. Int J Clin Pract 69(4):501–508
Campbell SE, Seymour DG, Primrose WR, Project A (2004) A systematic literature review of factors affecting outcome in older medical patients admitted to hospital. Age Ageing 33(2):110–115
Wou F, Gladman JR, Bradshaw L, Franklin M, Edmans J, Conroy SP (2013) The predictive properties of frailty-rating scales in the acute medical unit. Age Ageing 42(6):776–781
Edmans J, Bradshaw L, Gladman JR, Franklin M, Berdunov V, Elliott R, Conroy SP (2013) The Identification of Seniors at Risk (ISAR) score to predict clinical outcomes and health service costs in older people discharged from UK acute medical units. Age Ageing 42(6):747–753
Cei M, Bartolomei C, Mumoli N (2009) In-hospital mortality and morbidity of elderly medical patients can be predicted at admission by the modified early warning score: a prospective study. Int J Clin Pract 63(4):591–595
Kellett J, Deane B (2006) The Simple Clinical Score predicts mortality for 30 days after admission to an acute medical unit. QJM 99(11):771–781
Janssen KJ, Vergouwe Y, Donders AR, Harrell FE Jr, Chen Q, Grobbee DE, Moons KG (2009) Dealing with missing predictor values when applying clinical prediction models. Clin Chem 55(5):994–1001
Charlson ME, Pompei P, Ales KL, MacKenzie CR (1987) A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 40(5):373–383
Frenkel WJ, Jongerius EJ, Mandjes-van Uitert MJ, van Munster BC, de Rooij SE (2014) Validation of the Charlson comorbidity index in acutely hospitalized elderly adults: a prospective cohort study. J Am Geriatr Soc 62(2):342–346
Royston P, Altman DG, Sauerbrei W (2006) Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 25(1):127–141
Steyerberg EW (2009) Clinical prediction models: a practical approach to development, validation, and updating. Springer, New York
Charlson ME, Pompei P, Ales KL, MacKenzie CR (1987) A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 40(5):373–383
Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) APACHE II: a severity of disease classification system. Crit Care Med 13(10):818–829
Mooijaart SP, Broekhuizen K, Trompet S, de Craen AJ, Gussekloo J, Oleksik A, van Heemst D, Blauw GJ, Muller M (2015) Evidence-based medicine in older patients: how can we do better? Netherlands J Med 73(5):211–218
Broekhuizen K, Pothof A, de Craen AJ, Mooijaart SP (2015) Characteristics of randomized controlled trials designed for elderly: a systematic review. PLoS One 10(5):e0126709
McCusker J, Bellavance F, Cardin S, Trepanier S, Verdon J, Ardman O (1999) Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc 47(10):1229–1237
Yao JL, Fang J, Lou QQ, Anderson RM (2015) A systematic review of the identification of seniors at risk (ISAR) tool for the prediction of adverse outcome in elderly patients seen in the emergency department. Int J Clin Exp Med 8(4):4778–4786
The authors acknowledge the contribution of Mary Ann Etty for the data extraction from patient records.
Conflict of interest
The authors declare that they have no conflict of interest.
Statement of human and animals rights
The procedures followed were in accordance with the standards of the medical ethics committee and with the Helsinki Declaration of 1975, as revised in 2008.
This study does not require informed consent.
The Institute for Evidence-Based Medicine in Old Age (IEMO) is funded by the Dutch Ministry of Health and Welfare and supported by ZonMW (project number 62700.3002).
J de Gelder, J.A. Lucke, N. Heim, A.J.M. Craen, B. de Groot, A.J. Fogteloo, G.J. Blauw, S.P. Mooijaart: conception and design of the study. J. de Gelder, S.D. Lourens: acquisition of data. J. de Gelder, E.W. Steyerberg, S. P. Mooijaart: Analysis and interpretation of data. J. de Gelder and S. P. Mooijaart: Drafting the article. J. A. Lucke, N. Heim, S. D. Lourens, A. J. M. Craen, B. de Groot, A. J. Fogteloo, E. W. Steyerberg, G. J. Blauw: Revision for important intellectual content. All authors gave final approval of the current version of the article.
The sponsor had no role in the study.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
de Gelder, J., Lucke, J.A., Heim, N. et al. Predicting mortality in acutely hospitalized older patients: a retrospective cohort study. Intern Emerg Med 11, 587–594 (2016). https://doi.org/10.1007/s11739-015-1381-7
- Acute hospitalization
- Older adults