Acute hospitalization is a hazardous event for older patients, as it is associated with functional and cognitive decline, in-hospital mortality and short-term mortality.1,2 In the next 20 years, the percentage and absolute number of people of 65 years and older in the Dutch population will almost double.3 Therefore, early recognition of patients at high risk of mortality and other negative health outcomes is needed, not only for advanced care planning and informing patients about prognosis and treatment perspectives, but also to control health care costs.

Although prognostication is a core element of medical practice, it is also the part in the care process about which physicians feel most insecure.4 However, it is unclear whether this feeling of uncertainty is justified with acutely hospitalized older patients. Research on prognostication has mainly focused on (terminally ill) cancer patients and patients in the intensive care unit (ICU). These studies revealed several issues on prognostication. First of all, physicians were able to differentiate which groups of patients had a higher mortality risk, but the individual patient prognosis was usually inaccurate.5 Second, physicians were in general too optimistic in the prognosis of their patients, even in patients with a very short life expectancy.5,6 And third, the predictive ability of physicians in the ICU was better than standardised prediction models.7

In the process of prognostication, the role of nurses has mainly been studied in ICU patients showing conflicting results when compared with physicians.711 One study showed that nurses could predict survival rate better,10 whereas other studies indicated that physicians were better predictors.7,8,12 Only one study combined scores of both disciplines, and it was found that this improved prognostication.7 It is likely that nurses have a different point of view on older patients’ severity of illness and risk of mortality, as they spend more time at the bedside of patients than the physicians.

Therefore, the aim of this study was to answer the following two clinical questions using a prospective cohort design: (i) Which factors are associated with 90-day mortality risk in a group of acutely hospitalized older medical patients? And (ii) does adding a clinical impression score of nurses or physicians improve the discriminatory ability of mortality prediction?


Study Population

This prospective cohort study was conducted during a 32-month period from 01 November 2002 until 01 July 2005 at the Academic Medical Centre (AMC) in Amsterdam, The Netherlands, a 1024-bed tertiary university teaching hospital. All consecutive patients aged 65 years or older acutely admitted to the Department of Internal Medicine were enrolled. Patients were excluded if (a) they were unable to speak or understand Dutch, (b) if they or their relatives did not give informed consent for the study, (c) if they came from or were transferred to another ward than the medical ward, (d) or they were discharged from the hospital within 48 hours after admission. Inclusion had to take place within 48 hours after admission and informed consent was obtained before inclusion. The study was approved by the Medical Ethics Committee of the AMC.

Data Collection on Prognostication by Nurses and Physicians

The attending nurse and physician were employed on the medical wards where patients were admitted. A member of our research team interviewed both the attending nurse and physician independently of each other before the assessment of the patient, but within 48 hours of admission of the patient. In the Netherlands, one nurse is responsible for a patient’s total nursing care during one shift. On average the nurse takes care of four to five patients in a one-day shift. The physicians work on a ward and have a case load of about six to seven patients daily. Physicians change wards every two to three months, because they are still in training.

The attending nurse and physician were blinded to the data collection of the patient by the research nurse and were interviewed during a day shift. On average they would have seen the patient for four hours during their shift and sometimes even on a shift the day before the interview. So they knew their patient for at least three hours to a maximum of sixteen hours. They were asked to label the clinical impression of illness where the patient was admitted for, in relation to mortality, expressed in a global clinical score ranging from 10 (high possibility of a bad outcome including mortality) until 1 (high possibility of a good outcome). There were no other special instructions to nurses and physicians apart from the above information. Thus, every patient had one clinical impression score from the attending nurse and one clinical impression score from the attending physician.

All attending nurses were registered nurses with at least a bachelor’s degree, with a wide range of experience ranging from 1–35 years, and all attending physicians were residents, who were still in training, with a clinical experience ranging from 1–3 years. We did not choose to interview faculty members, as they are present on the wards on an irregular basis, which made it difficult to 1) compare their clinical impression scores and 2) to complete data collection. We did not register personal details of nurses and physicians. However, details of experience were gathered from the personnel department. During the study, approximately 110 different nurses and 28 different residents were employed in the Department of Internal Medicine.

Data Collection on Patients

An initial multidisciplinary evaluation of the patient was performed by members of the geriatric research team. The team was composed of two physicians in geriatric medicine, a fellow in geriatric medicine, a clinical nurse specialist and five research nurses trained in geriatric nursing.

Eligible patients were screened by research nurses for the following factors associated with short-term mortality as described in the literature: functional status, cognitive impairment, delirium, severity of illness, diagnosis at admission, co-morbidities and age.2,1318 The data collection had to be completed within 48 hours after admission.

In-hospital functional status was assessed by the Barthel index,19, this questionnaire assesses actual activities of daily living (ADL) functioning at the time of admission. Patients were scored on 10 items (defecation, bladder function, external care, toileting, eating, transfer, mobility, dressing, climbing stairs and bathing) and scores ranged from 0–20. A high score reflects more intact ADL functions. The presence and degree of global cognitive impairment were assessed using the 30-item Mini Mental State Examination (MMSE).20 Global cognitive functioning of the patient was also assessed from the closest relative using the Informant Questionnaire on Cognitive Decline in the Elderly-short form (IQCODE-SF).21,22 In this 16-item questionnaire, relatives were asked to compare cognitive functioning two weeks before admission with cognitive functioning of the patient ten years ago. Relatives could indicate on a five-point scale if cognition had improved, slightly improved, did not change, slightly decreased or decreased. Possible range of scores varies between 16 and 80. Cognitive impairment was defined as earlier diagnosed dementia by a physician in geriatric medicine or neurologist or a score of 23 or less on the MMSE. If the MMSE was not available or the patient was delirious (see below), we used the IQCODE-SF. Patients with a mean IQCODE-SF score of 3.9 or more were considered as having pre-existing cognitive impairment.

One of the two physicians in geriatric medicine assessed the presence of delirium with the confusion assessment method (CAM).23 The CAM rates patients based on the presence of symptoms of delirium (acute start, decreased attention, unorganised thinking and level of consciousness). Patients were rated as delirious if both the acute start and decreased attention were present in combination with either unorganised thinking or changed level of consciousness, or both unorganised thinking and changed level of consciousness. The physician in geriatric medicine also reviewed the patient’s medical chart for medical problems at admission, expressed in the differential diagnosis. This means that one patient could have more than one diagnosis. We grouped the differential diagnosis in major internal problems based on ICD-9 codes: infectious disease, malignancy, disease of the digestive system, cardiovascular disease, pulmonary complaints, endocrine problems (including diabetes) and neurological disease. The patients’ first laboratory results after admission were collected from the patient data system (PDS), which included sodium, creatinin, urea nitrogen, haemoglobin and leucocytes serum levels. Furthermore, the Charlson co-morbidity index was scored, indicating the number and severity of co-morbidities.24 Possible range of scores of the Charlson co-morbidity index varies between 0 and 31, with a higher score indicating more and/or more severe co-morbidities.

Living arrangement was collected and was divided in several categories. A senior residence is a complex where older people live independently. They are not provided with meals or with personal care, but people can ask for assistance if needed in an emergency situation. An old peoples home is a care provision for older people where they have their own room, but are provided with meals, household assistance and, if needed, personal care. In a nursing home older people need assistance with all personal care, such as washing, toileting and eating. We also collected demographic data and marital status.

Registration of Mortality

Survival time was registered up to 90 days after hospital admission. It was expected that in this period the acute disease leading to admission will affect survival status, but after 90 days other factors may also contribute, such as a new episode of illness. Date of death was verified in the PDS, if the exact date of death was not registered or unclear, we contacted the general practitioner of the patient to verify the date of death.

Statistical Analysis

Standard descriptive statistics were used. Differences in scores of continuous variables were tested with a Student t-test. The Chi-square test was used to compare the distribution of categorical data. A correlation coefficient was calculated for clinical impression scores given by the nurses or physicians. To answer our research questions, data-analyses were divided into two steps.

The first step was to identify factors independently associated with mortality 90 days after the patient’s admission. For this purpose, we performed a logistic regression analysis. We started with a univariate logistic regression analysis. Risk factors known to be associated with mortality, as described above were included in the regression analysis, completed with laboratory results (haemoglobin, urea nitrogen, creatinine, sodium, leucocytes), living arrangement and marital status. All variables with a p-value ≤ 0.2 in the univariate logistic regression analysis were included in the multivariate analysis. A backward selection procedure was used, accepting a p-value of ≤ 0.05. This resulted in a set of clinical variables that were significantly associated with mortality, which will be used in a baseline model referred to as model 1. These analyses were performed in Statistical Package for Social Sciences (SPSS), version 14.0.2.

In order to estimate the additional value of the clinical impression scores, we added this impression as covariate to model 1. In total three additional multivariate logistic regression analyses were performed. Model 2 consisted of the variables of model 1 plus the clinical impression score of the physician. Model 3 consisted of the variables of model 1 plus the clinical impression score of the nurse, and in model 4 both clinical impression scores were added to model 1. For all four models an area under the curve (ROC) was computed along with its 95% confidence interval. Next, differences between these four models were calculated based on 1000 bootstrap samples, with 95% confidence intervals. All models were then set out in ROC curves. The analysis regarding estimation of the additional value of clinical impression score was performed in the statistical package R version 2.0.0.


Study Population

There were 785 eligible patients. We excluded 173 patients (22.1%), because they did not give informed consent; 26 patients (3.3%) because they were too ill; 28 patients (3.6%) because they were not able to speak or understand Dutch; and 95 patients (12.1%) because they were screened for eligibility more than 48 hours after admission. In total, 463 patients were included in this study. Table 1 presents the baseline characteristics of the study population. Mean age was 78.1 years (SD 7.8) and 42.5% were male. Mean clinical impression score given by the nurses and physicians was 6.3 (1.9) and 6.4 (1.9) (p = 0.1), respectively. Figure 1 shows a scatter plot of combined clinical impression scores of nurses and physicians. The correlation between these scores was r = 0.45 (p < 0.001)

Figure 1
figure 1

Clinical impression scores of nurses and physicians, n = 463 patients. Size of the bubble indicates the number of times a specific combination of nurse and physician score was given. The smallest bubble indicates a frequency of 1; the largest bubble indicates a frequency of 36.

Table 1 Baseline Characteristics for Acutely Admitted Medical Patients

Factors Associated with 90-day Mortality

In total, 110 patients (23.8%) died within 90 days of admission. We analysed factors associated with 90-day mortality in acutely hospitalized older patients (Table 2). In the multivariate analysis four risk factors were significantly associated with mortality of acutely hospitalized older patients within 90 days of admission; Barthel index, diagnosis malignancy, urea nitrogen and Charlson co-morbidity index. Delirium was significant in the univariate analysis, but not in the multivariate analysis. We checked for colinearity between delirium and other variables entered in the multivariate analysis. Delirium and Barthel index were highly correlated (r = 0.48, p < 0.001), no further significant correlations were found with other variables. Table 3 shows adjusted odds ratios for 90-day survival of acutely admitted older patients and the area under the curve of four different models

Table 2 Unadjusted and Adjusted Odds Ratios for 90-Day Survival of Acutely Admitted Older Patients
Table 3 Adjusted Odds Ratios for 90-Day Survival of Acutely Admitted Older Patients and the Area Under the Curve of Four Different Models

Baseline Prediction Model of Mortality

Model 1 represents the baseline set of clinical variables. A higher score on the Barthel index, indicating a patient is more independent, was associated with a decreased risk of dying within 90 days of admission. An increase of one point on the Barthel index was significantly associated with a 10% decreased risk of dying. A higher score on the Charlson comorbidity index was associated with an increased risk of dying within 90-days after admission. An increase of one point was significantly associated with a 20% increased risk of dying. Diagnosis malignancy at admission was significantly associated with a 300% increased risk of dying within 90 days of admission. An increase of 1 mmol per liter in urea nitrogen serum level was associated with a 2% increased risk of dying. The area under the curve for this model was 0.76 (95% CI 0.71 to 0.82).

Adding Clinical Impression Scores of Physicians and Nurses

Next, the clinical impression score of the physician was added to the baseline set of clinical variables (model 2). The area under the curve (AUC) for this model was 0.77 (95% CI 0.71 to 0.82). When adding this clinical impression score of the physician, urea nitrogen did not contribute significantly to the overall model anymore.

Adding a clinical impression score of the nurse (model 3) resulted in an AUC of 0.76 (95% CI 0.71 to 0.82). Finally, we added both the clinical impression score of the physician and nurse to model 1 resulting in model 4. The AUC for this model was 0.77 (95% CI 0.72 to 0.82). The ROC curves of the four models were set out in Figure 2.

Figure 2
figure 2

ROC curves for four models predicting mortality. B is baseline set of clinical variables (model 1), B + P is model 1 with clinical impression score of physician (model 2), B + N is model 1 with clinical impression score of nurse (model 3) and B + P + N is model 1 with clinical impression score of both physicians and nurses (model 4).

To determine whether adding clinical impression score to model 1, models 2 to 4 were compared with model 1 by applying 1000 bootstrap samples. Adding the clinical impression score did not significantly improve the discriminating value of model 1: the absolute difference in the AUC between model 1 and model 2 was -0.004 (95% CI -0.017 to 0.008). The AUC of model 3 was also not statistically significantly better than that of model 1: the difference in AUCs between model 1 and model 3 was 0.0004 (-0.003 to 0.002). Also comparing the AUC model 1 to model 4 resulted in a statistically not significant difference in AUCs of -0.006 (-0.02 to 0.01).

Because of colinearity between delirium and Barthel index, we replaced the Barthel index covariate with delirium and repeated the analyses of the four different models. The AUC values of the four resulting models were slightly worse (model 1: 0.747, model 2: 0.753, model 3: 0.750, model 4: 0.753) and bootstrapping the differences between any two of the AUCs could not show any statistically significant differences.


In this study among 463 acutely admitted older medical patients, the 90-day mortality risk was found to be increased in acutely admitted medical patients with more functional impairment expressed in a lower score on Barthel index, more comorbidities expressed in a higher score on the Charlson co-morbidity index, malignancy and an elevated urea nitrogen levels. With these four clinical variables the mortality risk of acutely admitted older patients could be predicted quite well. Adding a clinical impression score from attending physicians, from attending nurses or from both disciplines did not contribute significantly to the accuracy of mortality prediction in this patient group.

To our knowledge this is the first study adding clinical impression score of physicians or nurses or both on prediction of mortality among acutely admitted older medical patients. The clinical risk factors for mortality we found were in concordance with the literature. A high score on the Charlson co-morbidity index,15,18,25,26 more functional impairment,17,18,25,27 diagnosed malignancy18,25,28 and high urea nitrogen serum levels29 were all factors identified in literature as risk factors for mortality in this population. We did, however, not find age and delirium as risk factors for mortality in our study. Nevertheless, in patients who died within 90 days of admission, prevalence of delirium was almost twice as high as in patients who survived. Delirium was, however, only an independent risk factor for mortality in the univariate analysis. Delirium was strongly correlated with the Barthel index in the multivariate analysis. This indicates that delirious patients were those patients with more functional impairment. In other studies delirium was a risk factor for mortality, but these studies measured pre-morbid functional status, whereas we measured functional status at admission.30,31 We did, however, only measure prevalent delirium and not the incidence rate during hospitalization. Other studies have revealed that the incidence rate of delirium in medical patients varied widely, between 3% and 29%.32 Replacing the Barthel index for delirium worsened the baseline model for mortality. So, functioning is a more valid indicator for mortality and should be used when implementing this model. It is also easier for nurses and physicians to screen on daily functioning, as it is known that there is a large underrecognition of delirium in daily practice.33

The four simple-to-measure variables, we found as risk factors for mortality were fairly good in predicting mortality risk. Physicians and nurses should, therefore, be encouraged to use these variables in the process of prognostication. In a large study on prognostication in a sample of internists, the majority pointed out that prognostication was stressful to them, and they waited until patients asked them about prognosis.4 They also indicated they did not feel well trained in prognostication during their professional education. Using the variables identified in this study as a starting point in the process of prognostication could strengthen this process and enhance difficult decision making.

Interestingly, adding a clinical impression score from attending physicians, from attending nurses or from both disciplines did not significantly improve the discriminatory ability of mortality prediction. This is in contrast to some studies performed on the ICU where predictions of both physicians and nurses were sometimes even better than standardized prediction models.7 Notwithstanding the inherent differences between the predictive ability of models in these two settings there might also be some other explanations. One possible explanation is that both physicians and nurses on ICUs were better trained and physicians had more clinical experience. In our study we asked residents, who were still in training, to give a clinical impression score of patients’ disease. And although nurses working on general medical wards in the Netherlands do have a bachelor’s degree in nursing, they often do not have an extra training or specialization as ICU nurses do. It is fair to assume that, with extra training and, for physicians, having extra years of clinical experience, prognostication might be improved. At least this was shown in studies in cancer patients where faculty members gave more accurate predictions. Thus, it would be of surplus value to know if faculty members or geriatricians are better in predicting mortality risk than residents. If so, their opinion should be more actively used in the prognostication process.

A second possible explanation is that physicians and nurses working on an ICU have more information on clinical variables, as patients on these wards are monitored continuously. This gives a continuous stream of prognostic information to physicians and nurses, which might improve the mortality prediction.

Further research on prognostication and risk factors for mortality in this population should focus on several points. First, more possible modifiable risk factors for mortality should be studied, such as malnutrition, decubitus, depression, functional decline just before the acute hospitalization and polypharmacy. These are common problems in geriatric patients which might be related to mortality and may be taken care of during hospitalization. Secondly, research should further focus on methods to improve prognostication performance. What could be beneficial in this process is developing risk profiles for patients. This could assist nurses and physicians in prognostication, but it is also useful in the communication with patients and relatives about hospital outcomes, expectations and burden of treatment. Finally, aside from discrimination measures, such as the area under the ROC curve, accuracy and calibration measures should be investigated. While the area under the ROC curve reveals the ability to discriminate between survivors and non-survivors, it is not sensitive to calibration. Measures such as the Hosmer–Lemeshow statistics and the Brier score should be investigated as well.

Our results could have been biased due to selective inclusion of patients as not all patients gave informed consent or were not included within 48 hours. However, the short-term mortality rate we found of acutely hospitalized elderly (23%) was within the range reported by other studies, between 15% and 48%.2,15,18 In our study we only interviewed nurses and physicians separately from each other, but it would be interesting to know how discussions would affect the opinion of both disciplines, especially in patients where opinions diverge extensively. This might improve prognostication.

In conclusion, this study showed that a baseline set of four clinical variables—functioning, co-morbidity, malignancy and urea nitrogen serum level—can predict mortality of acutely hospitalized older patients quite well. Adding the clinical impression score of nurses or physicians did not improve the discriminating ability of the prediction model. Nurses and physicians should be encouraged to use these four factors in the process of prognostication and clinical decision-making as it gives patients more clarity about their prognosis. These factors are easy to collect and therefore useful for the clinical practice, where the group of older medical patients is rapidly increasing.