Risk stratification of older patients in the emergency department (ED) is seen as a promising and efficient solution for handling the increase in demand for geriatric emergency medicine. Previously, the predictive validity of commonly used tools for risk stratification, such as the identification of seniors at risk (ISAR), have found only limited evidence in German geriatric patient samples. Given that the adverse outcomes in question, such as rehospitalization, nursing home admission and mortality, are substantially associated with cognitive impairment, the potential of the short portable mental status questionnaire (SPMSQ) as a tool for risk stratification of older ED patients was investigated.
To estimate the predictive validity of the SPMSQ for a composite endpoint of adverse events (e.g. rehospitalization, nursing home admission and mortality).
This was a prospective cohort study with 260 patients aged 70 years and above, recruited in a cardiology ED. Patients with a likely life-expectancy below 24 h were excluded. Follow-up examinations were conducted at 1, 3, 6 and 12 month(s) after recruitment.
The SPMSQ was found to be a significant predictor of adverse outcomes not at 1 month (area under the curve, AUC 0.55, 95% confidence interval, CI 0.46–0.63) but at 3 months (AUC 0.61, 95% CI 0.54–0.68), 6 months (AUC 0.63, 95% CI 0.56–0.70) and 12 months (AUC 0.63, 95% CI 0.56–0.70) after initial contact.
For longer periods of observation the SPMSQ can be a predictor of a composite endpoint of adverse outcomes even when controlled for a range of confounders. Its characteristics, specifically the low sensitivity, make it unsuitable as an accurate risk stratification tool on its own.
Die Risikostratifizierung von älteren Patienten in der Notaufnahme gilt als vielversprechender und effizienter Lösungsansatz, um die steigende Nachfrage nach geriatrischer Notfallmedizin zu bewältigen. Bisher zeigte sich die prädiktive Validität des am häufigsten eingesetzten Instruments, dem Identification of Seniors at Risk (ISAR), für deutsche Stichproben jedoch als begrenzt. Da die interessierenden Outcomes, wie Rehospitalisierung, Pflegeheimübersiedlung und Mortalität deutlich mit kognitiver Beeinträchtigung zusammenhängen, war es unser Ziel, das Potenzial des Short Portable Mental Status Questionnaire (SPMSQ) als Instrument zur Risikostratifizierung von älteren Notaufnahmepatienten zu überprüfen.
Schätzung der prädiktiven Validität des SPMSQ für einen kombinierten Endpunkt adverser Outcomes (Rehospitalisierung, Pflegeheimübersiedlung und Mortalität).
Daten und Methode
Es handelte sich um eine prospektive Kohortenstudie mit 260 Patienten im Alter von mindestens 70 Jahren, die in einer kardiologischen Notaufnahme rekrutiert worden waren. Patienten mit einer Lebenserwartung von unter 24 h wurden exkludiert. Follow-ups fanden nach 1, 3, 6, und 12 Monaten statt.
SPMSQ war signifikanter Prädiktor für den kombinierten Endpunkt adverser Outcomes zwar nicht für 1 Monat (AUC: 0,55; 95 % KI 0,46–0,63), aber für 3 Monate (AUC: 0,61; 95 % KI 0,54–0,68), 6 Monate (AUC: 0,63; 95 % KI 0,56–0,70) und 12 Monate (AUC: 0,63; 95 % KI 0,56–0,70) nach Erstkontakt.
Für längere Beobachtungszeiträume scheint der SPMSQ, auch unter Kontrolle potenziell konfundierender Variablen, ein Prädiktor für adverse Outcomes zu sein. Seine Eigenschaften, insbesondere die niedrige Sensitivität, machen ihn jedoch für den Einsatz als alleiniges Screening-Instrument wenig tauglich.
For many older patients the emergency department (ED) is an entry point into the healthcare system. Geriatric emergency medicine is a resource intensive process and with ongoing demographic aging the already high demand is expected to rise even further . At present, special needs of geriatric patients are likely to be overlooked in the ED . To face this challenge and improve pathways towards optimal geriatric healthcare, the geriatric medical concept of the state government of Baden-Württemberg recommends screening older patients for those at high risk for adverse outcomes at the very beginning of the medical treatment, which is often in the ED of acute care hospitals .
Fundamentally, risk stratification is intended to be part of a two-step process: first, a screening tool is used for the brief risk stratification of all presenting older patients. Second, those patients that screen positive undergo a multimodal geriatric assessment or some other elaborate diagnostic procedure, which then in consequence enables the clinician to reliably identify the needs of geriatric patients [4, 22]. Risk stratification of older patients in the ED therefore strives to enable the healthcare system to manage its resources as efficiently as possible. Additionally, the goal is to provide the identified high-risk patients with a more thorough diagnostic process than exerted in usual ED care; however, despite a growing body of relevant literature, implementing risk stratification processes targeted at older patients in German EDs seems to fall short . The reasons for this situation include the complex characteristics of the ED setting, ambiguous results about the validity of the potentially useful instruments, as well as the questionable clinical utility.
Characteristics of the ED setting and risk stratification with identification of seniors at risk (ISAR)
The key to any systematic screening in the ED is feasibility as ED settings provide limited time and room as well as often noisy and busy surroundings. Not only are multimodal geriatric assessments not suited for this environment, some of the screening methods designed specifically for the risk stratification of older adults are likely too long and effortful for efficient use in EDs [14, 36]. In a consensus statement for the identification of geriatric patients in the ED setting in Germany, the German Geriatric Society as well as the German Society of Gerontology and Geriatrics mentioned a number of potential tools for the risk stratification of older adults in the ED setting [6, 16, 21]. Specifically, the use of the identification of seniors at risk screening (ISAR) tool was recommended for settings where no other instruments or geriatric expertise are available, mainly because of the ISAR’s simple administration and its existing extensive body of international literature ; however, while positive and negative results on the predictive validity of the ISAR have been reported in the international literature [9, 29], meta-analyses found it to have either insufficient or only modest predictive accuracy [5, 13]. In light of negative results, Hwang and Carpenter argued that while more accurate tools are being developed the ISAR should continue to be used to ensure awareness and understanding of geriatric patients beyond the acute problem . The only study that examined the predictive validity of the ISAR in a German sample of ED patients found it to have “acceptable” predictive validity . There are two major aspects that complicate the integration of the ISAR tool into the clinical routine: first, with the risk of adverse events (e.g. rehospitalization, nursing home admission and mortality) it measures a construct of general risk, which is difficult to grasp and unspecific regarding its medical indications. Second, in the studies conducted using German ED samples, the ISAR classified more than 80% of patients as high-risk patients [30, 37], thus questioning its specificity and ability to strengthen the effective use of resources.
Addressing the previously mentioned concerns and the suggestion made in the literature to explore alternative variables for the risk stratification of older ED patients , the objective of this study was to examine the predictive validity of the cognitive screening tool short portable mental status questionnaire (SPMSQ ) for adverse events after an ED hospital stay. The SPMSQ is an established short cognitive test that has already found application in the ED setting ; it has also been shown to predict adverse events in older patients [21, 31]. Furthermore, cognitive impairment is common but often remains undetected or clinically unused in older ED patients . According to the recommendations of the Society for Academic Emergency Medicine cognitive screening can even be seen as one of the major quality indicators in geriatric emergency medicine . To the best of our knowledge, there is no study that has examined the predictive validity of the SPMSQ for adverse events in a sample of German ED patients across a considerable observational period. Consequently, due to the need for risk stratification and cognitive screening in the ED and the existing strong relationships between cognitive impairment and undesired outcomes, this study examined the suitability of cognitive screening with the SPMSQ as a tool for risk stratification up to a 1-year interval.
Study design and participants
This was a single center, exploratory and prospective cohort study with 260 consecutively recruited ED patients. The data on the predictive validity of the SPMSQ were drawn from the usual care group of an ongoing intervention study. The study was approved by the ethics committee at the medical faculty of Heidelberg University (S-455/2016). Since the study was based on the usual care group of an ongoing intervention study, no specific power calculation was conducted; however, the overall sample size of 260 can be qualified as similar to comparable studies in the existing literature (e.g. [3, 29]).
Recruitment was done by the first author and took place 7 days a week during the day shifts in a cardiological ED (chest pain unit) affiliated with a university hospital, with 12 beds in the ED and a total of 114 beds in the associated cardiology department. The first patient was recruited in July 2017 and the last patient in May 2018. Patients aged 70 years or above were included. Exclusion criteria were missing informed consent or a likely life expectancy of less than 24 h. Due to procedural reasons, patients that had to undergo isolated care were also not included in the study. Patients were asked to participate in the study after the initial medical examination. On agreement, a respective informed consent document to participate in all data waves was signed. Follow-ups were conducted 1, 3, 6 and 12 months after initial contact via telephone interviews. The data were combined with hospital files, online death recording via obituaries and registry office information.
Measures and outcomes
In addition to several demographic characteristics, the patient’s cognitive performance was assessed by use of the SPMSQ tool, which can be retrieved from the original publication  or other available resources [12, 18]. The SPMSQ score is derived from the amount of errors based on a 10-item list by coding errors as “1” and correct answers as “0”. Items include tasks on orientation (“What is the date today?”), memory (“What was your mother’s maiden name?”) and attention (“Subtract 3 from 20 and keep subtracting 3 from each new number, all the way down”). Thus, individual cognitive scores ranged from 0 to 10 errors, with lower values indicating better cognitive performance.
As outcomes unplanned rehospitalizations (ED and general) were recorded as well as nursing home admissions and all-cause mortality. For the primary analysis all outcomes were combined into a binary coded composite adverse outcome variable, meaning at least one of the events had occurred within 1 month after initial contact, if the composite outcome was coded as positive for the first follow-up. This composite outcome was examined for primarily 1 month, but further also for 3, 6 and 12 months after initial contact. For a secondary analysis the all-cause mortality within 1 year after initial contact was also examined.
Descriptive statistics of the sample were calculated using means and standard deviations for continuous normally distributed variables, median and interquartile range for continuous/discrete but not normally distributed variables and absolute and relative frequencies for categorical variables. Group differences across these variables were calculated for cognitively impaired and unimpaired patients (SPMSQ error score ≥3 and <3). Given the binary coding of the primary outcome, logistic regression models were used to test for the relationship between the SPMSQ score and the primary outcome. In these analyses, pairwise deletion was used for missing data. Receiver operating characteristic (ROC) curves were used to illustrate the discriminatory performance of the cognitive risk screening. For the secondary outcome, a survival analysis was conducted with Kaplan-Meier estimates. Statistical analyses were performed using SAS Version 9.4 (SAS Institute Inc., Cary, NC, USA).
Descriptive statistics for the 260 included patients can be found in Table 1. Patients were mostly male 163/260 (63%). The mean age was 79 years (SD = 5.97 years), 37/260 (14%) of the patients had no education beyond the basic school level, 156/260 (60%) completed an apprenticeship, 48/260 (19%) finished a university degree and 19/260 (7%) held a PhD. As also displayed in Table 1, using the SPMSQ cut-off of ≥3 errors, patients identified by the SPMSQ as cognitively impaired (60/260 or 23%) were older (MDiff = 3.91 years; t(91) = 4.44; p <0.001) and less educated (U = 4264.00; p <0.001; r = −0.24) than those with a negative SPMSQ result (200/260 or 77%).
Results for logistic regression regarding the composite outcome and related patient attrition are reported in Table 2. The composite endpoint occurred in 64/250 (26%) at 1 month, 117/249 (47%) at 3 months, 145/245 (59%) at 6 months and 165/245 (67%) at 12 months after initial contact. Thus, until 12 months after initial contact 15/260 (6%) patients or indirect follow-ups provided insufficient information on the outcomes for the cases to be included in the analysis. In the univariate logistic regression model SPMSQ was a statistically significant predictor of the composite endpoint at 3 months (odds ratio, OR: 1.34, 95% confidence interval, CI 1.12–1.60), at 6 months (OR: 1.47, 95% CI 1.20–1.80) and at 12 months (OR: 1.54, 95% CI 1.22–1.93) but not at 1 month (OR: 1.13, 95%CI 0.94–1.36) after initial contact. Statistical significance was retained, when controlling for a range of possible confounders (e.g. age, sex, education, body mass index, and comorbidity).
Fig. 1 displays the exact discriminatory performance of the SPMSQ score (continuous) for those time points where it was found to be a significant predictor of the composite outcome, hence the 3, 6, and 12-month intervals. Associated areas under the curve (AUC) for all time points were 0.55 (95% CI 0.46–0.63) for 1 month, 0.61 (95% CI 0.54–0.68) for 3 months, 0.63 (95% CI 0.56–0.70) for 6 months and 0.63 (95% CI 0.56–0.70) for 12 months after initial contact. Sensitivities and specificities of the SPMSQ for the prediction of the composite outcome across different possible cut-off values can be found in the Supplementary material Table 1. For the time points where SPMSQ was a significant predictor of adverse outcomes, the associated sensitivities and specificities using the ≥3 errors cut-off were as following: 34% sensitivity and 88% specificity (3 months), 30% sensitivity and 91% specificity (6 months) and 29% sensitivity and 94% specificity (12 months).
Kaplan-Meier curves were used to analyze patient survival probabilities depending on positive or negative SPMSQ results (with cut-off ≥3; see Fig. 2). Of the total sample (N = 260) one patient with negative SPMSQ was lost to follow-up at 6 months after initial contact and therefore censored. Overall, the log-rank test showed no statistically significant differences between the resulting two survival curves, although there was a trend that lowered cognitive performance was associated with higher all-cause mortality (χ2 (1) = 2.92, p = 0.087).
To the best of our knowledge this is the first study that examined the predictive validity of the cognitive screening tool SPMSQ for the risk stratification of older ED patients regarding adverse outcomes (e.g. rehospitalization, nursing home admission, mortality). The key findings can be summarized as following:
SPMSQ seems to be a useful predictor of adverse outcomes in older German ED patients; however, not for brief (e.g. 1 month) but only for longer observation periods (e.g. 1 year). This relationship remained stable when controlling for a range of confounders.
The suggested ≥3 errors cut-off appeared to be the most useful when predicting adverse outcomes at different points in time.
While the specificity is high sensitivity is low. Overall, these characteristics can be regarded as insufficient for use as a screening tool.
Although a tendency was observed for a decreased 1‑year survival probability of patients with a SPMSQ score of ≥3 errors when compared to those with <3 errors, results were not statistically significant.
Several limitations must be considered when interpreting the results. Interviewer bias may be possible because data collection and follow-up were conducted entirely by the first author of this study, who was not blinded regarding the study goal; however, fully standardized measures and objective outcomes were used that are not open to interpretation. Even though this was an exploratory study, multiplicity should also be addressed. The results remained significant when multiplicity was adjusted for by using the established Bonferroni-Holm correction of the alpha significance level. Multiplicity is therefore seen as a relatively minor problem in this analysis. Another possible source of bias is that screening was only possible with patients where an informed consent procedure was feasible. Patients with very severe medical problems were consequently excluded. Since these patients are obviously high-risk patients, they cannot be regarded as the target group for geriatric screening; however, the fact that patients undergoing isolated care could not be included poses a risk to the external validity of the results presented in this article. Additionally, due to mostly conducting follow-ups via phone calls, it was not possible to provide reliable incidences of the outcomes that were combined into the composite outcome separately. For example, the outcome mortality may have masked a previous rehospitalization because it was difficult to retrieve this information. Finally, it must be emphasized that patients were recruited from a university affiliated cardiology ED that may not be representative of the general ED population due to different morbidities and due to its popularity with private patients from different locations. This also can be seen as a risk to the external validity.
In longer observation periods the cognitive screening tool SPMSQ can be a predictor of adverse outcomes, even when controlled for a range of relevant confounders. Its characteristics, however, specifically the low sensitivity, make it unsuitable as an accurate risk stratification tool alone. Combinations with other risk screening procedures may however be promising.
The SPMSQ proved to be feasible for use in the ED setting in this sample and was a predictor of adverse outcomes.
The SPMSQ does not have the capacity to replace risk stratification with common geriatric screening tools like the ISAR.
Further research into risk stratification with different cognitive screening tools and combinations with other risk stratification devices may produce results with higher sensitivity.
Aminzadeh F, Dalziel WB (2002) Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions. Ann Emerg Med 39:238–247
Bassuk SS, Wypij D, Berkmann LF (2000) Cognitive impairment and mortality in the community-dwelling elderly. Am J Epidemiol 151:676–688
Braes T, Flamaing J, Sterckx W et al (2009) Predicting the risk of functional decline in older patients admitted to the hospital: a comparison of three screening instruments. Age Ageing 38:600–603
Carpenter CR, Emond M (2016) Pragmatic barriers to assessing post-emergency department vulnerability for poor outcomes in an ageing society. Neth J Med 74:327–329
Carpenter CR, Shelton E, Fowler S et al (2015) Risk factors and screening instruments to predict adverse outcomes for undifferentiated older emergency department patients: a systematic review and meta—analysis. Acad Emerg Med 22:1–21
Cornette P, Swine C, Malhomme B et al (2005) Early evaluation of the risk of functional decline following hospitalization of older patients: development of a predictive tool. Eur J Public Health 16:203–208
De Gelder J, Lucke JA, Blomaard LC et al (2018) Optimization of the APOP screener to predict functional decline or mortality in older emergency department patients: Cross-validation in four prospective cohorts. Exp Gerontol 110:253–259
De Gelder J, Lucke JA, De Groot B et al (2016) Predicting adverse health outcomes in older emergency department patients: the APOP study. Neth J Med 74:342–352
Di Bari M, Salvi F, Roberts AT et al (2011) Prognostic stratification of elderly patients in the emergency department: a comparison between the “Identification of Seniors at Risk” and the “Silver Code”. J Gerontol A Biol Sci Med Sci 67:544–550
Farid K, Zhang Y, Bachelier D et al (2013) Cognitive impairment and malnutrition, predictors of all-cause mortality in hospitalized elderly subjects with cardiovascular disease. Arch Cardiovasc Dis 106:188–195
Fogg C, Meredith P, Culliford D et al (2019) Cognitive impairment is independently associated with mortality, extended hospital stays and early readmission of older people with emergency hospital admissions: a retrospective cohort study. Int J Nurs Stud. https://doi.org/10.1016/j.ijnurstu.2019.02.005
Gallo JJ (2006) Handbook of geriatric assessment. Jones & Bartlett Learning, Burlington
Galvin R, Gilleit Y, Wallace E et al (2017) Adverse outcomes in older adults attending emergency departments: a systematic review and meta-analysis of the Identification of Seniors At Risk (ISAR) screening tool. Age Ageing 46:179–186
Graf CE, Zekry D, Giannelli S et al (2011) Efficiency and applicability of comprehensive geriatric assessment in the Emergency Department: a systematic review. Aging Clin Exp Res 23:244–254
Hustey FM, Meldon SW, Smith MD et al (2003) The effect of mental status screening on the care of elderly emergency department patients. Ann Emerg Med 41:678–684
Hustey FM, Mion LC, Connor JT et al (2007) A brief risk stratification tool to predict functional decline in older adults discharged from emergency departments. J Am Geriatr Soc 55:1269–1274
Hwang U, Carpenter C (2015) Assessing geriatric vulnerability for post emergency department adverse outcomes: Challenges abound while progress is slow. Emerg Med J 33. https://doi.org/10.1136/emermed-2015-204983
Inouye SK (2003) The Confusion Assessment Method (CAM): training manual and coding guide
Johansson B, Zarit SH (1997) Early cognitive markers of the incidence of dementia and mortality: a longitudinal population—based study of the oldest old. Int J Geriatr Psychiatry 12:53–59
Lucke JA, De Gelder J, Heringhaus C et al (2018) Impaired cognition is associated with adverse outcome in older patients in the Emergency Department; the Acutely Presenting Older Patients (APOP) study. Age Ageing 47:679–684
Mccusker J, Bellavance F, Cardin S et al (1999) Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc 47:1229–1237
Mccusker J, Jacobs P, Dendukuri N et al (2003) Cost-effectiveness of a brief two-stage emergency department intervention for high-risk elders: results of a quasi-randomized controlled trial. Ann Emerg Med 41:45–56
O’donnell M, Teo K, Gao P et al (2012) Cognitive impairment and risk of cardiovascular events and mortality. Eur Heart J 33:1777–1786
Perna L, Wahl H‑W, Mons U et al (2014) Cognitive impairment, all-cause and cause-specific mortality among non-demented older adults. Age Ageing 44:445–451
Pfeiffer E (1975) A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc 23:433–441
Prückner S, Madler C (2009) Der demographische Wandel. Notfall + Rettungsmedizin 12:13
Salthouse TA (2016) Theoretical perspectives on cognitive aging
Salvi F, Morichi V, Grilli A et al (2007) The elderly in the emergency department: a critical review of problems and solutions. Intern Emerg Med 2:292–301
Salvi F, Morichi V, Grilli A et al (2009) Predictive validity of the identification of seniors at risk (ISAR) screening tool in elderly patients presenting to two Italian emergency departments. Aging Clin Exp Res 21:69–75
Singler K, Heppner HJ, Skutetzky A et al (2014) Predictive validity of the identification of seniors at risk screening tool in a german emergency department setting. Gerontology 60:413–419
Söderqvist A, Ekström W, Ponzer S et al (2009) Prediction of mortality in elderly patients with hip fractures: a two-year prospective study of 1,944 patients. Gerontology 55:496–504
Sozialministerium Baden-Württemberg (2014) Geriatriekonzept Baden-Württemberg 2014
Stuck AE, Siu AL, Wieland GD et al (1993) Comprehensive geriatric assessment: a meta-analysis of controlled trials. Lancet 342:1032–1036
Terrell KM, Hustey FM, Hwang U et al (2009) Quality indicators for geriatric emergency care. Acad Emerg Med 16:441–449
Thiem U, Greuel H, Reingräber A et al (2012) Positionspapier zur Identifizierung geriatrischer Patienten in Notaufnahmen in Deutschland. Z Gerontol Geriatr 45:310–314
Thiem U, Heppner HJ, Singler K (2015) Instruments to identify elderly patients in the emergency department in need of geriatric care. Z Gerontol Geriatr 48:4–9
Weinrebe W, Schiefer Y, Weckmuller K et al (2019) Does the identification of seniors at risk (ISAR) score effectively select geriatric patients on emergency admission? Aging Clin Exp Res. https://doi.org/10.1007/s40520-018-1105-8
This study was funded by the Robert Bosch Foundation within the Graduate Program People with Dementia in Acute Care Hospitals (GPPDACH), located at the Network Aging Research (NAR), University of Heidelberg, Germany. The open access publication was supported by Robert Bosch Stiftung.
Conflict of interest
A. Schönstein, H.-W. Wahl, H. A. Katus and A. Bahrmann declare that they have no competing interests. The supplement containing this article is not sponsored by industry.
All procedures performed in the study were in accordance with the ethical standards of the ethical board of the Medical Faculty at Heidelberg University and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants in the study.
Caption Electronic Supplementary Material
About this article
Cite this article
Schönstein, A., Wahl, HW., Katus, H.A. et al. SPMSQ for risk stratification of older patients in the emergency department. Z Gerontol Geriat 52, 222–228 (2019). https://doi.org/10.1007/s00391-019-01626-z
- Adverse outcomes
- Adverse Outcomes