Clinician’s capsule

What is known about the topic?

Self-assessment tools may help increase compliance to geriatric ED guidelines and to better identify patients with impaired function.

What did this study ask?

What is the inter-rater reliability and feasibility of the Older Americans Resources and Services scale as administered by a research assistant versus self-assessment by older patients?

What did this study find?

Self-assessing functional status by older ED patients is feasible, and moderate-to-good inter-rater reliability results were obtained.

Why does this study matter to clinicians?

A self-assessed Older Americans Resources and Services scale may help clinicians identify patients in need of further geriatric/functional assessment who may otherwise have been left unscreened.

Introduction

Geriatric Emergency Department (ED) guidelines recommend the screening of older ED patients for geriatric syndromes [1], including falls and functional decline. Even though ED clinicians are well positioned to perform such screening, the time pressured, over-crowded environment renders this difficult and compliance issues have been observed [2]. Patient self-assessment could be a solution to this problem, thus providing information to ED clinicians without adding to their workload. A number of self-assessment tools are available [3] and could facilitate geriatric syndrome screening.

Baseline functional impairment and impaired mobility are common risk factors for geriatric syndromes [4]. It was also shown that self-assessed general, mental, and physical health was predictive of functional decline and mortality [5]. However, there are few available validated functional status self-assessment tools [6], and those do not always allow assessment of all the functional dimensions of activities of daily living and instrumental activities of daily living.

The Older Americans Resources and Services scale [7] is widely used to quantify patients’ ability to perform seven activities of daily living and seven instrumental activities of daily living. Its reliability as a self-assessment tool in the ED has never been assessed. The objective of this study is to evaluate the inter-rater reliability and feasibility of the self-assessed Older Americans Resources and Services scale using a tablet compared to its standard administration by a research assistant in older ED patients.

Methods

Study design and time period

This is a planned sub-analysis of a randomized cross-over pilot study that took place between 2018/05 and 2018/07 [8]. The main study aimed at assessing the acceptability of older patient self-assessment in the ED, compared to standard assessment by research assistants. A simple randomization list was computed and determined which assessment patients experienced first (research assistant assessment/self-assessment using a tablet) and each group contained an equal number of patients. The study was performed in the ED using a pragmatic approach and research assistants were not blinded to the randomization list.

Study setting and population

This study was conducted at the CHU de Québec-Université Laval (Hôpital de l'Enfant-Jésus). Trained research assistants used the ED’s patient tracking software to identify potential participants and obtained informed consent. Inclusion criteria: ED patients aged ≥ 65. Exclusion criteria: (1) unstable medical condition; (2) unable to speak French; (3) physical condition preventing the use of the tablet.

Outcome measures

Research assistants collected socio-demographic data and assessed baseline cognitive status using the Montreal Cognitive Assessment. Information regarding comorbidities was collected from medical charts using the Charlson Comorbidity Index.

The main outcome of this study was the inter-rater reliability of the Older Americans Resources and Services scale scoring between trained research assistants and older patients. The Older Americans Resources and Services scale is a 28-point validated tool that quantifies patients’ ability to perform seven activities of daily living (eating, dressing, grooming, walking, transfers, bathing, and using the toilet) and seven instrumental activities of daily living (using the telephone, driving or using public transportation, shopping, cooking, housework, managing medications, and managing finances) [7]. A higher score on this scale indicates a higher level of function. The same tool was modified for self-assessment using a tablet by reformulating the questions in the first person (i.e., "Can you eat…?" became "I can eat"), but the same multiple-choice answers were presented.

Our feasibility outcome was the completion rate of the self-assessment.

Data analysis

Descriptive analyses were performed for socio-demographic variables and baseline evaluations and completion rate. Intraclass correlation [ICC(3,1)] were calculated according to Shrout and Fleiss. Analyses were performed using SAS, version 9.4 (SAS Institute, Inc., Cary, NC).

Sample size

Because this is a sub-analysis, sample size was determined for the primary outcome of the main trial [8].

Results

A total of 67 patients were included in the main study, 60 of which completed self-assessment. Mean age was 74.4 ± 7.6 years and 34 (56.7%) participants were women. Mean MoCA score was 22.8 ± 4.2 (see Online resource 1).

Mean Older Americans Resources and Services scale score according to research assistant was 25.1 ± 3.3 and mean self-assessed Older Americans Resources and Services scale score was 26.4 ± 2.5 [ICC: 0.8 (95% CI: 0.7–0.9)]. Mean activities of daily living scores were 12.5 ± 1.8 for research assistant assessment and 13.5 ± 0.9 for self-assessment [ICC: 0.6 (95% CI: 0.4–0.7)]. Mean instrumental activities of daily living scores were 12.6 ± 1.8 and 12.9 ± 1.8 for research assistant assessment and self-assessment, respectively [ICC: 0.9 (95% CI: 0.8–0.9)]. Table 1 shows mean scores for each Older Americans Resources and Services scale item according to research assistant and according to the study participant.

Table 1 Older Americans Resources and Services scale scores according to research assistants compared to self-assessed
Full size table

We obtained a 90% completion rate (60/67), as 7 patients refused to self-assess. Reasons for refusal are mostly related to the use of technology [8].

Discussion

Interpretation of findings

Our results show a good overall inter-rater reliability for the Older Americans Resources and Services scale, with a 1.3-point difference between patients and the research assistants giving overall lower scores. When dichotomizing the Older Americans Resources and Services scale’s components, activities of daily living assessment obtained a moderate reliability (1-point difference), while instrumental activities of daily living assessments had a good inter-rater reliability (0.3-point difference). Our good completion rate confirms that self-assessing functional status is feasible for some older ED patients.

Comparison to previous studies

The literature on functional status self-assessment is scant, especially in an older patient population. A 2018 study aimed to validate the EASYCare Standard 2010 (49-item questionnaire assessing health, functional status, and general well-being) for self-assessment by older adults [9]. No significant differences were found and excellent-to-good agreement between self-assessment and professional scores were obtained in their population of 100 volunteer community-dwelling older adults (aged ≥ 60). Even though their population is different from ours as they included non-frail volunteers in seniors’ centers, and the tools they used were different, our data also suggest that self-assessment is possible in some older adults and that the results could be reliable.

Strengths and limitations

This study’s limitations include single-center convenience sampling due to logistical constraints. Functional abilities may have been overestimated when self-assessed or under-estimated when assessed by research assistants. However, given the detailed training received by research assistants and the fact that the Older Americans Resources and Services scale has been widely used and validated, the first hypothesis is more likely. Because patients underwent both types of evaluations, the interviews were a little lengthy, and patients may have been less focused during the second type of assessment. Our randomized cross-over design aimed to minimize this, as well as a potential order bias.

Clinical implications

Even if non-statistically significant, a one-point difference means an older adult unable to perform one activities of daily living or instrumental activities of daily living may go undetected with self-assessment alone. Nevertheless, since a screening-based preventive approach is recommended [1], a self-assessed score may identify patients in need of further geriatric/functional assessment who may otherwise have been left unscreened. The potential impacts of this research on clinical care include: providing ED clinicians with valuable information without adding to their workload, better clinical assessments/follow-up plans for older ED patients, and better compliance to geriatric ED guidelines. In today's context, where there is a particular emphasis on a patient-centered approach, the patient’s role in the management of their health is being radically transformed from a passive care recipient to a proactive partner, making self-assessment tools even more relevant.

Research implications

Even though further research with larger sample size is required before its implementation in EDs, this work could serve as a basis for a new area of research as machine-learning and digitalisation slowly emerge into the field of emergency medicine.

Conclusions

The study confirms that self-assessment of functional status using the Older American Resources and Services Scale is feasible for some older patients, and moderate-to-good inter-rater reliability results were obtained.