To assess the inter-rater reliability and feasibility of the self-assessed Older Americans Resources and Services scale compared to its administration by a research assistant in older Emergency Department (ED) patients.
This is a planned sub-analysis of a single-center randomized cross-over pilot study. A convenience sample of ED patients aged ≥ 65 was constituted at the CHU de Québec-Université Laval (Hôpital de l'Enfant-Jésus) between 2018/05 and 2018/07. Research assistants assessed participants’ functional status using the Older Americans Resources and Services scale and patients self-assessed using a modified Older Americans Resources and Services scale. Test administration order was randomized. The main outcome, inter-rater reliability, was measured using intraclass correlation (ICC). Feasibility was measured using self-assessment completion rate.
67 patients were included and 60 completed self-assessment. Mean age was 74.4 ± 7.6 and 34 (56.7%) participants were women. Mean research assistant-assessed Older Americans Resources and Services scale score was 25.1 ± 3.3, while mean self-assessed Older Americans Resources and Services scale score was 26.4 ± 2.5 [ICC: 0.8 (95% CI: 0.7–0.9)]. Mean activities of daily living scores were 12.5 ± 1.8 for research assistant assessment and 13.5 ± 0.9 for self-assessment [ICC: 0.6 (95% CI: 0.4–0.7)]. Mean instrumental activities of daily living scores were 12.6 ± 1.8 and 12.9 ± 1.8 for research assistant assessment and self-assessment, respectively [ICC: 0.9 (95% CI: 0.8–0.9)].
Our results indicate that self-assessment of functional status by older ED patients is feasible, and good-to-moderate inter-rater reliability results were obtained. A self-assessed score may identify patients in need of further geriatric/functional assessment who may otherwise have been left unscreened.
Évaluer la fidélité interjuges et la faisabilité pour les patients âgés de s’autoévaluer avec l’outil Older Americans Resources and Services scale au Département d’urgence (DU) comparativement à son administration par un assistant de recherche.
Il s'agit d'une sous-analyse planifiée d'une étude pilote croisée randomisée unicentrique. Un échantillon de convenance de patients âgés de ≥ 65 ans consultant au DU du CHU de Québec-Université Laval (Hôpital de l’Enfant-Jésus) entre 2018/05 - 2018/07 a été constitué. Les assistants de recherche ont évalué le statut fonctionnel des participants en utilisant le Older Americans Resources and Services scale et les patients se sont autoévalués en utilisant une version modifiée de cet outil. L'ordre des types d’administration a été randomisé. L’issue principale, la fidélité interjuges, a été mesurée à l'aide de coefficients de corrélation intraclasse (ICC). La faisabilité a été mesurée à l'aide du taux d'achèvement de l'autoévaluation.
67 patients ont été inclus dans l’étude principale dont 60 ont complété l'autoévaluation. L'âge moyen était de 74,4 ± 7,6 ans et 34 (56,7 %) participants étaient des femmes. Le score moyen du Older Americans Resources and Services scale évalué par l'assistant de recherche était de 25,1 ± 3,3 tandis que le score moyen autoévalué était de 26,4±2,5 (ICC: 0,81 [95% CI : 0,7-0,9]). Les scores moyens des 7 activités de la vie quotidienne étaient de 12,5 ± 1,8 pour l'évaluation de l'assistant de recherche et de 13,5 ± 0,9 pour l'autoévaluation (ICC:0,6 [95% CI : 0,4-0,7]). Les scores moyens des 7 activités instrumentales de la vie quotidienne étaient de 12,6 ± 1,8 et 12,9 ± 1,8 pour l'évaluation de l'assistant de recherche et l'autoévaluation, respectivement (ICC:0,9 [IC 95% : 0,8-0,9]).
Nos résultats indiquent que l'autoévaluation du statut fonctionnel par les patients âgés est possible au DU, et des résultats de fidélité interjuges allant de bons à modérés ont été obtenus. L’autoévaluation avec le Older Americans Resources and Services scale pourrait permettre d'identifier des patients nécessitant une évaluation gériatrique ou fonctionnelle plus approfondie, qui n'auraient pas été dépistés autrement.
What is known about the topic?
Self-assessment tools may help increase compliance to geriatric ED guidelines and to better identify patients with impaired function.
What did this study ask?
What is the inter-rater reliability and feasibility of the Older Americans Resources and Services scale as administered by a research assistant versus self-assessment by older patients?
What did this study find?
Self-assessing functional status by older ED patients is feasible, and moderate-to-good inter-rater reliability results were obtained.
Why does this study matter to clinicians?
A self-assessed Older Americans Resources and Services scale may help clinicians identify patients in need of further geriatric/functional assessment who may otherwise have been left unscreened.
Geriatric Emergency Department (ED) guidelines recommend the screening of older ED patients for geriatric syndromes , including falls and functional decline. Even though ED clinicians are well positioned to perform such screening, the time pressured, over-crowded environment renders this difficult and compliance issues have been observed . Patient self-assessment could be a solution to this problem, thus providing information to ED clinicians without adding to their workload. A number of self-assessment tools are available  and could facilitate geriatric syndrome screening.
Baseline functional impairment and impaired mobility are common risk factors for geriatric syndromes . It was also shown that self-assessed general, mental, and physical health was predictive of functional decline and mortality . However, there are few available validated functional status self-assessment tools , and those do not always allow assessment of all the functional dimensions of activities of daily living and instrumental activities of daily living.
The Older Americans Resources and Services scale  is widely used to quantify patients’ ability to perform seven activities of daily living and seven instrumental activities of daily living. Its reliability as a self-assessment tool in the ED has never been assessed. The objective of this study is to evaluate the inter-rater reliability and feasibility of the self-assessed Older Americans Resources and Services scale using a tablet compared to its standard administration by a research assistant in older ED patients.
Study design and time period
This is a planned sub-analysis of a randomized cross-over pilot study that took place between 2018/05 and 2018/07 . The main study aimed at assessing the acceptability of older patient self-assessment in the ED, compared to standard assessment by research assistants. A simple randomization list was computed and determined which assessment patients experienced first (research assistant assessment/self-assessment using a tablet) and each group contained an equal number of patients. The study was performed in the ED using a pragmatic approach and research assistants were not blinded to the randomization list.
Study setting and population
This study was conducted at the CHU de Québec-Université Laval (Hôpital de l'Enfant-Jésus). Trained research assistants used the ED’s patient tracking software to identify potential participants and obtained informed consent. Inclusion criteria: ED patients aged ≥ 65. Exclusion criteria: (1) unstable medical condition; (2) unable to speak French; (3) physical condition preventing the use of the tablet.
Research assistants collected socio-demographic data and assessed baseline cognitive status using the Montreal Cognitive Assessment. Information regarding comorbidities was collected from medical charts using the Charlson Comorbidity Index.
The main outcome of this study was the inter-rater reliability of the Older Americans Resources and Services scale scoring between trained research assistants and older patients. The Older Americans Resources and Services scale is a 28-point validated tool that quantifies patients’ ability to perform seven activities of daily living (eating, dressing, grooming, walking, transfers, bathing, and using the toilet) and seven instrumental activities of daily living (using the telephone, driving or using public transportation, shopping, cooking, housework, managing medications, and managing finances) . A higher score on this scale indicates a higher level of function. The same tool was modified for self-assessment using a tablet by reformulating the questions in the first person (i.e., "Can you eat…?" became "I can eat"), but the same multiple-choice answers were presented.
Our feasibility outcome was the completion rate of the self-assessment.
Descriptive analyses were performed for socio-demographic variables and baseline evaluations and completion rate. Intraclass correlation [ICC(3,1)] were calculated according to Shrout and Fleiss. Analyses were performed using SAS, version 9.4 (SAS Institute, Inc., Cary, NC).
Because this is a sub-analysis, sample size was determined for the primary outcome of the main trial .
A total of 67 patients were included in the main study, 60 of which completed self-assessment. Mean age was 74.4 ± 7.6 years and 34 (56.7%) participants were women. Mean MoCA score was 22.8 ± 4.2 (see Online resource 1).
Mean Older Americans Resources and Services scale score according to research assistant was 25.1 ± 3.3 and mean self-assessed Older Americans Resources and Services scale score was 26.4 ± 2.5 [ICC: 0.8 (95% CI: 0.7–0.9)]. Mean activities of daily living scores were 12.5 ± 1.8 for research assistant assessment and 13.5 ± 0.9 for self-assessment [ICC: 0.6 (95% CI: 0.4–0.7)]. Mean instrumental activities of daily living scores were 12.6 ± 1.8 and 12.9 ± 1.8 for research assistant assessment and self-assessment, respectively [ICC: 0.9 (95% CI: 0.8–0.9)]. Table 1 shows mean scores for each Older Americans Resources and Services scale item according to research assistant and according to the study participant.
We obtained a 90% completion rate (60/67), as 7 patients refused to self-assess. Reasons for refusal are mostly related to the use of technology .
Interpretation of findings
Our results show a good overall inter-rater reliability for the Older Americans Resources and Services scale, with a 1.3-point difference between patients and the research assistants giving overall lower scores. When dichotomizing the Older Americans Resources and Services scale’s components, activities of daily living assessment obtained a moderate reliability (1-point difference), while instrumental activities of daily living assessments had a good inter-rater reliability (0.3-point difference). Our good completion rate confirms that self-assessing functional status is feasible for some older ED patients.
Comparison to previous studies
The literature on functional status self-assessment is scant, especially in an older patient population. A 2018 study aimed to validate the EASYCare Standard 2010 (49-item questionnaire assessing health, functional status, and general well-being) for self-assessment by older adults . No significant differences were found and excellent-to-good agreement between self-assessment and professional scores were obtained in their population of 100 volunteer community-dwelling older adults (aged ≥ 60). Even though their population is different from ours as they included non-frail volunteers in seniors’ centers, and the tools they used were different, our data also suggest that self-assessment is possible in some older adults and that the results could be reliable.
Strengths and limitations
This study’s limitations include single-center convenience sampling due to logistical constraints. Functional abilities may have been overestimated when self-assessed or under-estimated when assessed by research assistants. However, given the detailed training received by research assistants and the fact that the Older Americans Resources and Services scale has been widely used and validated, the first hypothesis is more likely. Because patients underwent both types of evaluations, the interviews were a little lengthy, and patients may have been less focused during the second type of assessment. Our randomized cross-over design aimed to minimize this, as well as a potential order bias.
Even if non-statistically significant, a one-point difference means an older adult unable to perform one activities of daily living or instrumental activities of daily living may go undetected with self-assessment alone. Nevertheless, since a screening-based preventive approach is recommended , a self-assessed score may identify patients in need of further geriatric/functional assessment who may otherwise have been left unscreened. The potential impacts of this research on clinical care include: providing ED clinicians with valuable information without adding to their workload, better clinical assessments/follow-up plans for older ED patients, and better compliance to geriatric ED guidelines. In today's context, where there is a particular emphasis on a patient-centered approach, the patient’s role in the management of their health is being radically transformed from a passive care recipient to a proactive partner, making self-assessment tools even more relevant.
Even though further research with larger sample size is required before its implementation in EDs, this work could serve as a basis for a new area of research as machine-learning and digitalisation slowly emerge into the field of emergency medicine.
The study confirms that self-assessment of functional status using the Older American Resources and Services Scale is feasible for some older patients, and moderate-to-good inter-rater reliability results were obtained.
Availability of data and materials
Available upon reasonable request.
American College of Emergency Physicians, The American Geriatric Society, Emergency Nurses Association, Society for Academic Emergency Medicine. Geriatric Emergency Department Guidelines. Ann Emerg Med. 2014; 63(5). https://doi.org/10.1016/j.annemergmed.2014.02.008.
Asomaning N, Loftus C. Identification of seniors at risk (ISAR) screening tool in the emergency department: implementation using the plan-do-study-act model and validation results. J Emerg Nurs: JEN . 2014;40(4):357–64. https://doi.org/10.1016/j.jen.2013.08.014.
Canadian Institute for Health Information. PROMs Background Document. 2015. https://www.cihi.ca/sites/default/files/document/proms_background_may21_en-web.pdf.
Inouye SK, Studenski S, Tinetti ME, Kuchel GA. Geriatric syndromes: clinical, research, and policy implications of a core geriatric concept. J Am Geriatr Soc. 2007;55(5):780–91. https://doi.org/10.1111/j.1532-5415.2007.01156.x.
Lee Y. The predictive value of self assessed general, physical, and mental health on functional decline and mortality in older adults. J Epidemiol Community Health. 2000;54(2):123–9.
Ashford S, Brown S, Turner-Stokes L. Systematic review of patient-reported outcome measures for functional performance in the lower limb. J Rehabil Med. 2015;47(1):9–17.
Fillenbaum GG, Duke U. Multidimensional functional assessment of older adults: the Duke Older Americans Resources and Services procedures. vol xii, 179 p. Hillsdale, NJ: L. Erlbaum Associates; 1988.
Boucher V, Lamontagne ME, Lee J, Carmichael PH, Dery J, Emond M. Acceptability of older patients' self-assessment in theEmergency Department (ACCEPTED)-a randomised cross-over pilot trial. Age Ageing. 2019;48(6):875–80. https://doi.org/10.1093/ageing/afz084.
Tobis S, Jaracz K, Talarska D, Kropinska S, Zasadzka E, Pawlaczyk M, et al. Validity of the EASYCare Standard 2010 assessment instrument for self-assessment of health, independence, and well-being of older people living at home in Poland. Eur J Ageing. 2018;15(1):101–8. https://doi.org/10.1007/s10433-017-0422-7.
We would like to thank Pierre-Hugues Carmichael, Alexandra Nadeau, Matthieu Robitaille, Joannie Blais, and Elisabeth Nguyen for their help and support with this project. Dr. Jacques Lee is supported by the Schwartz/ Reisman Emergency Medicine Institute Inaugural Research Chair in Geriatric Emergency Medicine.
This work was supported by the Fondation du CHU de Québec-Université Laval (grant number 2853). VB has received a scholarship from the Université Laval, and from the Fonds de recherche du Québec-Santé (FRQ-S) and the Strategy for Patient-Oriented Research (SPOR). Those sponsors played no role in the design or in the conduct of this study.
Conflict of interest
The CHU de Québec-Université Laval Research Ethics Board approved this study (#2017–3527).
Consent to participate
Informed consent was obtained from all individual participants included in the study.
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Boucher, V., Lamontagne, ME., Lee, J. et al. Self-assessment of functional status in older emergency department patients: a cross-over randomized pilot trial. Can J Emerg Med 23, 337–341 (2021). https://doi.org/10.1007/s43678-020-00073-9
- Self-rated health
- Patient-reported outcome measure
- Functional status
- Emergency department