Abstract
Objective
To validate an Italian version of the Rasch-Built Overall ALS Disability Scale (ROADS) in a broad population of patients and assess its longitudinal performance over time.
Methods
270 ALS patients referring to the Motor Neuron Disease Clinic of the University of Padova and Modena (Italy) accepted to compile the Italian version of the ROADS and results were correlated with the ALSFRSr and ALSAQ-40 scores, FVC values, and creatinine or albumin blood levels. To verify test–retest reliability, patients were asked to fill in a second copy of the scale within 5–7 days. Thirty-nine patients compiled a further copy of questionnaire during the follow up visit (after 133 days on average) which allowed us a longitudinal assessment of the scale.
Results
We found a good external construct validity between ROADS and either ALSFRS-R (correlation coefficient = 0.85) or ALSAQ-40 (correlation coefficient = − 0.84). Test–retest reliability was excellent with a concordance-correlation coefficient of 0.93. Yet, we observed a significant correlation between changes over time of the ROADS normalised sum score (− 2.18 point loss per month) and those of both the ALSFRS-R (positive correlation; Rho = 0.64, p ≤ 0.0001) or the ALSAQ-40 (negative correlation; Rho = − 0.60, p = 0.014).
Conclusions
The Italian version of ROADS proved to be a reliable marker to monitor overall disability in ALS patients. Further studies are necessary to assess its longitudinal performance.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
The Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS-R) is a widespread test used to evaluate progression of disability in ALS patients. It is commonly adopted in clinical trials as a primary outcome measure [1, 2], since it offers a reasonable correlation with other less accessible endpoints (survival, electrophysiologic measures, forced vital capacity, etc.) [3]. Nevertheless, the ALSFRS-R has also a series of weaknesses which hind its clinical and research utility, for instance the lack of responsiveness in a brief observation period or in advanced stages of the disease, and the presence of items that show improvement due to change in symptoms management [4]. Other important key points are the lack of linearity and unidimensionality [5,6,7].
Rasch-built scales are linearly weighted, meaning that 1-point change is a measurable unit across the scale, and unidimensional as all questions are measuring the same domain. The American Medical Association created and validated a Rasch-Built Overall ALS Disability Scale (ROADS), for use as a clinical outcome measure in patients with ALS [8]. A recent study assessed an Italian version of ROADS, pointing to the reliability of its administration to patients versus their respective caregivers and the correlation to the corresponding ALSFRS-R [9]. The goal of this study is to further validate the Italian version of the scale according to a test–retest methodology in a wider population of patients and also to examine its longitudinal performance over time.
Methods
From July 2020 to March 2021, we consecutively proposed the compilation of the Italian Version of the ROADS to all patients with a diagnosis of ALS [10] referring to the Motor Neuron Disease Clinic of the University of Padova and the University of Modena. Comorbidity for dementia was considered an exclusion criteria. As for the original ROADS, response options to each item of the Italian version of the scale included 0 (unable to perform), 1 (able to perform but with difficulty or help), 2 (no abnormality). Patients were instructed to answer based on how they performed tasks before symptom onset, allowing caregivers to mark responses in case of patient’s inability to writing. A blank copy of the ROADS questionnaire was provided to be similarly completed at home within 5–7 days and returned to the study team by webmail after completion (test–retest analysis). Test–retest data were stratified for some possible interfering factors including: site of recruitment (University of Padova vs University of Modena), disease duration at questionnaire compilation (4 classes were considered: < 7 months, 7–13 months, 14–30 months, > 30 months), and presence of respiratory dysfunction (3 classes based on the ALSFRS-R item 10 score: 0–1, 2–3 points or 4 points). Along with the ROADS, the following outcome measures were collected as part of the routine follow-up protocol: the ALSFRS-R and the Amyotrophic Lateral Sclerosis Assessment Questionnaire – 40 (ALSAQ-40) scores [11], the Forced vital capacity (FVC) value (% of the predicted) [12], and serum creatinine and albumin levels [13]. To assess the longitudinal performance of the ROADS, patients evaluated twice during the study period were asked to fill in another copy of the questionnaire during the second visit. The time between the two visits was recorded.
A signed informed consent was obtained from all participants. The study was approved by the local Ethical Committees.
Translations and cultural adaptation of the ROADS
The ROADS questionnaire was translated to Italian by two native Italian speakers fluent in English who have not seen the scale before. The Italian version was back-translated to English by two native English speakers fluent in Italian, who also have not seen the scale before. Adaptations were necessary to some of the 28 items of the scale, during the translation process, in consideration of cultural and language differences. Item 2, “ride in a car”, was easily misunderstandable, so we translated this item specifying “ride in a car as a passenger”. Since drinking milkshake or smoothies is not a widespread habit among Italians, item 4 was modified indicating a drink having the same consistency (changed to “drink a fruit juice”). For same reasons, item 8 was modified as “eat a cracker”. Italian translation of item 11, “eat a large meal”, raised the doubt between a sizeable meal and a full meal, so we changed it to “eat a full meal”.
Statistical analyses
External construct validity was evaluated by assessing correlations of the ROADS normalised sum score (i.e. the raw sum score of the scale transformed into a linearly weighted score reflecting any 1-point change as a measurable and consistent unit of disability across the entire scale) [8] to ALSFRS-R, ALSAQ-40, serum creatinine, albumin and forced vital capacity (FVC). Correlation analysis was performed using Spearman’s correlation coefficients for ordinal data and Pearson correlation coefficient for continuous variables. Internal consistence was evaluated using Cronbach’s alpha test for ALSFRS-R, ROADS and ALSAQ-40 scores, setting the coefficient at 0 for null hypothesis. Lin's concordance test was used to assess the ROADS test–retest reliability considering the concordance correlation coefficient, CCC, and the confidence interval (CI) was calculated by the bootstrap method which is based on 2000 resamplings. For the two questionnaires, the first filled out at baseline and the second one completed 2–7 days apart, we considered either the overall normalised sum score and each single item score. We used the Bland–Altman graph for data plotting and the Wilcoxon paired test to compare the two ROADS questionnaire results. Finally, we applied the Spearman's rank correlation coefficient to compare the difference in ROADS normalised sum score averages between the first and second visit with ALSFRS-R and ALSAQ-40 score variations, considering the difference between the two visit values divided by the time elapsed in months. Significance was set at p < 0.05.
Results
A total of 270 ALS patients accepted to fill in the ROADS questionnaire during one formal visit in hospital, although 38 missed to return the other copy of the scale. For each patient, all the other functional measures were also recorded. Characteristics of total cohort of patients are shown in Table 1S. We found a good external construct validity between ROADS and either ALSFRS-R or ALSAQ-40 (Fig. 1S), with a correlation of 0.85 (CI [0.81; 0.88], p < 0.0001) and − 0.84 (CI [− 0.88: − 0.80], p < 0.0001), respectively. Neither serum creatinine (correlation coefficient 0.44, CI [0.29; 0.57], p < 0.0001) nor albumin (correlation coefficient 0.26, CI [0.04; 0.45], p = 0.0094) showed a good correlation with the ROADS score, whereas FVC values mildly correlated (correlation coefficient 0.57, CI [0.44; 0.68], p < 0.0001). Cronbach’s alpha showed a good internal consistency for the ROADS (α = 0. 9548) as well for the ALSFRS-R (α = 0.8891) and the ALSAQ-40 (α = 0.9602) scales. Test–retest reliability, evaluated from those 232 patients who completed both ROADS copies, was excellent (CCC = 0.93; CI [0.77; 0.97]). The mean test value was 82.40 (standard deviation = 20.29) and the mean retest value 81.25 (standard deviation = 20.68), with a bias of 1.15 (standard deviation = 7.6), resulting in a p value of 0.026 with Wilcoxon paired-test. Analogous results were obtained also considering each single item. Likewise, Bland–Altman graph showed low dispersion with short agreement limits (Fig. 1) and concordance–correlation analysis for every single item showed great stability. None of the considered confounding factors showed to interfere with test–retest analysis (Figs. 2, 3 and 4S).
Thirty-nine patients, among those who were evaluated twice during the study period, filled in a further copy of the ROADS questionnaire during the follow up visit. The mean interval between their two visits was of 144 days (median 133 days). By comparing data collected at the first and the second visit (follow-up visit), we could observe a significant correlation between changes of the ROADS normalised sum score and those of both ALSFRS-R (positive correlation; Rho = 0.64; p < 0.0001) or ALSAQ-40 (negative correlation; Rho = − 0.60; p = 0.014) (Fig. 2). The ROADS normalised sum score declined by 2.18 point per month on average. Longitudinal FVC or creatinine and albumin data were insufficient for a proper statistical analysis.
Discussion
We further investigated the performance of the Italian ROADS version in a broader sample of ALS patients and considered the level of response agreement administering the scale to each patient in two different times. In full accordance with the previous Italian study [9], we confirmed both an excellent internal and external (by comparison with the most widely used ALSFRS-R) consistency of our scale version. Although the intra-patient test–retest analysis returned an excellent concordance coefficients for each item responses including those related to the most challenging actions, an up to 16-point variability of the scale score was observed in some patient. This emphasizes the need to be accurate in explaining to patients how to exactly answer to the questionnaire. Our findings resulted unchanged after splitting cases into the two cohorts of patients, i.e. Padova and Modena, involved in the study and neither disease duration or respiratory functioning showed to affect consistency of answers.
Similarly, the setting where the questionnaire was compiled, i.e. hospital or home, appeared to have no bearing on the type of response. This is quite interesting as it implies that ROADS could be reliably used also for remote patient monitoring and thus it may represent a strategic tool for the ordinary clinical practice or in the context of clinical trials.
To keep the same conditions as the original version of the questionnaire, we had to modify the translation of some questions. Nevertheless, translation changes did not affect items functioning. In any case, we agree with the Chinese team [14] that a vignette representing the function explored by each question could help patients in completing/fulfilling the scale.
The novelty of our study is the longitudinal assessment of the ROADS which pointed to a stable correlation of the scale with progression of disability measured by the ALSFRS-R and quality of life according to the ALSAQ-40. It would have been interesting to compare the sensitivity as progression measure of the ROADS with ALSFRS-R being both functional scales; however, the small number of patients evaluated twice along with the short average time elapsed between the first and the second visit prevented us from drawing definite conclusions.
Finally, the Italian version of ROADS proved to be a reliable marker to monitor overall disability in ALS patients. Notwithstanding, further longitudinal studies are necessary to establish the ability of the scale to intercept disease changes over time even compared to measures already in use.
Change history
22 February 2023
A Correction to this paper has been published: https://doi.org/10.1007/s00415-023-11620-6
References
Mora JS, Genge A, Chio A et al (2020) Masitinib as an add-on therapy to riluzole in patients with amyotrophic lateral sclerosis: a randomized clinical trial. Amyotroph Lateral Scler Frontotemporal Degener 21(1–2):5–14. https://doi.org/10.1080/21678421.2019.1632346
Shibuya K, Misawa S, Kimura H et al (2015) A single blind randomized controlled clinical trial of mexiletine in amyotrophic lateral sclerosis: efficacy and safety of sodium channel blocker phase II trial. Amyotroph Lateral Scler Frontotemporal Degener 16(5–6):353–358. https://doi.org/10.3109/21678421.2015.1038277
Shefner JM, Liu D, Leitner ML et al (2016) Quantitative strength testing in ALS clinical trials. Neurology 87(6):617–624. https://doi.org/10.1212/WNL.0000000000002941
Jackson CE, Gronseth G, Rosenfeld J et al (2009) Randomized double-blind study of botulinum toxin type B for sialorrhea in ALS patients. Muscle Nerve 39(2):137–143. https://doi.org/10.1002/mus.21213
Vanhoutte EK, Hermans MCE, Faber CG, Gorson KC, Merkies ISJ, Thonnard J-L (2015) Rasch-ionale for neurologists. J Peripher Nerv Syst 20(3):260–268. https://doi.org/10.1111/jns.12122
Franchignoni F, Mandrioli J, Giordano A, Ferro S (2015) A further Rasch study confirms that ALSFRS-R does not conform to fundamental measurement requirements. Amyotroph Lateral Scler Frontotemporal Degener 16(5–6):331–337. https://doi.org/10.3109/21678421.2015.1026829
Franchignoni F, Mora G, Giordano A, Volanti P, Chiò A (2013) Evidence of multidimensionality in the ALSFRS-R Scale: a critical appraisal on its measurement properties using Rasch analysis. J Neurol Neurosurg Psychiatry 84(12):1340–1345. https://doi.org/10.1136/jnnp-2012-304701
Fournier CN, Bedlack R, Quinn C et al (2020) Development and validation of the rasch-built overall amyotrophic lateral sclerosis disability scale (ROADS). JAMA Neurol 77(4):480–488. https://doi.org/10.1001/jamaneurol.2019.4490
Manera U, Solero L, Fournier CN et al (2021) Validation of the Italian version of the Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS) administered to patients and their caregivers. Amyotroph Lateral Scler Frontotemporal Degener. https://doi.org/10.1080/21678421.2021.2013892
Brooks BR, Miller RG, Swash M, Munsat TL (2000) El Escorial revisited: revised criteria for the diagnosis of amyotrophic lateral sclerosis. Amyotroph lateral Scler other Mot neuron Disord Off Publ World Fed Neurol Res Gr Mot Neuron Dis 1(5):293–299. https://doi.org/10.1080/146608200300079536
Palmieri A, Sorarù G, Lombardi L et al (2010) Quality of life and motor impairment in ALS: Italian validation of ALSAQ. Neurol Res 32(1):32–40. https://doi.org/10.1179/174313209X385734
Czaplinski A, Yen AA, Appel SH (2006) Forced vital capacity (FVC) as an indicator of survival and disease progression in an ALS clinic population. J Neurol Neurosurg Psychiatry 77(3):390–392. https://doi.org/10.1136/jnnp.2005.072660
Chiò A, Calvo A, Bovio G et al (2014) Amyotrophic lateral sclerosis outcome measures and the role of albumin and creatinine: a population-based study. JAMA Neurol 71(9):1134–1142. https://doi.org/10.1001/jamaneurol.2014.1129
Sun C, Fournier CN, Ye S, Zhang N, Ma Y, Fan D (2021) Chinese validation of the rasch-built overall amyotrophic lateral sclerosis disability scale. Eur J Neurol 28(6):1876–1883. https://doi.org/10.1111/ene.14811
Acknowledgements
EP, LB, GS are members of the European Reference Network for Neuromuscular Diseases (ERN EURO-NMD). We would like to thank Christina Nicole Fournier MD for the kind contribution to the translation process. The authors are also grateful to Umberto Manera for his help and to ASLA Onlus.
Funding
Open access funding provided by Università degli Studi di Padova within the CRUI-CARE Agreement.
Author information
Authors and Affiliations
Contributions
GS and JM designed the research; AF, FC, LB, GG and IM performed evaluations; AF and DS conducted statistical analysis; AF, AF, DS and LB performed data analysis.
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare no competing interests.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Fortuna, A., Sabbatini, D., Frigo, A. et al. Italian version of the Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS): validation and longitudinal performance. J Neurol 270, 1452–1456 (2023). https://doi.org/10.1007/s00415-022-11483-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00415-022-11483-3