FormalPara Key Summary Points

Measuring outcomes in adolescent and adult patients with SMA is challenging, and currently there is no consensus on how disease progression should be evaluated in them.

The development of new tools that are easy-to-use, psychometrically valid, reliable and sensitive enough to detect small changes in all patients is a priority in these patients for both clinical practice and research purposes.

In this study, a new clinical outcome measure meeting these criteria has been developed and a validation plan has been designed.

Introduction

Spinal muscular atrophy (SMA) is an autosomal-recessive neurodegenerative disorder characterized by lower motor neuron loss in the spinal cord and lower brainstem, leading to progressive proximal muscle weakness and atrophy. The most common cause of SMA is a biallelic mutation in the survival motor neuron 1 (SMN1) gene, located on chromosome 5q13 [1, 2]. Even though SMA is a rare disease, before the discovery of disease-modifying treatments (DMT), it represented the most frequent genetic cause of infant mortality. However, with the advances in supportive care and, more recently, with the approval of DMT, patients survive more frequently into adulthood. It is estimated that adolescents and adults now comprise up to two thirds of all patients with SMA (unpublished data of our population-based cohort) [3], and their prevalence will continue to increase. Despite representing such a large proportion, there is limited knowledge of the natural history, clinical management and therapeutic interventions in adults [3].

Since the approval of DMT for patients with SMA of all disease stages and ages [4,5,6,7,8], measuring outcomes in adult patients has become a priority [9, 10].

Adult patients are usually assessed with motor scales that have been designed for and validated in children [3]. These have shown floor and ceiling effects in patients with the most severe and mildest symptoms, respectively [11], and show limited sensitivity to detect clinically meaningful individual changes in adult patients with SMA [12]. Adult patients with SMA show great heterogeneity in age and severity, including a wide range of motor (walkers, sitters and non-sitters) and non-motor symptoms, as a result of impairments in muscle strength and respiratory and bulbar function. They also show different effects of scoliosis, contractures, hip dislocations and nutritional problems which may lead to chronic pain, osteoporosis and fractures [13, 14]. Thus, measuring outcomes in adult patients with SMA is challenging, and currently there is no consensus on how disease progression should be evaluated in them [3].

Several combinations of strength measurements, motor function rating scales, multidimensional bedside functional scales and patient-reported outcomes (PROs) have been proposed [15,16,17]. This usually results in time-consuming (around 90 min) protocols, using different scales according to the baseline function of the patient [16, 17].

Thus, the development of new tools is a priority in adult patients with SMA [16, 18]. Such tools should ideally assess both motor and non-motor symptoms to cover the whole range of disability of patients with SMA, while remaining easy and fast to administer as well as psychometrically valid, reliable and sensitive enough to detect small changes in all patients.

The objective of the present study is the development of a new clinical outcome measure that meets these criteria and can be used in adolescent and adult patients with SMA, both in clinical practice and research. Moreover, this tool should be customizable, including one questionnaire that can be combined with other outcome measures frequently assessed in clinical practice.

Methods

Development of the New Disease Measurement: Toolkit

PR and JFVC reviewed several clinimetric scales and PROs used in adult patients with motor neuron disease such as the Rasch Overall ALS Disability Scale (ROADS), Egen Klassifikation Scale Version 2 (EK2), Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS-R), SMA Functional Rating Scale (SMAFRS), SMA-tool and SMA independence scale (SMAIS), from both a clinical and a psychometric point of view. The researchers agreed on the need to assess five domains relevant for adult patients with SMA: bulbar function (communication and feeding), breathing, gross and fine motor function, and fatigability. Finally, the researchers decided that the ROADS [19] could be a good starting point, although motor items had to be adapted to patients with SMA and new items (from other scales or newly developed) should be included to evaluate non-motor domains. Moreover, the questionnaire is not intended to be a PRO but a clinimetric scale, where the clinician fulfils the items following a structured interview with the patient, and the questions were adapted accordingly.

Based on this previous work, an expert panel (two neurologists, one rehabilitation physician, one medical expert from FundAME—the main patient association, one methodological expert in questionnaire development, and one medical expert from Roche) selected a list of 51 items. These items were also reviewed by one walker, one sitter and one non-sitter patient, who introduced some modifications and added two more items.

Study Design

An observational prospective longitudinal study will be conducted to build the final clinical outcome measure or “toolkit” (with both global score and domain scores) and to assess its usability, reliability, validity and sensitivity to change.

This study will require primary data collection (items included in the toolkit along with patient data) every 6 months. Face-to-face visits will be performed at baseline and 12 and 24 months, and will include the complete toolkit made up of the 53 Q&A items together with the quantitative clinical measures. The Motor Function Measure-32 (MFM-32) [21] or the Expanded Hammersmith Functional Motor Scale (HFMSE) [22], as per routine clinical practice, and the Revised Upper Limb Module (RULM) [23] will also be recorded in these visits. In addition, sociodemographic, genetic and clinical information for patients will be collected at the first visit, along with the Patient-Ranked Order of Function (PROOF) questionnaire [20]. Changes in the active drug treatment (if any) and the change at 12 and 24 months from clinical global impressions (CGI) and patient global impressions of improvement (PGI-I) point of view, will be collected in these visits. Caregivers will answer the SMA Independence Scale (SMAIS) [24]. Telematic visits (phone or video call) will be performed at 6 and 18 months and will include only the 53 items of the toolkit. Thus, the study includes three face-to-face visits and two telematic visits along 2 years of follow-up. Additionally, a subgroup of patients from two sites will answer the list of 53 items by telematic administration at 2–3 weeks from baseline, to assess the test–retest reliability of the telematic administration.

Data will be collected directly in an electronic Case Report Form (e-CRF), by using a tablet or touch panel, except for the SMAIS Upper Limb Module (SMAIS-ULM) questionnaire, which will be completed by caregivers in paper.

A summary of the study procedures in each visit can be found in Table 1.

Table 1 Summary of the study procedures in each visit
Full size table

Population

In this study, patients with a confirmed diagnosis of 5q autosomal recessive SMA (biallelic mutation of the SMN1 gene) aged 16 years and above, who sign informed consent, will be consecutively included in five referral centres in Spain (Hospital la Fe, Hospital Bellvitge, Hospital Vall d’Hebrón, Hospital La Paz, Hospital Virgen del Rocío). In patients between 16 and 17 years of age, both the assent of the patients themselves and the specific informed consent of their parents or caregivers will be obtained. Subjects excluded will be those with any medical or psychological condition that, in the investigator’s opinion, might compromise the ability of the patient to give informed consent or to understand and answer the questionnaires.

Sample Size and Statistical Plan

A sample of a minimum of 80 and a maximum of 120 adult patients with SMA is the target enrolment, considering the heterogeneity, prevalence of the disease and number of patients being followed up in participating centres. Although the Rasch model, which will be used in this study, requires large samples to obtain robust item parameter estimates, samples of n ≥ 100 have been considered large enough [25]. Moreover, the patient-reported version of the SMAIS-ULM has been recently validated in a sample of less than 100 patients [24].

With the results of the baseline visit, an interim statistical analysis will be performed to assess psychometric properties of the toolkit by applying Rasch analysis, based on item-response theory, and statistics of classical test theory: (a) goodness of fit (Rasch analysis for global score of the toolkit), based on four different indicators, namely ordering of item response options, ordering of item thresholds, two statistical indicators (fit residual; χ2) and one graphical indicator (item characteristic curve); (b) dependency (Rasch analysis for global score of the toolkit), assessed by examining residual correlations; (c) reliability (Rasch analysis for global score of the toolkit), assessed using the person separation index (PSI), which is comparable to Cronbach’s alpha; (d) stability (Rasch analysis for global score of the toolkit), assessed by analysing the differential item functioning in patients with SMA by SMA type, age and other clinical variables; and (e) construct validity. The convergent validity will be assessed by correlating the toolkit dimension scores and the global score with the scores of HFMSE or MFM-32, RULM and SMAIS at different follow-up times (baseline and 12 months). For the divergent validity, the ability to differentiate between patients’ subgroups (walkers, sitters and non-sitters) will be tested; and (f) test–retest reliability, by means of the intraclass coefficient.

After the 12-month visit, the following parameters will be calculated: (a) sensitivity to change and minimal clinically important difference (MCID) will be assessed by comparing the global score and the dimension scores of the toolkit at baseline and 12-month visits, in the group of patients who change clinical status according to doctor and patient opinion (CGI and PGI); and (b) minimal detectable change (MDC) by using the standard error of measurement. The analysis of sensitivity to change and MDC will be carried out again at the end of follow-up (24 months ± 30 days), as it is expected that in most adolescent and adult patients, clinically significant changes will happen after 2 years of follow-up.

Compliance with Ethics Guidelines

The SMA life study will be conducted in accordance with the Good Clinical Practice Guidelines of the International Conference on Harmonisation and with the ethical principles of the Declaration of Helsinki. The study has been approved by the institutional review board of the Hospital Universitari de Bellvitge (Barcelona, Spain; reference code: PR264/20). Written informed consent will be obtained from all subjects.

Results

The final tool comprises one questionnaire and four quantitative measurements, considered to be clinically relevant. The questionnaire includes 53 items grouped into seven domains (Table 2):

  1. 1.

    Bulbar function (10 items)

  2. 2.

    Respiratory function (4 items)

  3. 3.

    Axial function (5 items)

  4. 4.

    Upper limb function (14 items)

  5. 5.

    Lower limb function (13 items)

  6. 6.

    Fatigability (5 items)

  7. 7.

    Other symptoms (2 items)

Table 2 Items of the “toolkit” (translated into English)
Full size table

The quantitative measurements included are the pinch strength measured with Myopinch®, the 6-minute walk test (6MWT), the percentage-predicted forced vital capacity (FVC%) and the body mass index (BMI).

The best items of the questionnaire will be selected, according to Rasch and exploratory factor analysis, for the final version of the scale. If a decision is made to delete some items, the list will be adapted accordingly for the follow-up visits. Final items will be combined with other quantitative measurements, through a second Rasch analysis, to obtain both a global score and individual scores for each domain. The global score will provide different weights for each answer of each item or clinical measure selected to be part of the “toolkit”, according to the Rasch analysis performed. The dimension score will provide the same weight for each answer of each final item selected to be part of each area of the “toolkit”, according to the exploratory factor analysis. Moreover, patients will be asked for the order of importance of each area following the PROOF questionnaire [20], so that the most important areas for each patient are considered when interpreting the global score, and the clarity, relevance and acceptability of the questionnaire.

Discussion

The heterogeneity of SMA, both in age and in disability, precludes the use of the same outcome measure in all patients. Outcome measures in paediatric patients mostly evaluate the achievement of developmental milestones (the ability to sit, walk, crawl, eat, etc.), which have been well defined among these patients. However, in the adult population, these measures have shown several pitfalls that limit their ability to detect changes [11]. Moreover, they do not directly capture the impact of disability in the daily life of Patients with SMA.

Up to now, most (if not all) SMA outcome measures have been designed for paediatric patients, because the clinical trials were mostly focused on this population, despite the fact that the highest prevalence of SMA is in adolescents and adults.

With the approval of DMT for SMA, there is an urgent need for new outcome measures for adolescent and adult patients, where current measures do not capture the huge heterogeneity of the disease, show psychometric pitfalls and lack sensitivity to detect changes (see below). To overcome these limitations, adult patients are usually assessed with a combination of outcome measures [16, 17, 26], including motor and functional scales, strength measurement, bulbar function, respiratory function (pulmonary function tests) and PROs, among others. However, that combination differs in each centre, hindering the comparison of the results. Moreover, the proposed protocols usually require more than 90 min to be applied and are difficult to implement in routine clinical practice. In this study, we aimed to integrate the information provided by some assessments routinely performed in the clinical practice, together with the patient’s and clinician’s perception of change, to build a clinimetric tool that captures the impact of the disease in the daily life of adolescents and adult patients, while providing the physician with an overview of the disease progression of response to treatments. Clinimetric indexes are arbitrary ratings for the various clinical phenomena that are observed or experienced by clinicians, while incorporating patients’ perceptions. They are usually easy and fast to administer, showing excellent reliability with little training, which makes them ideal to be administered in routine clinical practice. Obviously, the structure, choice of variables and organization of output scales may greatly vary depending on their aim [27], but a unique feature of clinimetric tools is that they provide a broad global rating of the clinical phenomena by including different domains (e.g., motor and non-motor symptoms) and modes of rating (e.g., rater’s assessed items are frequently combined with patients’ assessed ones). Thus, they require the collaboration of the patient [28], but not to the extent covered by PROs, which are focused exclusively on reflecting the patients’ subjective perceptions about their disease. Importantly, due to their multidomain and multimodal design, they usually show multidimensionality and are not linearly weighted, limiting their use in research. To avoid that, clinimetric tools can be standardized to have reliability and validity and must be specifically designed for the target population to be able to detect clinically relevant changes over time [29].

Hereafter, we will review the pros and cons of the outcome measures most frequently used in adult patients with SMA and will justify the design of this new clinimetric tool.

Motor scales consist of a series of exercises that patients must carry on and are scored by a trained rater. In SMA, these scales are usually long (> 20 min) and require significant training and experience to achieve acceptable reproducibility. Moreover, non-motor symptoms and fatigability are usually not captured by them. Among the motor scales, the HFMSE [30,31,32], the MFM-32 [21] and the RULM [23] are the most widely used but have been developed and validated mostly in paediatric populations [10, 11, 15]. As a consequence, HFMSE cannot be used in non-sitter adult patients and shows a significant floor effect in weak sitters [11, 33], while RULM shows a ceiling effect in ambulant adults [11]. Moreover, these tools are little sensitive to detect subtle changes in adult patients, in whom the disease progresses slowly [11, 18]. Finally, these scales are ordinal and not linearly weighted, and there is no consensus on which change can be considered clinically meaningful [12]. One exception is the 6MWT, a quantitative scale that has been widely used in neuromuscular diseases and is easy and fast (6 min) to administer. However, it can only be used in ambulant patients. For our new tool, we decided to include the 6MWT in ambulant patients, given that it is quantitatively scored and may also inform about fatigability.

The strength measurement has been proposed as an alternative to motor scales in SMA [33]. However, the Medical Research Council scale of graded muscle strength has low sensitivity and reliability, and quantitative approaches are preferable [34]. Hand-held dynamometry was the most commonly used approach in motor neuron diseases, but it shows reliability problems, and in strong patients, it measures the strength of the rater rather than the strength of the patient. To avoid these pitfalls, novel devices such as the Myotools® and Neuromyotype (a smart keyboard) have been developed and validated in adult patients with SMA [18, 35]. For this study, we decided to measure the pinch strength with Myopinch®, since it has been shown to be sensitive to detect small changes in patients with SMA and it is commercially available [18].

Among the bedside functional scales, the most frequently used are the revised version of the ALSFRS-R, recently validated in adult patients with SMA [11], the SMAFRS [36] and the EK2 [37]. However, the ALSFRS-R was not designed for patients with SMA, and some items (such as the respiratory ones) are not well suited to them, while SMAFRS and EK2 can only be used in ambulant and non-ambulant patients, respectively.

A Rasch analysis was conducted to gain a better understanding of the psychometric properties of nine scales commonly used in SMA and found pitfalls in all of them [38]. These authors developed the SMAIS-ULM [24], a PRO that follows the Rasch method and can be assessed by both caregivers and patients. However, it only provides information about upper limb function. Other PROs such as the SMA Health Index (SMA-HI), SMA-tool and PROfuture assessing different domains have also been developed in recent years to capture the global changes perceived by the patients [33]. While incorporating patients’ perspectives is key, PROs are designed to be self-administered, and their results do not directly inform the clinician about which items are impaired or how clinically relevant this impairment is. Moreover, they usually consist of more than 100 items, limiting their use in routine clinical practice. Consequently, PROs are increasingly used in research, but still have a limited role in clinical practice, where the clinician needs to make therapeutic decisions for individual patients. In this study, we designed a questionnaire that could be administered by the clinician in routine clinical practice in less than 10 min, while assessing multiple domains of interest. To this end, we selected and adapted to patients with SMA the motor items of ROADS [19] (a PRO designed for ALS patients using the Rasch method) and added some extra items from the SMA-tool [39] and the PROfuture questionnaire, assessing fatigability and bulbar and respiratory function. The Spanish patients’ association (FundAME) participated in the process to ensure that meaningful factors at an individual patient level were included.

Other strategies for measuring changes in neuromuscular diseases include the use of wearables, such as the SV95C, which has been recently qualified by the EMA as primary endpoint in studies in ambulatory Duchenne muscular dystrophy [40]. While these devices are highly promising in a research context, up to now they are more difficult to interpret clinically by the treating physician and are not readily applicable to the clinical practice.

Along with traditional motor and functional scales, other variables are routinely acquired in the routine clinical practice of adolescent and adult patients with SMA, to assess the respiratory and nutritional status. Although the peak cough flow (PCF) may be the most sensitive index of respiratory impairment in patients with SMA [3], we selected the FVC% because it is the most widely available measure, and data on its natural history are also available [41]. For the same reasons, we selected BMI to measure the nutritional status. Thus, the final toolkit includes a multidomain questionnaire (< 10 min), together with other measures that are readily available in the clinical practice (pinch strength, 6MWT, FVC% and BMI). Remarkably, both the complete toolkit and the questionnaire will be validated in this study, and a global score as well as single-domain scores will be obtained, so that this toolkit will be customizable and adaptable to different situations in research and clinical practice.

Given that this study was performed in centres treating adult Patients with SMA, only patients older than 16 years were included. However, we think it could be eventually used in patients older than 12 years as it has been shown in SMAIS [24]. For its use in patients between 2 and 12 years, a caregiver version should be developed.

In summary, the present study will provide a new clinical outcome measure that can be easily incorporated into routine clinical practice and research. This outcome measure will be multidimensional and multimodal to capture the heterogeneity of the disease, as well as customizable, so that it can be adapted to the disability of all patients. Interestingly, a similar approach was successfully tested in a recent longitudinal study [18]. Querin et al. showed that a composite score including pinch strength measurement, SMAFRS and Motor Unit Number Index (MUNIX) readily captured disease progression. However, this tool was designed and validated only in SMA type III and IV patients, and MUNIX is not that widely available and easy to implement.

Risks and Contingency Plans

The main risk of the present study is related to its prospective design, which can lead to incomplete data from some participants who are lost to follow-up. Another risk could be the limited sample size, given that SMA is a rare disease. To overcome these limitations, five referral centres in Spain were included and the statistical sample has been calculated to accomplish the main study objectives. Another risk is that the SMA landscape is rapidly evolving, and new variables can be incorporated into routine clinical practice in the next few years. However, our toolkit is customizable and allows the separate use of the questionnaire. Thus, if for example the PCF is proved to be more useful than the FVC%, this specific variable could be easily replaced, and the tool would remain useful after a small validation study.

Study Status

This study is ongoing. The expected end date of patient recruitment is November 2023 and the end of the study is expected in November 2025.

Conclusions

This study provides the structure of a new clinimetric multidimensional outcome measure and the design for its validation. This tool will allow the clinical evaluation of adult and adolescent patients with SMA and the quantification of their response to new treatments in both clinical practice and research.