Skip to main content

Psychometric Properties of the SymptoMScreen Questionnaire in a Mild Disability Population of Patients with Relapsing–Remitting Multiple Sclerosis: Quantifying the Patient’s Perspective

Abstract

Crucial elements for achieving optimal long-term outcomes in multiple sclerosis (MS) are patient confidence and effective physician–patient communication. Patient-reported instruments may provide the means to fill the gap in currently available clinician-rated measures. The SymptoMScreen (SMSS) is a brief self-assessment tool for measuring symptom severity in 12 neurologic domains commonly affected by MS. We conducted a non-interventional study to assess the dimensional structure and item characteristics of the SMSS. A total of 218 patients with relapsing–remitting MS and mild disability (median Expanded Disability Status Scale score 2.0) were studied. Symptom severity was low (SMSS score 13.5, interquartile range 4.2–27), fatigue being the domain with the highest impact. A non-parametric item response theory, i.e., Mokken analysis, found that the SMSS is a robust one-dimensional scale (overall scalability index H  0.60) with high reliability (Cronbach’s alpha  0.94). The confirmatory factor analysis model confirmed the unidimensional structure (comparative fit index 1.0, root-mean-square error of approximation  0.001). Samejima’s model fitted well an unconstrained model with different item difficulties. The SMSS shows appropriate psychometric characteristics and may constitute a valuable and easy-to-implement addition to measure the symptom severity in clinical practice.

FormalPara Key Summary Points
Patients’ perspectives may differ from healthcare professionals’ priorities
Understanding the impact of multiple sclerosis (MS) symptoms from the patients’ perspective may facilitate shared decision-making
The SymptoMScreen is a self-rated questionnaire to assess symptom severity in different key neurologic domains commonly affected by MS. In addition to its good psychometric properties, it is a rapid and reliable tool to administer in clinical practice

Introduction

Multiple sclerosis (MS) is a chronic inflammatory demyelinating disease of the central nervous system that affects different functional systems and is associated with progressive disability [1]. The last 10 years has witnessed a dramatic increase in the diversity of available therapies for the management of MS. In this context, patients, who are commonly less informed than heathcare professionals on such aspects as treatment efficacy and the profiles of side effects, have a limited number of tools available on which to base their decision-making regarding therapy [2,3,4].

The assessment of patient perspectives can provide invaluable clinical information that may otherwise be lost when clinical evaluation is the primary source of information [3, 4]. Patient-reported outcomes are designed to capture patient perceptions of their health condition, functional level, and quality of life. However, they have been inconsistently employed in the assessment of patients with MS [5, 6].

The SymptoMScreen questionnaire (SMSS) is a self-assessment tool for measuring symptom severity in 12 critical neurologic domains commonly affected by MS: mobility, hand function/dexterity, spasticity and stiffness, pain, sensory symptoms, bladder control, fatigue, vision, dizziness, cognition, depression, and anxiety [7]. In the few studies on the SMSS conducted to date, the SMSS total score was found to correlate with clinical disability, patient-perceived physical and psychological impact, and work productivity (as assessed by the Expanded Disability Status Scale [EDSS], 29-item Multiple Sclerosis Impact Scale, and Working Productivity and Impairment Questionnaire, respectively) [7,8,9]. The SMSS may encourage patients to communicate symptom severity in multiple overt and less tangible key standardized domains. In addition, this instrument can help clinicians focus the discussion of symptom management during the visit, track severity, and serve as basis for making specific referrals [7]. The SMSS takes less than 5 min to complete and may constitute a useful tool to quantify symptoms in clinical practice [7]. However, the whole spectrum of its psychometric properties has not yet been studied and reported.

The aim of this study was to assess the dimensional structure and item characteristics of the SMSS in the management of MS.

Methods

A non-interventional, cross-sectional study was conducted in 17 MS units throughout Spain (PERCEPTIONS-MS Study). Key eligibility criteria included an age of at least 18 years, a diagnosis of relapsing–remitting MS according to the 2010 revised McDonald criteria [10], and an EDSS score of 0–5.0 [11]. Written informed consent was obtained from all subjects. The study was approved by the institutional review board of the Hospital Universitari Clínic (Barcelona, Spain) and conformed with the Helsinki Declaration of 1964, as revised in 2013, concerning human and animal rights, as well as with Springer’s policy concerning informed consent.

The SMSS consists of 12 items which are assessed on a seven-point Likert scale that ranges from 0 (not at all affected) to 6 (total limitation) [7]. The total score ranges from 0 to 72, with higher scores indicating more severe symptom endorsement.

Statistical Analysis

The distribution of the SMSS scores was evaluated across different levels of MS severity, including disease duration, total number of relapses, EDSS score, and number of disease-modifying therapies. The early-stage MS definition was used to establish the disease duration cutoff (≤ 3 years from first documented clinical attack) [8]. We evaluated mean differences and linear trends using one-way analysis of variance and linear regression.

A non-parametric item response theory (IRT) procedure, the Mokken scale analysis, and a confirmatory factor analysis (CFA) were performed to assess the underlying dimensions of the SMSS. Each of the 12 items was required to have a scalability coefficient (Hi) of ≥ 0.30 and an overall scale scalability index (H) of ≥ 0.30 [12, 13]. A parametric Samejima’s Grade Response Model (GRM) was also conducted to further assess the information and discrimination of SMSS items [14]. We considered a comparative fit index (CFI) value > 0.95 as an acceptable model fit and root-mean-square error of approximation (RMSEA) values of < 0.08 and < 0.05 as adequate and good model fit, respectively. We performed all analyses with R version 3.6.0 (https://cran.r-project.org/bin/windows/base/old/3.6.0/) using the libraries mokken, lavaan, and ltm.

Results

A total of 218 patients were enrolled in the study and included in the analysis (mean age 39.1 ± 9.5 years, 70.6% female). Symptom severity was low, and fatigue was the domain with the highest score (Table 1).

Table 1 Clinical characteristics of patients enrolled in the study

The SMSS scores showed a significant linear trend relationship with EDSS severity (p value < 0.001), with means ranging from 7.5 (lower EDSS category) to 28.9 (EDSS value 4), and total number of relapses (≤ 2 vs. ≥ 3; p = 0.02). No significant relationships were found between SMSS scores and disease duration (≤ 3 vs. > 4 years; p = 0.07) and between SMSS scores and number of MS treatments (1, 2, ≥ 3; p = 0.08), but the average SMSS scores (± standard deviation) did show a clear linear trend for disease duration (12.8 ± 11.2 and 16.6 ± 14, respectively) and for total number of treatments (13 ± 13.8, 15.6 ± 12.8, and 18.4 ± 13.3, respectively).

Internal Reliability

The SMSS showed high reliability (Cronbach’s alpha 0.94, bootstrap 95% confidence interval 0.93–0.95).

Non-Parametric (Mokken) Item Response Theory

The SMSS is a robust unidimensional scale with an overall scalability index of H = 0.60, and all items showing scalability coefficients of Hi > 0.50 (Table 2).

Table 2 Psychometric characteristics of the patients as assessed according to the 12 items on the SMSS

Item Characteristics: Grade Response Model

The item characteristic curves from Samejima’s GRM showed adequate probability patterns for response categories with an item-ranking order for discrimination that also agreed with the observed order for item information (Table 2).

Factor Structure

The CFA model confirmed the unidimensional structure suggested by Mokken analysis (RMSEA 0.001, CFI  1.0) (Table 2). All item coefficients are intrinsically linked regardless of the different techniques (non-parametric IRT, CFA, parametric IRT) as would be expected when a unidimensional domain underlies all of them (Fig. 1).

Fig. 1
figure 1

Confirmatory factor analysis (CFA) model. Lower diagonal displays the correlation graphs, upper diagonal presents Pearson’s correlation estimates, main diagonal displays the distribution of parameter estimates. Discrimination and information coefficients are from Samejima’s Grade Response Model (GRM), Hi Scalability coefficient

Discussion

Fatigue, pain, heat intolerance, mobility problems, visual impairment, anxiety, depression, and cognitive impairment are commonly found to be associated with work difficulties and poor productivity in MS patients [15]. MS is a complex disease that cannot be reduced to walking capacity, and patients must be assessed in many different aspects in each clinical examination [3, 5, 6]. In addition, the perspectives and preferences of a patient may and often do differ from the priorities of healthcare professionals. Heesen et al. found that vision, cognition, and mobility were the most important bodily functions from a patient perspective in a sample of 171 subjects with relapsing–remitting MS [16]. In the present study, visual impairment was not considered to be relevant by physicians—in contrast to patients’ opinion. The results of another quality of life assessment in MS involving 380 MS patients and 135 neurologists found a disagreement related to factors impacting health-related quality of life, with physicians focusing on physical aspects while patients considered vitality, pain, and mental health also to be relevant [17]. The management of such disruptive MS-related problems should be part of patient-centered care [3].

Clinician-rated EDSS is still the most common instrument used to assess the effectiveness of clinical interventions and to monitor disease progression despite the several shortcomings of this assessment tool [18], including inadequate assessment of several critical domains, such as cognitive performance, quality of life, and energy level [18, 19]. In this context, patient-reported outcome instruments may provide the means to fill the current gap in patient-centered impairment measures.

Our study found that the SMSS is a robust unidimensional scale that shows good psychometric properties regarding internal consistency and item discrimination and information. The SMSS shows appropriate convergent validity with the EDSS and total number of relapses as well as a linear trend with other surrogate indicators of severity.

Green et al. [7] developed the SMSS to help patients to identify and communicate symptom severity in multiple functional domains relevant for MS. Using this scale, patients rate symptoms that lead to symptom-related limitations, as opposed to measures of disability like the EDSS [7, 8]. In a sample of 1865 patients with MS, the pain, ambulation, and fatigue domains of the SMSS significantly predicted patients’ perception of health [20].

This study has a major limitation. The study population included a sample of clinically stable relapsing–remitting MS patients with low physical disability. The results may thus not be generalizable to less stable subjects and other clinical subtypes (secondary or primary progressive forms). Despite this limitation, the sample of 218 subjects was managed in 17 different MS units on a national level, representing relapsing–remitting patients across the country.

Conclusion

Understanding the impact of symptoms from the patients’ perspective may facilitate shared decision-making in MS care. The SymptoMScreen is a quick, comprehensive, and reliable instrument to assess the severity of symptoms in clinical practice.

References

  1. Kobelt G, Thompson A, Berg J, et al. New insights into burden and costs of multiple sclerosis in Europe. Mult Scler. 2017;23:1123–36.

    Article  Google Scholar 

  2. Saposnik G, Montalban X. Therapeutic inertia in the new landscape of multiple sclerosis care. Front Neurol. 2018;9:174.

    Article  Google Scholar 

  3. Yeandle D, Rieckmann P, Giovannoni G, et al. Patient power revolution on multiple sclerosis: navigating the new frontier. Neurol Ther. 2018;7:179–87.

    Article  Google Scholar 

  4. Colligan E, Metzler A, Tiryaki E. Shared decision-making in multiple sclerosis. Mult Scler. 2017;23:185–90.

    Article  Google Scholar 

  5. D’Amico E, Haase R, Ziemssen T. Review: patient-reported outcomes in multiple sclerosis care. Mult Scler Relat Disord. 2019;33:61–6.

    Article  Google Scholar 

  6. Khurana V, Sharma H, Afroz N, et al. Patient-reported outcomes in multiple sclerosis: a systematic comparison of available measures. Eur J Neurol. 2017;24:1099–107.

    CAS  Article  Google Scholar 

  7. Green R, Kalina J, Ford R, Pandey K, Kister I. SymptoMScreen: a tool for rapid assessment of symptom severity in MS across multiple domains. Appl Neuropsychol Adult. 2017;24:183–9.

  8. Kister I, Hartung HP, Vermersch P, et al. Construct validity and internal consistency of patient-reported SymptoMScreen in phase IIIb trials of ocrelizumab in relapsing multiple sclerosis (ENSEMBLE and CASTING). Int J MS Care. 2019;21[Suppl 1]:75–6.

    Google Scholar 

  9. Fitzgerald KC, et al. Validation of the SymptoMScreen with performance-based or clinician-assessed outcomes. Mult Scler Relat Disord. 2019;29:86–93.

    Article  Google Scholar 

  10. Polman CH, Reingold SC, Banwell B, et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol. 2011;69:292–302.

    Article  Google Scholar 

  11. Kurtze JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. 1983;33:1444–52.

    Article  Google Scholar 

  12. Stochl J, Jones PB, Croudace TJ. Mokken scale analysis of mental health and well-being questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers. BMC Med Res Methodol. 2012;12:74.

    Article  Google Scholar 

  13. Meijer RR, Sijtsma K, Smid NG. Theoretical and empirical comparison of the Mokken and the Rasch approach to IRT. Appl Psychol Meas. 1990;14:283–98.

    Article  Google Scholar 

  14. Brown TA. Confirmatory factor analysis for applied research. New York: Guilford Press; 2006.

    Google Scholar 

  15. Raggi A, Covelli V, Schiavolin S, et al. Work-related problems in multiple sclerosis: a literature review on its associates and determinants. Disabil Rehabil. 2016;38:936–44.

    Article  Google Scholar 

  16. Heesen C, Haase R, Melzig S, et al. Perceptions on the value of bodily functions in multiple sclerosis. Acta Neurol Scand. 2018;137:356–62.

    CAS  Article  Google Scholar 

  17. Ysrraelit MC, Fiol MP, Gaitán MI, et al. Quality of life assessment in multiple sclerosis: different perception between patients and neurologists. Front Neurol. 2018;8:729.

    Article  Google Scholar 

  18. Meyer-Mook S, Feng Y-S, Maeurer M, et al. Systematic literature review and validity evaluation of the Expanded Disability Status Scale (EDSS) and the multiple sclerosis functional composite (MSFC) in patients with multiple sclerosis. BMC Neurol. 2014;14:58.

    Article  Google Scholar 

  19. Van Munster CE, Uitdehaag BM. Outcome measures in clinical trials for multiple sclerosis. CNS Drugs. 2017;31:217–36.

    Article  Google Scholar 

  20. Green R, Cutter G, Friendly M, et al. Which symptoms contribute the most to patients’ perception of health in multiple sclerosis? Mult Scler J Exp Transl Clin. 2018;3:2055217317728301. doi: https://doi.org/10.1177/2055217317728301

    Google Scholar 

Download references

Acknowledgements

We thank the participants of the study. The authors appreciate the support of Dr. Elena Hernández Martínez de Lapiscina for her contribution in the design of the study.

Funding

This study was funded by the Medical Department of Roche Farma Spain. The sponsor also funded the journal´s Rapid Service fee.

Authorship

All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.

Author Contributions

Dr Ballesteros and Dr Maurino developed the research question and co-designed the study with Dr Saposnik. All authors contributed to the analysis and interpretation of the results. All authors contributed to and approved the final manuscript.

Prior Presentation

The abstract of this paper was presented at the European Congress of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) as a poster presentation with interim findings (Poster PND110; Copenhagen, Denmark; November 2–6, 2019).

Disclosures

Gustavo Saposnik reported receiving grants and personal fees from Roche and reported being supported by the Heart and Stroke Foundation of Canada Career Award. Elena García-Arcelay and Jorge Maurino are employees of Roche Farma Spain. José Meca-Lallana, Miguel Ángel Hernández-Pérez, Ángel P Sempere, Luis Brieva, María Terzaghi and Javier Ballesteros have nothing to disclose.

Compliance with Ethics Guidelines

This study was approved by the institutional review board of the Hospital Universitari Clínic (Barcelona, Spain). Written informed consent was obtained from all subjects. The study was approved by the institutional review board of the Hospital Universitari Clínic (Barcelona, Spain) and conformed with the Helsinki Declaration of 1964, as revised in 2013, concerning human and animal rights, as well as with Springer’s policy concerning informed consent.

Data Availability

Qualified researchers may request access to individual patient level data through the clinical study data request platform (www.clinicalstudydatarequest.com). Further details on Roche’s criteria for eligible studies are available here (https://clinicalstudydatarequest.com/Study-Sponsors/Study-Sponsors-Roche.aspx). For further details on Roche’s Global Policy on the Sharing of Clinical Information and how to request access to related clinical study documents, see here (https://www.roche.com/research_and_development/who_we_are_how_we_work/clinical_trials/our_commitment_to_data_sharing.htm).

Open Access

This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jorge Maurino.

Additional information

Enhanced Digital Features

To view enhanced digital features for this article go to https://doi.org/10.6084/m9.figshare.11534172.

Rights and permissions

This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.

About this article

Verify currency and authenticity via CrossMark

Cite this article

Meca-Lallana, J., Maurino, J., Hernández-Pérez, M.Á. et al. Psychometric Properties of the SymptoMScreen Questionnaire in a Mild Disability Population of Patients with Relapsing–Remitting Multiple Sclerosis: Quantifying the Patient’s Perspective. Neurol Ther 9, 173–179 (2020). https://doi.org/10.1007/s40120-020-00176-6

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40120-020-00176-6

Keywords

  • Multiple sclerosis
  • Patient-reported outcome
  • Psychometric methods
  • Symptom severity