Introduction

The use of self-reported outcomes in subjects with cognitive dysfunction is of particular concern [1]. The main argument against using self-reported quality of life (QoL) information from patients with cognitive dysfunction is that QoL instruments were not developed for use with cognitively impaired individuals. Some authors argue that cognitively impaired individuals suffering from multiple sclerosis (MS) are unable to produce valid QoL measures [2, 3], whereas other studies have suggested that cognitive decline in these patients does not compromise the reliability/validity of self-reported health measures [4–7]. In these previous studies, the assessment of cognitive function focused on aspects of memory [5, 7] and executive functions [6].

Attention plays a role in cognitive processing and learning. Attention as a basic cognitive function represents an essential part of conscious perception and higher order cognitive functions and plays an important role in normal functioning. Attention is a broad term that is used to designate the processes that mediate the appropriate allocation of cognitive resources to relevant stimuli, such as objects, locations or moments. The limitation of this capacity may result from deficits in the automatic or controlled processing of information, as well as the incapacity to store and manipulate temporary information, the function known as working memory. In MS patients, cognitive performance is globally affected and impacts divided attention, focused attention, sustained attention, attentional flexibility, and response speed [8, 9]. Despite the high occurrence of working memory and attention disturbances in MS patients [10, 11], even during the early stages of the disease [12], no study has specifically examined the extent to which MS patients with attention dysfunction can validly self-report their QoL. Herein, we provide new evidence regarding the suitability of using self-reported QoL information in this impaired population. The aim of this study was to explore the construct validity, reliability and external validity of a MS-specific QoL instrument in MS subjects with or without attention impairment.

Methodology

This study relied on a cross-sectional design and was performed in the neurology department of a public French academic teaching hospital (Marseille, France). The inclusion criteria were as follows: a MS diagnosis according to the McDonald criteria [13], age ≥ 18 years, any subtype of MS, outpatient status, no neurological disease (other than MS), no history of severe mental illness (except depressive disorder), no dementia (Mini Mental State Examination score < 24), no history of alcohol/drug abuse, and native French-speaking. The French Ethics Committee approved the study (Comité Consultatif de Protection des Personnes dans la Recherche Biomédicale, Marseille 2, France) and patients gave their written consent to participate. Sociodemographic (age, gender, marital status, education level, and occupational activity) and clinical (MS subtype and disease duration) data were recorded for each patient. MS disability was assessed using the Expanded Disability Status Scale (EDSS).

QoL was assessed using the French version of the Multiple Sclerosis International Quality of Life questionnaire (MusiQoL) and the Short Form 36 (SF36). The MusiQoL is a well-validated MS-specific questionnaire that describes the nine dimensions and yields a global index score: activity of daily living (ADL), psychological well-being (PWB), symptoms (SPT), relationships with friends (RFr), relationships with family (RFa), relationships with health care system (RHCS), sentimental and sexual life (SSL), coping (COP), and rejection (REJ). The SF36 is a generic questionnaire [14] describing eight subscales: physical function (PF), social function (SF), role physical (RP), role emotional (RE), mental health (MH), vitality (V), bodily pain (BP), and general health (GH). Two composite scores (physical and mental, PCS-SF36 and MCS-SF36) were also calculated.

Attention performance was measured using two tests: the French version of the Paced Auditory Serial Addition Test (PASAT) [15] and the attention/concentration subscale of the French version of the Wechsler Memory Scale (attention_WMS) [16]. The attention_WMS score was calculated using 3 subtests (mental control, digit span, and visual memory). By comparing with French normative values according to age group [16], a patient was considered cognitively impaired if their score was more than 1 standard deviation (SD) below the normative value. The PASAT score was generated using the PASAT-3 s version of the test (number of correct responses). By comparing with French normative values according to sex, age, and educational level [17], a patient was considered to be cognitively impaired if their PASAT score was more than 1 SD below the normative value.

Statistical analysis

Statistical analyses were performed on the populations defined above using the same procedure reported in the initial validation publication (reference population) [18]. For each group, psychometric properties were compared to those reported from the reference population. The structures of the MusiQoL were explored in the non-impaired and impaired populations using principal component factor analyses with varimax rotation [19, 20] to determine how these structures matched with the initial structure of the MusiQoL. For each population, the proportion of factors identified from the initial nine-factor structure of MusiQoL and the proportion of items projected to their initial dimension were retrieved.

The multidimensional structure (construct validity) was verified using the multi-trait/multi-item analysis program [21]. Internal structural validity was assessed by calculating item-dimension correlations. Item internal consistency (IIC) was calculated by correlating each item with its scale, and item discriminant validity (IDV) was assessed by determining the extent to which items correlated with the dimension they were hypothesized to represent compared to correlations with other dimensions. Floor and ceiling effects were determined to assess the distribution of the responses. For each dimension, internal consistency reliability was evaluated using Cronbach’s alpha coefficient [22].

The unidimensionality of each dimension was calculated by computing item goodness-of-fit statistics (INFIT) issued from Rasch analyses [23]. INFIT values ranging from 0.7 to 1.2 ensure that all scale items tend to measure the same concept.

To assess external validity, Spearman’s correlation coefficients were used to determine relationships between the MusiQoL and SF36 dimensions in each group, and the associations between MusiQoL dimension scores and sociodemographic and clinical features were reported as in the validation study. For qualitative variables, the mean dimension scores of the MusiQoL were compared across patient groups (e.g., gender, educational level, marital status, and occupational status) using one-way analysis of variance. Quantitative variables (e.g., age, EDSS score, and MS duration) were analyzed using Spearman’s correlation coefficients. Acceptability was assessed by calculating the percentage of missing data per dimension. Data analyses were performed using SPSS 11.0, MAP-R, LISREL and WINSTEP software.

To quantify how each of the 4 structures matched with the initial structure (reference structure), suitability indices were calculated in accordance with a previous study [7]. Decision rules were established by experts in QoL and used to define satisfactory properties according to appropriate standards [19, 20]. The means of different proportions were calculated to produce the suitability index of the ‘construct validity’ and the suitability index of the ‘external validity’.

Results

A total of 124 patients participated in the study. The mean age was 45 years (SD 11), 71 (57,3%) of the patients were women, and 58 (47,2%) had more than 12 years of education. MS subtypes included 61 (49,2%) secondary progressive, 36 (29,0%) relapsing-remitting, 20 (16,1%) primary progressive, and 7 (5,6%) clinically isolated syndromes. The EDSS median was 4.75 (minimum 1.00, maximum 8.00), and the median disease duration was 9.86 years (minimum 0, maximum 31). After comparison to the French normative values, 75 non-impaired and 39 impaired individuals were identified based on the attention_WMS, and 32 non-impaired and 73 impaired individuals were identified based on the PASAT.

Construct validity

The 9-factor structure of the MusiQoL was clearly detected in the PASAT-impaired population, and 8 factors were detected in the attention_WMS-impaired population (the REJ dimension was not detected). The percentage of items well-projected to their initial dimension was higher in the impaired populations (range: 80,6 to 100%) compared to the non-impaired populations (Additional files 1 and 2). The proportion of dimensions with IIC and the proportion of dimensions with IDV that were greater than 0.2 from the reference dimension were moderately satisfactory. In contrast, the correlation of each item with its associated dimension was higher than with the others in 6 dimensions in the PASAT-impaired population and 7 dimensions in the attention_WMS-impaired populations. Cronbach’s alpha coefficients were satisfactory in at least 6 of 9 dimensions in the two impaired populations. The INFIT statistics were satisfactory in the attention_WMS-impaired population, but less satisfactory for the PASAT-impaired group. The proportions of dimensions with floor and ceiling effects less than 10% from the reference are described in Table 1. Additional details are provided in Table 2. The suitability indices of construct validity in the impaired populations were populations were detailed in Table 1.

Table 1 Suitability indices of construct validity and external validity according to the attention status
Full size table
Table 2 Internal structural validity/reliability/unidimensionality according to the cognitive status
Full size table

External validity

Eight dimensions satisfied the 3 conditions defining the suitability of relationships between MusiQoL and SF36 scores, both for the PASAT- and attention_WMS-impaired populations (Tables 3 and 4). The suitability indices of age, disease duration, EDSS, gender, and marital status were satisfactory. Associations between the MusiQoL scores and educational level and occupational status were less satisfactory (Additional files 3 and 4). In summary, the suitability indices of external validity in the non-impaired populations were 83% (Table 1).

Table 3 Correlations between MusiQoL and SF36 scores according to the cognitive status based on Wechsler Memory Scale
Full size table
Table 4 Correlations between MusiQoL and SF36 scores according to the cognitive status based on PASAT
Full size table

Discussion/conclusion

As reported in previous studies [4, 6, 7, 24], our results support the conclusion that cognitively impaired MS patients, as defined by attention dysfunction, reliably and consistently answer the MusiQoL questionnaire. The suitability indices of the impaired populations were close to those of the non-impaired populations, and they could be considered completely acceptable considering the small sample size of some of the defined populations. The assessment of QoL in MS patients could be more widely used without concern over the adequacy of this approach for cognitively impaired patients.

Some strengths and limitations of the current study should be considered:

i) Considering only one type of dysfunction may not have accurately reflected global cognitive functioning. It has been well documented that it is unusual to observe one deficit in isolation in clinical practice. However, there is little consensus on the definition of ‘global’ cognitive dysfunction [25]. Despite this limitation, the findings of the present study support the relevance of self-reported quality of life assessments for patients with cognitive disorders. Future studies should provide similar explorations in MS populations with cognitive dysfunction defined with other definitions integrating combination of different composites.

ii) We did not consider factors previously associated to cognitive performance, such as depression [26], fatigue [27], and MS medications [28]. However, the aim of this study was to provide evidence supporting the conclusion that cognitively impaired MS patients reliably answer a self-reported QoL questionnaire regardless of the presence or absence of other factors that could have influenced their performance.

iii) The suitability indices used to define satisfactory properties relied on debatable decision rules and non-standardized calculation methods. Analyses of sensitivity should be performed.

iv) Replication of these findings in larger groups of patients is required.

v) Similar studies should be performed in other populations suffering from cognitive impairment, such as severe mental illness and elderly individuals, to determine whether these findings are specific to patients with MS.