Background

The assessment of quality of life (QoL) has been considered increasingly important with regard to the evaluation of disease progression, treatment and the management of care provided to patients with chronic illness. The US Food and Drug Administration and the European Medicines Agency have provided detailed recommendations for QoL assessment [1, 2]. Despite the acknowledged need to consider QoL issues, QoL assessment remains under-utilized in clinical practice [3, 4].

QoL is commonly assessed using self-reported questionnaires. The extent to which patients with cognitive dysfunction can accurately self-report their QoL has been a crucial issue that has only been partially examined [5]. Few studies have investigated this issue in specific populations, such as in patients with multiple sclerosis (MS) [6, 7], patients with serious chronic mental illnesses [8, 9], in elderly populations [10, 11], and in patients with traumatic brain injury [12]. These studies have produced conflicting results, where some studies have demonstrated that the cognitively impaired are unable [13, 14] or able [6, 7, 9] to produce valid and reliable QoL measures. However, these studies provided restricted data regarding validity and reliability and did not report whether the structure of the questionnaire described in the impaired populations was well adapted when the QoL measure was used in these individuals. More recent studies have reported strong arguments in support of the conclusion that cognitively impaired patients can reliably and consistently respond to disease-specific questionnaires, where such studies used an interesting approach based on quantification of the relevance of the QoL assessment [15–17]. This quantification relied on suitability indices based on decision rules. These rules defined expected psychometric properties according to the appropriate standards [18, 19] and the properties reported in the reference population in the initial validation publication of the QoL questionnaire. This approach enables the determination of the relevance of QoL measures in different populations independent of the questionnaire used.

In this study, we compared the relevance of the QoL assessment between 2 populations of subjects with cognitive dysfunction to determine if the nature of the disease influenced the individual’s ability to accurately report his or her life experience. Cognitive deficits occur in a large proportion of patients with MS [20, 21] as well as in patients suffering from schizophrenia (SCZ) [22, 23]. In these 2 populations, similar neuroimaging abnormalities were reported [24, 25] and all cognitive domains, such as executive functions, memory, and attention/concentration are affected. The aim of this study was to explore the validity of disease-specific QoL questionnaires in patients with impaired cognitive function and to compare these findings between MS and SCZ patients.

Methods

Study design and participants

This study utilized a cross-sectional design. The inclusion criteria for MS patients included the following: an MS diagnosis according to the McDonald criteria [26], any subtype of MS, no neurological disease (other than MS), and no history of severe mental illness (except depression disorders). The inclusion criteria for SCZ patients included the following: age, diagnosis of SCZ according to the Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV) criteria [27], and no neurological disease. All of the patients were over 18 years old, outpatients, in a stable disease phase (no relapse during the last 3 months), and native French speakers. MS patients were recruited from the neurology department of a public French academic teaching hospital (Marseille, France) and SCZ patients were recruited from two psychiatric hospitals (Marseille and Toulon, France). The French Ethics Committee approved this study (Comité Consultatif de Protection des Personnes dans la Recherche Biomédicale, Marseille, France). Informed consent was obtained from all subjects.

Data collection

The collected data included sociodemographic information, clinical characteristics, QoL and cognitive assessments. The sociodemographic (age, gender, and education level) and clinical (disease duration) data were recorded for each patient. Disease severity was assessed using the Expanded Disability Status Scale (EDSS) [28] and the Positive and Negative Syndrome Scale (PANSS) for MS and SCZ patients [29], respectively.

QoL was assessed using the French versions of disease-specific questionnaires: the MusiQoL for MS patients and SQoL for SCZ patients. The MusiQoL is a well-validated MS-specific questionnaire [30] consisting of 31 items that describes nine dimensions and provides a global index score: activity of daily living (ADL), psychological well-being (PWB), symptoms (SPT), relationships with friends (RFr), relationships with family (RFa), relationships with the health care system (RHCS), sentimental and sexual life (SSL), coping (COP), and rejection (REJ). The SQoL is a self-administered QoL questionnaire designed for people with schizophrenia [31, 32]. It consists of 18 items that describes 8 dimensions (psychological well-being (PsW), self-esteem (SE), family relationships (RFa), relationships with friends (RFr), resilience (RE), physical well-being (PhW), autonomy (AU) and sentimental life (SL)) and provides a total score (index). For the two questionnaires, the dimension and index scores range from 0, which indicates the lowest QoL, to 100, which indicates the highest QoL.

Cognition assessment

For both patient groups, cognitive assessment included executive function, memory, and attention performances. The executive function performance was assessed using both the French versions of the Stroop color-word test [33] and the Trail Making test (TMT) [34]. Memory performance was assessed using the long-short-delay tests of the French version of Grober and Buschke [35] in MS patients, and the long-short-delay subscales of the French version of the Wechsler Memory Scale [36] in SCZ patients. Attention performance was evaluated using the attention/concentration subscale of the French version of the Wechsler Memory Scale [36]. The same trained psychologist administered the tests in a standardized manner and the same instructions were given to the subjects prior to each test. Applying French normative values according to age, gender, and educational level (except for the Wechsler Memory Scale), the patients were divided into the following categories of cognitive function: 1) executive function: non-impaired (normal Stroop test and normal TMT) and impaired (other cases) [37]; 2) memory function: non-impaired (normal short-delay memory and normal long-delay memory) and impaired (other cases) [35, 36]; 3) attention function: non-impaired (normal attention/concentration subscale) and impaired (other cases) [36]. These categorizations enabled the characterization of each patient as a no or low-impaired (no impairment or one impaired function) and highly-impaired global function (2 or 3 impaired functions).

Statistical analysis

Statistical analyses were performed on the 4 populations defined above, i.e., no/low-impaired and highly-impaired groups for the 2 diseases, using the same procedure reported in the initial validation publications (reference populations) for MusiQoL [30] and SQoL [32]. For each group, the psychometric properties were compared to those reported in the reference population as assessed in the validation study of each questionnaire [30, 32]. The structures of the MusiQoL/SQoL were examined in the no/low-impaired and highly-impaired populations using factor analysis to determine how these structures matched with the initial structure of the questionnaire using principal component factor analyses with a varimax rotation [18, 19]. For each population, the proportion of factors identified from the initial factor structure of the MusiQoL/SQoL and the proportion of items projected to their initial dimension were obtained.

The multidimensional structure (construct validity) was verified using the multi-trait/multi-item analysis program [38]. The internal structural validity was assessed by investigating item-dimension correlations. The item internal consistency (IIC) was calculated by correlating each item with its scale, and the item discriminant validity (IDV) was assessed by determining the extent to which the items correlated with the dimension that they were hypothesized to represent compared to the correlations with other dimensions. Floor and ceiling effects were determined to assess the homogeneous repartition of the response distribution. For each dimension, the internal consistency reliability was evaluated using Cronbach’s alpha coefficient [39].

The unidimensionality of each dimension was explored by computing the item goodness-of-fit statistics (INFIT), which was obtained from Rasch analyses [40]. The INFIT values ranged from 0.7 to 1.2 to ensure that all of the scale items measured the same concept.

To explore external validity, Spearman’s correlation coefficients were used to investigate the relationships between the MusiQoL and SF36 dimensions in each group, and the associations between the MusiQoL dimension scores and sociodemographic and clinical features were reported similar to the validation study. For qualitative variables, the mean dimension scores of the MusiQoL were compared across patient groups (e.g., gender, educational level, marital status, and occupational status) using one-way analysis of variance. Quantitative variables (e.g., age, EDSS score, and MS duration) were analyzed using Spearman’s correlation coefficients. Acceptability was assessed by calculating the percentage of missing data per dimension. Data analyses were performed using SPSS 11.0, MAP-R, LISREL and WINSTEP software.

To quantify the manner in which each of the 4 structures matched with the initial structure (2 reference structures), suitability indices were calculated as previously described [15]. Decision rules, which were established by experts in QoL, were used to define satisfactory properties according to appropriate standards [18, 19]. The means of the different proportions were calculated to produce a suitability index of the ‘construct validity’ and a suitability index of the ‘external validity’.

Results

One hundred and twenty-four consecutive MS patients and 113 patients with schizophrenia were enrolled in this study. The sociodemographic and clinical features are listed in Table 1. At the timepoint of performing QoL assessment, 118 (95%) MS patients were taking disease-modifying drugs and 109 (96%) patients with schizophrenia were taking atypical antipsychotics (average dose of chlorpromazine of approximately 570 mg). According to the French normative values and to the definition of global cognitive status, 50 MS patients were categorized as cognitively no/low-impaired and 49 patients were categorized as cognitively highly-impaired. In addition, 40 SCZ patients were categorized as cognitively no/low-impaired and 67 patients were categorized as cognitively highly-impaired.

Table 1 Sample characteristics
Full size table

Construct validity

In the MS sample population, seven of the 9 initial factors were identified in the no/low-impaired group (REJ and SSL dimensions were not clearly identified), and 8 initial factors were found in the highly-impaired group (COP dimension was not detected). Twenty-seven and 29 of the 31 items contributed to the initial dimension in the no/low-impaired and highly-impaired groups, respectively.

The proportion of dimensions with IIC, which was greater than 0.2 compared to the reference population, was higher in the highly-impaired population compared to the no/low-impaired population. The proportion of dimensions with IDV, which was not greater than 0.2 compared to the reference population, was moderately satisfactory in both populations. The correlation for each item with its contributive dimension (IIC) was higher compared to the other dimensions (IDV) for 6 and 7 dimensions in the no/low- and highly-impaired populations, respectively. Cronbach’s alpha coefficients were satisfactory for 6 of the 9 dimensions in the no/low-impaired group. In addition, both populations showed satisfactory INFIT statistics. The floor effects were less than 10% compared to the reference population for 8 and 4 dimensions in the no/low-impaired and highly-impaired groups, respectively. The proportion of dimensions with a ceiling effect exceeding 10% compared to the reference population was higher in the highly-impaired population compared to the no/low-impaired population. The suitability indices of the construct validity were 82% and 78% in the highly-impaired and no/low-impaired populations, respectively.

In the SCZ sample, the 8-factor structure of the SQoL was clearly detected in the no/low and highly-impaired groups. Sixteen of the 18 items contributed to the initial dimension in the no/low-impaired and highly-impaired groups, respectively. The proportion of dimensions with IIC that was greater than 0.2 from the reference population was unsatisfactory in both populations. Moreover, the correlation for each item with its contributive dimension (IIC) was higher compared to the others (IDV) for 5 dimensions in the no/low- and highly-impaired populations. Cronbach’s alpha coefficients were satisfactory for 6 of the 8 dimensions in the no/low-impaired group. Furthermore, the INFIT statistics were satisfactory for both populations. The floor effects were less than 10% compared to the reference population for 6 dimensions in both groups, and the ceiling effects were less than 10% from the reference population for 4 and 5 dimensions in the no/low- and highly-impaired populations, respectively.

The suitability indices of the construct validity were 80% in the highly-impaired population and 74% in the no/low-impaired population (Figure 1). Further details are provided in Table 2 and Table 3.

Figure 1
figure 1

Suitability indices of construct validity according to the disease. LI no/low-impaired (0 or 1 impaired composites: executive, memory, and attention functions); HI highly-impaired (2 or 3 impaired composites). MS multiple sclerosis, SCZ schizophrenia.

Full size image
Table 2 Internal structural validity, reliability, unidimensionality according to the cognitive status
Full size table
Table 3 Suitability indices of construct validity and external validity according to the cognitive status: a global approach
Full size table

External validity

In MS individuals, women generally reported lower scores compared to men excepted in the sentimental and sexual life independent of cognitive status. The suitability index was more satisfactory in the highly-impaired population compared to the no- or low-impaired population. The proportion of dimensions with an effect size of less than 0.2 compared to the reference population for educational level was moderately satisfactory in both populations. In SCZ individuals, the proportion of dimensions with an effect size of less than 0.2 compared to the reference population for gender and occupational status was more satisfactory in both the no/low and highly-impaired groups.

As expected, age and disease duration almost never correlated with the QoL dimensions in the MS and SCZ populations. Findings related to relationships with disease severity and QoL scores were close compared to those of the 2 reference populations.

Taken together, the suitability indices of external validity in the no/low- and highly-impaired populations were 80% and 87% for the MS population and 85% and 88% for the SCZ population, respectively (Figure 2). These results are summarized in Tables 3, 4, and 5.

Figure 2
figure 2

Suitability indices of external validity according to the disease. LI no/low-impaired (0 or 1 impaired composites: executive, memory, and attention functions); HI highly-impaired (2 or 3 impaired composites). MS multiple sclerosis, SCZ schizophrenia.

Full size image
Table 4 Associations between QoL dimension scores and sociodemographic characteristics according to the cognitive status
Full size table
Table 5 Correlations between QoL dimension scores and age and clinical features according to the cognitive status
Full size table

Discussion

Our results provide strong evidence supporting the relevance of self-reported quality of life assessments for patients with cognitive disorders, particularly in patients with severe cognitive dysfunction. It seems that the nature of multiple sclerosis and schizophrenia did not affect this type of assessment.

We examined these two diseases on the basis of the following points: i) the status of chronic illnesses with a high occurrence of reported cognitive deficits, even during the early disease stages [41, 42]; ii) three main composites of cognition were indiscriminately affected [42, 43]; iii) the homogeneous and extensive assessment of cognition, including tests assessing memory, attention, and executive function; iv) the availability of a disease-specific self-reported QoL questionnaire [30, 32]; and v) the surprising similarities related to changes in white matter structure or abnormalities in myelin [24, 25]. Moreover, some studies have suggested that changes in the integrity of white matter can result in impaired cognitive function in MS [44] and SCZ [45] patients.

These findings may support for the use of QoL assessment for clinicians who are still perplexed when interpreting the meaning of QoL scores for an individual with cognitive impairment. This present study suggests that cognitively impaired patients, as defined by a global cognitive dysfunction, can reliably and consistently respond to a specific QoL self-reported questionnaire. This assumption is underlined by the suitability indices found in the highly-impaired groups, i.e., 2 or 3 altered functions altered, in both MS and SCZ patients. These indices may be considered satisfactory compared to their respective reference populations. In the highly-impaired groups, factor analysis showed that the structure corresponded with the initial structure of the QoL questionnaires: 8 of the 9 dimensions were clearly identified in the MusiQoL and all the dimensions were identified in the SQoL. Although the IIC values reported in the highly-impaired population of MS individuals were similar to those identified in the reference population, the proportion of dimensions with IIC that exceeded 0.2 compared to the reference population was less satisfactory in SCZ individuals. For MS and SCZ populations, the proportion of dimensions with IDV values greater than the IIC values and the proportion of dimensions with IDV exceeding 0.2 compared to the reference population were less satisfactory, which may be explained by the very restricted definition of the decision rule. Internal consistency coefficients calculated in the highly-impaired groups were close to those of the initial reference populations. The floor and ceiling effects were slightly different compared to those reported in the initial validation publication independent of disease type. In addition, satisfactory INFIT statistics supported the unidimensionality of each of the dimensions.

Regarding external validity, highly-impaired populations showed satisfactory suitability indices. The links between QoL scores and age, severity disease score (EDSS and PANSS), and disease duration were closer to the initial reference populations independent of cognitive status and disease. However, links between QoL scores and gender and educational level were less satisfactory. In summary, the suitability indices of the highly-impaired population may be considered completely acceptable considering the small sample size of the defined populations.

Several previous studies have employed similar approaches to define cognitive dysfunction using a single composite, such as memory [7, 15], attention [16], and executive functions [17, 46]. It should be acknowledged that a single test of cognitive functioning would never be entirely appropriate to define an impaired cognitive population. One composite cannot be a perfect reflection of global cognitive function because patients suffer from several neuropsychological deficits. It would be unusual to observe one deficit in isolation [20, 47–49], and QoL measurement may be altered depending on the type of cognitive impairment [50]. Thus, it is necessary to report additional information according to other definitions of cognitive dysfunction using a combination of different composites. To the best of our knowledge, this is the first study that uses the definition of cognitive dysfunction, which integrates a combination of different composites (i.e., memory, attention, and concentration).

Several limitations and strengths of this study should be mentioned:

  1. 1.

    The representativeness of the samples should be discussed. Compared to international and European cohorts, the MS patients in this study exhibited a more severe disability profile [30, 51], and the SCZ patients had a longer illness duration [52]. Thus, an assessment of the reproducibility of our results is needed, using a larger and more diverse group of patients. However, the proportion of cognitively impaired subjects was consistent with the literature for MS [13, 20, 53] and SCZ [54, 55] domains.

  2. 2.

    One important aspect of this study concerns our definition of cognitive dysfunction because there is little consensus according to Achiron and Barak [56]. We defined cognitive impairment using tests in which the French norms have been previously published [35–37]. This eliminated the need for a control group and enabled a consensus in defining patients as non-impaired or impaired for each test. However, our definition may be questionable because of the absence of a consensus on a ‘global definition’ of a patient with global cognitive dysfunction on the basis of a combination of these tests. However, we are convinced that our findings, independent of the definition of cognitive dysfunction, can help researchers to better understand the relevance of self-reported quality of life assessments for patients with cognitive disorders.

  3. 3.

    The suitability indices used to define the satisfactory properties relied on arbitrary decision rules, each of which will be discussed. Nevertheless, this approach enabled the determination of the suitability or unsuitability of different structures using the same decision tree independent of the questionnaire and disease. Thus, future studies may be performed to test different decision trees and to discuss the implications of the subsequent results.

  4. 4.

    Factors previously associated to cognitive performance, such as depression and fatigue [57, 58], and medications [59] were not considered. However, the aim of this study was to provide evidence supporting the conclusion that cognitively impaired patients reliably answer a self-reported QoL questionnaire regardless of the presence or absence of other factors that could have influenced their performance.

Conclusion

These findings confirmed preliminary results, which suggest that cognitive decline, as defined using a global cognitive dysfunction, did not compromise the reliability or validity of self-reported health measures. This study should support the clinical relevance of QoL assessment, thereby enhancing the use of QoL measures in clinical practice for cognitively impaired patients.