Introduction

Huntington disease (HD) is a neurodegenerative disorder caused by a CAG repeat expansion in the HTT gene. It is characterized by the progressive development of motor, cognitive and behavioral alterations. Although clinical diagnosis is based on the presence of unequivocal motor abnormalities, almost all individuals with HD develop a gradual process of cognitive deterioration that runs from mild cognitive impairment, with minimal impact on functional independence, to frank dementia with a dramatic impact on functionality [1,2,3,4,5]. Moreover, relatively subtle cognitive and behavioural changes often precede the motor-based diagnosis of HD by several years [6,7,8]. However, given the lack of HD-specific approaches to the diagnosis of the various cognitive syndromes occurring in HD [9], the prevalence of the different categories of cognitive impairment at different stages of the disease remains unclear. Besides, the clinimetric properties of the cognitive instruments in use are presently underdeveloped [10].

While deficits in frontal-executive function, attention, and processing speed have been conceptualized [11, 12] as the most characteristic and representative cognitive features of the disease [13, 14], cognitive impairment in HD appears to be more complex, and several cognitive domains may be affected along the trajectory of neurodegeneration [15,16,17]. Large multicentric studies, such as the Enroll-HD Study [18], have focused on identifying reliable measures that offer sufficient sensitivity to track disease progression over time. Several cognitive measures with a strong dependence on frontal-related processes, but not devoid of a motor component (i.e.: the Symbol Digit Modality Test (SDMT) or the Stroop Word Reading Test (SWRT), have demonstrated a strong sensitivity to change [13, 19, 20]. Such measures have become essential components when building composite scores to monitor HD progression [21, 22]. Still, compared with measures for other neurological diseases, instruments currently available are insufficient to classify the degree of CI (i.e.: MCI, dementia) in the HD population. What´s more, this deficiency cannot be solved on the basis of addressing a single cognitive process or domain, such as when using the SDMT or the SWRT score. Currently, the only diagnostic approach available is that proposed by the DSM-5 and the use of cutoffs established in other diseases [23,24,25,26]. At the present time, therefore, it is difficult to select HD subjects for inclusion in studies of cognitive modifiers and to rate changes in cognition in these patients.

To further our knowledge regarding the utility of the currently instruments we took advantage of the collaborative effort of an international task force and designed a prospective multicentre study in which we administered a set of cognitive tests that addressed global cognitive function. Using a reliable gold-standard for cognitive status, we determined the prevalence of various categories of cognitive impairment in different HD groups. We then addressed and compared the discriminative properties of the UHDRS cog-score and two other global cognitive instruments that have also been used in HD (the MMSE and the PD-CRS). We determined the sensitivity to change of these assessments in the overall sample and in each cognitive group.

Participants and methods

Participants

This multicentre international HD study was coordinated by the Cognitive Phenotype Working Group (CPWG) of the European Huntington’s Disease Network (EHDN). Subjects were recruited from 10 centers in Spain, Italy, Germany, Portugal and Poland, and all were participants in the Enroll-HD study [18]. All participants were adults with no neurological disorder other than HD and free of concomitant illnesses such as history of head injury, drug/alcohol abuse, severe language difficulties, and major psychiatric illness potentially interfering with their performance.

We included 180 gene mutation carriers (CAG ≥ 39). Participants were classified as premanifest HD (PreHD) if they had a Unified Huntington’s Disease Rating Scale (UHDRS) total motor score (TMS) below 5, a diagnostic confidence level (DCL) < 3 and a Total Functional Capacity (TFC) of 13. They were classified as manifest HD (HD) if they had a UHDRS-TMS > 4 and a diagnostic confidence level (DCL) = 4, which means that motor abnormalities are unequivocal signs of HD with ≥ 99% confidence. We calculated the disease burden score assumed as a measure of lifelong exposure to mutant HTT using the formula DBS = [(CAG − 35.5) × age] [27].

Assessments

We recorded clinical and sociodemographic data regarding age, gender, education, and CAG repeat length. The severity of motor symptoms was rated using the Unified Huntington´s Disease Rating Scale Total Motor Score (UHDRS-TMS). The TFC and the UHDRS Independence Scale (IS) were administered to assess patients´ functionality [28]. Disease stage was determined based on Shoulson and Fahn criteria [29]. Therefore, patients were classified as Stage I when TFC > 10, Stage II when TFC was between 6 and 10, and Stage III when TFC < 6. Patients in stage IV and V were excluded.

To meet the minimum standards to deal with the lack of a validated gold standard for global cognitive status in HD, and in absence of specific and well-defined criteria, we developed a two-step approach similar to those previously used [4, 23]. The first step was the administration of the Clinical Dementia Rating Scale (CDR) [30] and the Functional Independence Score (FIS). The CDR is a global rating scale which assesses performance in memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care. It is based on a scale from 0 to 3, where 0 = no cognitive impairment (CDR = 0), 0.5 = mild cognitive impairment (CDR =), and ≥ 1 = moderate to severe dementia (CDR). In a second step, a rater blinded to the CDR and FIS score collected the clinician´s impression of the patient’s cognitive status based on a Likert-like scale, where 0 = absence of significant cognitive impairment, 1 = minor or mild cognitive impairment, and 2 = major cognitive impairment or dementia. Based on this two-step procedure, patients were classified into a given cognitive category when there was an agreement of 100% between the two raters. Accordingly, patients were categorised as cognitively normal (HD-NC) when they had a CDR = 0, a FIS score > 80%, and a second rater impression = 0. Cognitive impairment (HD-MCI) was considered mild if a CDR = 0.5, a FIS score > 80% and a second rater impression was 1. Dementia (HD-Dem) was categorized as a CDR > 0.5, a FIS score < 80%, and a second rater’s impression of 2 [4, 23, 31].

Neuropsychological assessment

Global cognitive status was assessed using the Mini-Mental State Examination (MMSE) [32] and the Parkinson’s Disease-Cognitive Rating Scale (PD-CRS) [33] as screening measures. The MMSE is part of the Enroll-HD protocol and the PD-CRS has recently shown to be a reliable instrument to assess global cognition in HD [23]. The MMSE covers orientation to time and place, attention-concentration, short-term memory, language skills, visuo-constructional abilities and ability to understand and follow instructions. The total MMSE score provides a general measure of global cognition and various cut-off scores are used to classify patients into cognitive categories ranging from mild cognitive impairment to severe dementia. The PD-CRS is composed of 9 subtests that assess immediate verbal memory, confrontation naming, sustained attention, verbal working memory, unprompted drawing of a clock, copy of a clock, delayed free recall, alternating verbal fluency and action verbal fluency. The PD-CRS provides a total score that both in HD and in Parkinson’s disease demonstrated to be psychometrically reliable in terms of differentiating patients into cognitive categories [33].

Cognition was also assessed using the set of cognitive tests comprising the UHDRS cognitive section or “cogscore”. Accordingly, we administered the Stroop color-naming, the word-reading and interference tests, the phonetic verbal fluency test with letters F, A and S (FAS), and the Symbol Digit Modalities Test (SDMT). There is no standard regarding the meaning of the total UHDRS cogscore in terms of classification of the severity of cognitive impairment based in this measure. Following randomization, the tests were administered at baseline visit and at two consecutive follow-up visits at 6 and at 12 months.

The study was approved by the local ethics committee at Hospital de la Santa Creu i Sant Pau, Barcelona, and reviewed by the institutional review board at each participating institution. The study was conducted in accordance with the 1964 Helsinki Declaration and its later amendments. All procedures were performed after completion of the written informed consent signed by each participant. The data used in this study are available under request.

Statistical analysis

Data are expressed as means ± standard deviation (SD) for the continuous variables and as mean range for the ordinal variables. Differences between groups were analyzed using independent two-tailed t tests and analyses of variance (ANOVA) for continuous variables, the Mann–Whitney test for ordinal data, and the χ2 test for categorical variables. The discriminative capacity of the various cognitive approaches was assessed using a Receiver Operating Characteristic (ROC) curve analysis to determine the sensitivity, specificity and Area under the Curve (AUC) of the different cut-off scores. Sensitivity to change over time was addressed following several approaches as a function of the interest of the analysis. First, we addressed the mean change from baseline at each time-point and group for each cognitive instrument. The percentage of change from baseline was subjected to repeated measures ANOVA to test for the existence of any kind of interaction in terms of change as a function of group category, time-point and instrument. To address the existence of differences in the dynamics on the rate of change between groups in all of these cognitive measures, we also performed between-groups ANOVA analysis comparing the scores obtained between visits. To compare differences between groups in the mean change between visits, we used a generalized linear mixed-effects model using age, education and UHDRS-TMS. This statistical model was used to mitigate the data missing from the follow-up visits. We also calculated the rate of change from one cognitive category to another along visits using the gold-standard approach. Using a sensitivity-based and specificity-based approach, we then estimated the minimally clinical relevant change associated with an increased risk of moving from one cognitive category to another at 12 months. Finally, we determined the Minimal Clinically Important Difference (MCID) at 6 months and at 12 months for each cognitive variable using an approach based on the determination of the minimum score 1.645 SD away from the mean ± SD of the difference (CI = 90%). All analysis was performed using IBM-SPSS software (version 26; SPSS. Inc. Armonk, NY) and significance was set at p < 0.05.

Results

Clinical and sociodemographic data

The sample consisted of 180 HD gene-mutation carriers (mean age = 49.6 ± 11.7; mean years of education = 12.3 ± 4.4; mean TFC = 10.4 ± 2.8). As seen in Table 1, from the total HD sample, 33 participants were classified as PreHD (mean age = 40.9 ± 8.7; mean CAG = 42.7 ± 2.5; mean years of education = 14.5 ± 4.4; mean TFC = 12.9 ± 0.4), and 147 as manifest HD (mean age = 51.6 ± 11.4; mean CAG = 43.5 ± 2.9; mean years of education = 11.8 ± 4.2; mean TFC = 9.8 ± 2.8. No discrepancies were found between raters regarding the cognitive status of patients. According to the gold-standard, among the symptomatic HD group, 52 were classified as HD-NC (mean age = 49.5 ± 9.6; mean CAG = 42.8 ± 2.4; mean years of education = 14.4 ± 3.3; mean TFC = 12.1 ± 1.0), 49 were classified as HD-MCI (mean age = 52.3 ± 11.1; mean CAG = 43.4 ± 2.3; mean years of education = 11.2 ± 4.0; mean TFC = 9.9 ± 1.6) and 46 were classified as HD-Dem (mean age = 53.2 ± 13.4; mean CAG = 44.3 ± 3.7; mean years of education = 9.5 ± 3.8; mean TFC = 7.1 ± 2.7). No participants in the PreHD group were classified as having mild or severe cognitive impairment.

Table 1 Clinical and sociodemographic characteristics of the sample
Full size table

Discriminative capacity of the cognitive approaches

In terms of discriminating between cognitive categories, the validity and reliability of the PD-CRS, the MMSE and the UHDRS cogscore were determined based on Analysis Under The Curve (AUC). This analysis was restricted to the sample of symptomatic participants because all preHD participants were classified as cognitively intact.

As seen in Fig. 1, a PD-CRS total score of 81/82 had a sensitivity of 0.83 and specificity of 0.81 (AUC = 0.898) to discriminate HD patients in the NC group from those in the MCI group. A MMSE total score of 25/26 had a sensitivity of 0.97 and a specificity of 0.39 (AUC = 0.816). Regarding the UHDRS cogscore, a score of 165/166 achieved a sensitivity of 0.84 and a specificity of 0.75 (AUC = 0.863). Focusing on the category of dementia, we found a PD-CRS total score of 61/62 had a sensitivity of 0.88 and specificity of 0.93 (AUC = 0.968). The MMSE total score 20/21 had a sensitivity of 0.97 and specificity of 0.36 (AUC = 0.803), and a UHDRS cogscore of 93/95 had a sensitivity of 0.90 and specificity of 0.54 (AUC = 0.856) to discriminate between the categories MCI and dementia.

Fig. 1
figure 1

Receiver operating characteristic (ROC) curves illustrating the discriminative properties of the PD-CRS, the MMSE and the UHDRS cogscore

Full size image

Bivariate correlation analysis showed the expected significant associations between the total scores of the various cognitive instruments, with the strongest associations being found between the PD-CRS total score and the UHDRS cogscore (r = 0.886; p < 0.0001), and between the PD-CRS total score and the MMSE (r = 0.742; p < 0.0001). The MMSE showed a mild but significant association with the UHDRS cogscore (r = 0.656; p < 0.0001).

Sensitivity to change over time

Sensitivity to change was addressed in the available population at 6 and 12 months.

Table 2 depicts the mean change at each time-point for each of the cognitive measures in each group based on a linear mixed-effects model using age, education and UHDRS-TMS as covariables. Repeated measures ANOVA showed a significant time × group interaction in terms of rate of change for the PD-CRS [F(6,59) = 2.337; p = 0.039], but not for the MMSE or for the UHDRS cogscore.

Table 2 Linear mixed-effects models with covariables (age, education and UHDRS)
Full size table

For the PD-CRS, in the preHD group, there was a significant change between baseline and 6-month visit [t(33) = − 2.2; p = 0.045] in terms of an improvement of 2.9 ± 5.3 points, mostly attributed to learning effects. No significant differences were found between other visits, either for the PD-CRS or the other instruments.

Focusing on the HD-NC, we observed a significant change between 6 and 12 months [t(52) = 2.5; p = 0.028] in terms of an improvement of 10.1 ± 16 points for the UHDRS cogscore. No significant differences were found between other visits with any instrument.

In the HD-MCI, there was a significant change for the PD-CRS between baseline and 12-month visit [t(49) = − 3.6; p = 0.001] in terms of a decrease of − 5.8 ± 8.4 points. We also noted a significant change between the 6-month and 12-month visit [t(49) = 2.2; p = 0.046] in terms of an improvement of 5.5 ± 9.7 points. No significant differences were found between baseline and 6-month visit. For the MMSE, no significant differences were found between visits. And for the UHDRS cogscore, significant differences were found between baseline and 12-months [t(49) = 2.2; p = 0.035] in terms of an improvement of 9.0 ± 19.3 points. No significant differences were found between baseline and the 6-month visit, or between the 6-month and 12-month visit.

The HD-Dem showed a significant change for the PD-CRS between baseline and 12-month visit [t(46) = − 3.4; p = 0.002], consisting of a decrease of − 4.6 ± 7.7 points. No significant differences were found between other visits. For the MMSE and the UHDRS cogscore, no significant differences were found between any visits.

Figure 2 depicts the trajectories of changes (represented as percentage of change) in cognitive measures and groups. Both the HD-MCI and HD-Dem groups showed a pattern of progressive worsening, clearly captured with the PD-CRS and with the UHDRS cogscore, but not with the MMSE. In the preHD group, no instrument captured significant decreases in performance, but learning effects were observed between baseline and 6-month follow-up.

Fig. 2
figure 2

Cognitive trajectory in terms of percentage of change with respect to baseline along follow-up visits for each cognitive instrument and group

Full size image

At the 12-month visit we calculated the rate of conversion from one category to another, both for the HD-NC and HD-MCI groups. For the HD-NC group, 7.7% of participants converted to HD-MCI, and 1.9% converted to HD-Dem. In the HD-MCI group, the proportion of participants who converted to HD-Dem reached 16.3%. To estimate the minimally clinical relevant change associated with an increased risk for moving from one category to another at 12 months, we performed a sensitivity-based and specificity-based approach. In the overall sample of symptomatic participants, ROC curves indicated that a change of 9/10 points in the PD-CRS total score indicated a significant change associated with moving to another cognitive category (sensitivity = 0.984; specificity = 0.800, AUC = 0.972; 95% CI 0.935–1.000). Regarding the other cognitive measures, we found an AUC = 0.436 for the MMSE, and AUC = 0.323 for the UHDRS cogscore. Thus, these measures did not reach minimal significance of AUC > 0.5.

Regarding each category, in the HD-NC group, a change of 5/6 points in the PD-CRS (sensitivity = 0.875; specificity = 1, AUC = 0.981; 95% CI 0.939–1.000) was associated with the transition to another cognitive category. With an AUC = 0.419 and an AUC = 0.452, neither the MMSE nor the UHDRS cogscore reached minimal significance. In the HD-MCI, a decrease of 7/8 points in the PD-CRS (sensitivity, 0.850%; specificity 1.000%, AUC = 0.975; 95% CI 0.927–1.000) indicated a significant change. The AUC for the other cognitive measures did not reach significance (AUC = 0.412 for the MMSE, and AUC = 0.176 for the UHDRS cogscore).

Regarding the MCID, Table 3 shows the minimum change with a 90% CI for being a significant change between visits, for each cognitive instrument and group.

Table 3 Minimally Clinically Important Difference (MCID) for each group, timepoint, and instrument
Full size table

Discussion

In this study, we aimed to determine the discriminative properties of cognitive instruments to establish a more reliable approach to the study of cognitive status in HD patients. We addressed the sensitivity of these instruments to change at different time-points, determined the rate of conversion among categories, and computed the minimum change expected to occur in the different scales as a function of an increased risk for worsening in terms of category conversion.

Our results favor the PD-CRS over the MMSE and the UHDRS cogscore, and provide reliable cutoff scores to differentiate cognitive categories in HD. Notably, these cutoff scores were equivalent to those obtained in a previous study and cohort, and showed excellent sensitivity and specificity values. Our results confirm both the validity of the PD-CRS to assess global cognitive status in HD individuals and the reliability of cutoff scores of 81/82 to discriminate HD subjects with MCI and with dementia, respectively. For the MMSE, the common cutoffs of 25/26 and 20/21, respectively, showed the best discriminative capacity between MCI and dementia. However, the sensitivity and specificity values associated with these cutoffs were significantly lower than the minimum standards required to consider the instrument valid [34]. Thus, according to our results, the MMSE does not appear to qualify as a suitable instrument to detect MCI and/or dementia in the context of HD.

The UHDRS cogscore has been used for several years in the context of multicenter observational clinical trials [18, 35]. Several measures comprising this composite score are extremely sensitive to overall disease progression as has been shown in previous studies. However, to date, no study has provided a meaningful clinical significance of the UHDRS cogscore in terms of its use to determine the cognitive status of the patients. Our study fills this gap and shows that the UHDRS cogscore cutoffs obtained for MCI and dementia have an acceptable sensitivity and specificity relationship. However, the UHDRS cogscore is still a combination of tests based primarily on executive, attentional and psychomotor processes that certainly do not cover the broad spectrum of cognitive impairment that can be heterogeneously present in HD people. Therefore, although our findings indicate that it is useful to discriminate between cognitive categories, a more global scale should be preferred when the objective is also to explore the global cognitive status of an HD subject.

In relation to sensitivity to change, it is noteworthy that the only instrument that showed significant differences in the rate of progression at 6 and 12 months was the PD-CRS. This means that while the UHDRS cogscore or any measure sensitive to disease progression changes over time, the trajectory of this change when using instruments such as the MMSE or the UHDRS cogscore is the same regardless of the cognitive status of the patients. Because it is conceptually evident that the trajectory of cognitive changes can be expected to differ depending on the cognitive status of the patients, it could be assumed that, from the point of view of cognitive assessment, an efficient instrument should be able to distinguish trajectories of the different cognitive groups. This does not occur in the case of the MMSE or the UHDRS cogscore, but it is evident and significant in the case of the PD-CRS. Specifically, the PD-CRS is the only instrument that showed significant involutional changes throughout the 12-month follow-up in both the HD-MCI group and the HD-Dem group.

Regarding the rate of conversion from one cognitive category to another one, we provide evidence that in our cohort, during the follow-up period, 7.7% of participants from the HD-NC group converted to HD-MCI, 1.9% converted to dementia, and 16.3% converted from HD-MCI to dementia. We consider that taking these category changes into account is relevant once we have been able to show that the trajectory of cognitive measures differs depending on the category to which the subjects belong. This implies that when monitoring the trajectory of HD patients (i.e., in the context of a clinical trial) it is essential to know each patient’s cognitive status to be able to accurately understand the meaning of the changes or stability observed in the measures used.

Analysis of the changes in the various cognitive measures associated with the rate of conversion from one category to another showed that a change of 9/10 points in the PD-CRS was associated with a change in cognitive category regardless of the category to which the subjects belonged. Focusing on individuals in the HD-NC group, a 5/6point change was reliably associated with conversion from one category to the other, whereas in the HD-MCI group, a 7/8-point change was required for transition to dementia. These measures of relevant cognitive changes were statistically valid only for the PD-CRS, and the values obtained for the MMSE or the UHDRS cogscore did not reach the minimum required. In this regard, this does not call into question the demonstrated sensitivity of the UHDRS cogscore to track disease progression, but it does demonstrate that it lacks validity for the purpose of tracking conversion to MCI or to dementia.

Finally, we considered it useful to study the Minimal Clinically Important Difference (MCID) for each instrument in the whole sample and in each group. This approach allows us to determine whether an observed change—regardless of whether we are talking about transition or change of cognitive category—is statistically significant. This is particularly useful in the context of monitoring disease progression, such as in interventional studies. Our results show that the MCID is different for each instrument, group, and time point. Therefore, when these instruments are used in the monitoring of cognitive status in people with HD, they must be adjusted according to the time interval studied and the group or cognitive category.

In summary, current diagnostic criteria for the cognitive syndromes in HD are limited and the need for reliable instruments to assess cognitive status is evident. The PD-CRS shows excellent discriminative properties that have been replicated in two studies and in two different cohorts [4, 23]. Despite the lack of specific definitions of MCI and dementia in HD, this tool shows adequate discrimination between cognitive groups. It is sensitive to change both in relation to conversion from one cognitive category to another and also in relation to monitoring the progression of cognitive deterioration. Besides, it is the only instrument available to date that is able to show the different trajectories of cognitive deterioration. In parallel, we provide the cutoffs that can be used for the PD-CRS, MMSE, and UHDRS cogscore to discriminate between cognitive groups. Especially with regard to the UHDRS cogscore, we believe our findings are particularly useful as these tests continue to be widely used in observational studies of large cohorts of patients (i.e.: Enroll-HD). Having this approximation based on the cutoffs thus allows us to exploit the data from this study according to this type of classification.

We are fully aware that the PD-CRS is merely a screening approach for overall cognitive status, which, although it has several strengths compared to other screening instruments, is not without limitations. Cognitive manifestations in HD encompass changes in processes that are not covered by the PD-CRS. This means that a comprehensive assessment of patients’ cognitive status should follow the standards of neuropsychological assessment, taking into account the role of processes such as certain language impairments [36, 37] or social cognition [38,39,40], as well as specific neuropsychiatric manifestations [41, 42]. Also, the PD-CRS does not explore relevant processes in the HD context such as psychomotor speed and word reading [11, 43]. However, further analysis of present data showing that a global scale such as the PD-CRS performs well in an HD population may serve to complement or recombine current instruments and contribute to the design of new tools.

This study has other points to consider. First, other instruments, such as the MoCA, could be explored in future studies for their discriminative capacity and sensitivity to change in HD. Second, the restrictions imposed by the pandemic contributed to an unexpectedly high rate of follow-up losses that may have somehow influenced some results. Nevertheless, a considerably representative sample was still examined at the three time points of the study.