Background

Huntington’s disease (HD) is an inherited neurodegenerative disease caused by expansion of CAG trinucleotide repeats secondary to mutation in the huntingtin gene on chromosome 4p16.3 [1]. The disease is characterized by involuntary hyperkinetic movements, cognitive impairment and behavioural disorders. Cognitive impairment, which may be evident even in gene-positive individuals yet to be clinically diagnosed [2–5], is progressive in nature and a contributing factor in the loss of everyday function [6]. Subtle cognitive impairment can be overlooked by clinicians during routine follow-up [7], indicating the need for easily administered and yet robust tool to detect cognitive changes in HD. Comprehensive neuropsychological testing is necessary to verify cognitive status. However, neuropsychological batteries are time-consuming and brief cognitive screening tools such as the Mini Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA) are commonly used in clinical settings and in a broad range of conditions. The Unified Huntington’s Disease Rating Scale (UHDRS), a standard assessment tool for HD, also includes a brief cognitive component.

The MMSE [8] comprises eleven questions spanning five aspects of cognitive function: executive function, language, memory function, visuospatial ability and orientation. It has good inter-rater, test and re-test reliability in differentiating cognitive status in dementia syndromes [9] and other disorders featuring cognitive impairment [10]. Nevertheless, it is influenced by demographic factors such as age, education and cultural background [9, 11, 12]. The MoCA places greater emphasis than the MMSE on naming, attention, abstraction and delayed recall, functions that are most likely to be compromised in the earlier stages of cognitive impairment and unlike the MMSE, it compensates for education level [13]. Both the MMSE and MoCA have been employed as measures of cognitive performance in manifest HD patients [14–17] and MoCA was also found to have higher sensitivity without losing specificity than the MMSE in identifying those with cognitive impairment in HD [16]. Furthermore, Bezdicek et al. [17] demonstrated a strong correlation between the MoCA scores and comprehensive neuropsychological assessment scores in manifest HD patients. The UHDRS cognitive component [18] includes three tests of executive function – letter fluency test, Symbol Digit Modalities test and Stroop test; which can be used with corrected norms to attenuate the impact of various demographic variables [19, 20].

The progressive nature of HD means that any cognitive assessments should also be useful longitudinally. HD patients are routinely followed up at clinics at 6-month and 12-month intervals thus it is preferable that brief cognitive assessment tools are sensitive to changes even over relatively short time intervals. Effective yet brief cognitive tools would enable easier detection of cognitive changes in HD patients in clinic settings than time consuming comprehensive cognitive assessment and also assist health care providers in designing treatment and care plans aimed at improving patient’s quality of life. MMSE and MoCA have been extensively evaluated in previous cross-sectional HD studies [14–16] but to our knowledge, there is no longitudinal data on the utility of these brief cognitive tests compared to UHDRS cognitive assessment in monitoring cognitive changes in HD patients. Therefore the objective of this study was to examine and compare the relative utility of two widely used brief cognitive tests (MMSE and MoCA) concurrently with the UHDRS cognitive assessment to a comprehensive neuropsychological test battery for monitoring cognitive changes in HD patients over a short interval of 12 months. Such a direct comparison has not been previously reported.

Methods

Study participants were a convenience sample of 22 manifest HD patients (10 males and 12 females) with mild to moderate disease severity and 22 age, gender and education matched control volunteers recruited through the New Zealand Brain Research Institute database (Table 1). Patients were genetically verified and clinically diagnosed by a movement disorders specialist (TJA). Participants identified themselves as native speakers of English and consented to participate in compliance with the requirements of the New Zealand Ministry of Health Ethics Committee.

Table 1 Demographic characteristics (mean and SD) of control and HD groups
Full size table

Experiment procedure

The MMSE, MoCA and a comprehensive neuropsychological test battery were administered to all participants. The comprehensive assessment of cognitive function used 19 neuropsychological tests to assess six domains of cognitive function (executive function; working memory and attention; learning and memory; processing speed; language; and visuospatial function). These tests were: executive function: letter, action and category fluency tests [21], Trail Making Test (Part B), Stroop-interference test; working memory and attention: digits forward, backward and sequencing tests [22], Symbol Digit Modalities Test [23] and Ruff 2 & 7 Cancellation Test – Accuracy [24]; learning and memory: Short California Verbal Learning Test-II and Brief Visuospatial Memory Test-Revised; processing speed: Stroop-word reading, Stroop-colour naming, Trail Making Test (Part A) and Ruff 2 & 7 Cancellation Test – Speed; language: Brief Boston Naming Test [25] and Indiana University Token Test [26]; and visuospatial function: Judgement of Line Orientation test (Form H) and Rey Complex Figure Copying test. For the MMSE, both alternatives (‘World’ spelled backwards and serial sevens) were assessed. The number of tests administered was evenly distributed over two separate sessions, one week apart and presented in the same order for all participants. Each session began with the MMSE in the first and MoCA in the second session. The three-part UHDRS, comprising motor, cognitive and behavioural components, was administered in the first session to the HD group only. All returning participants were reassessed in identical manner 12 months later.

Data analysis

Cognitive status – normal, mild cognitive impairment (MCI) or dementia – of HD participants was determined using evidence from the neuropsychological test battery and the UHDRS. Criteria for mild cognitive impairment (MCI) followed that described for Parkinson’s disease by Dalrymple-Alford et al. [27], with a requirement of 2 measures at -1.5SD or equivalent within a single domain; and dementia criteria followed that of Peavy et al. [5], which defined HD dementia as having cognitive deficits in at least two areas of cognition not limited to memory deficits in the context of impaired everyday function as determined through the UHDRS Functional Independence Scale.

The raw score of each component test in the neuropsychological test battery was converted to a standard z-score using test-specific norms so that objective comparison can be made across component tests, regardless of individual scale ranges and distributions. Domain-specific scores were mean aggregated scores of component tests within a cognitive domain and the average scores across all six cognitive domains determined the global cognitive z-score (overall global cognition). The MoCA scores were adjusted to participants’ education level [13]. The three cognitive tests (letter fluency, SDMT and Stroop tests) in the UHDRS cognitive component were part of the neuropsychological test battery so the mean aggregate z-score of these tests was used as the UHDRS cognitive score for both the HD and control groups.

Statistical analysis

For each of the measures, the differences between groups at baseline and the change over 12 months were determined using linear mixed-effects models [28]. These models take into account the correlated measurements within a participant when assessing the differences between groups and changes over time. The relationship between global cognition and brief cognitive tests, baseline and 12-month scores were assessed using R2 correlation coefficient from simple linear models. Bootstrapping [29] was used to assess differences between R2 correlation coefficients as standard analytical techniques were not applicable. Bootstrapping involved the original sample being resampled with replacement and the difference between R2 values in this new sample being determined. This was repeated 1000 times and the resulting distribution of differences in R2 values gave an indication of the mean difference and a 95% confidence interval. Cohen’s d was used to report effect sizes of differences between groups.

Results

Cognition at baseline and change over time

At baseline, six HD patients had normal cognition, 10 met criteria for MCI, and six had dementia [5]. All 22 controls had normal cognition. The HD group showed significantly reduced scores (t > 3.5, p ≤ 0.001) in overall global cognition and brief cognitive tests compared to controls both at baseline and at 12-month follow-up. The mean effect sizes for the two years combined in overall global cognition was d = 2.6 whereas in the brief cognitive tests, they ranged from d = 1.3 in the MMSE with ‘World’ spelled backwards to d = 2.4 in UHDRS cognitive assessment (Figure 1A, Table 2). In terms of domain-specific scores, the HD group had significantly lower scores (t > 4.7, p < 0.001) compared to controls across all cognitive domains and the mean effect sizes for baseline and 12-month combined ranged from the smallest (d = 1.5) in the language domain to the largest (d = 2.8) in the executive function domain (Figure 1B, Table 2). Individual component test scores in HD and control groups at baseline and 12-month are detailed in Additional file 1: Figure S1 and brief cognitive tests in Additional file 2: Table S1.

Figure 1
figure 1

Change in cognitive scores over 12 months. Baseline and 12-month scores for control and HD groups in: (A) overall global cognition; UHDRS cognitive assessment; MMSE-WORLD; MMSE-Sevens; MoCA; and (B) the six cognitive domains. Group mean and SD are shown.

Full size image
Table 2 Scores # at baseline, within-group changes and group over time interactions of control and HD groups
Full size table

There was an overall pattern of improvement in the control group across all cognitive tests after 12 months. In contrast, the HD group showed minimal change in their global cognitive z-score and a general decline across all brief cognitive tests scores after 12 months, for which it was statistically significant (t < - 2.0, p = 0.048) in UHDRS cognitive score and MMSE with ‘World’ spelt backwards (Figure 1A, Table 2). The control group exhibited an increase in score at 12 months in most cognitive domains (t > 2.2, p < 0.04) excepting executive function and visuospatial domains (t < 1.6, p > 0.1). Contrastingly, the HD group demonstrated a decline (t = 2.3, p = 0.03) in executive function domain but an increase (t = 2.7, p = 0.01) in language domain z-score 12 months later. There were no significant absolute changes (t < 1.2, p > 0.3) in the other cognitive domains in the HD group (Figure 1B, Additional file 2: Figure S1). A relative change over time (i.e. relative deterioration) in the HD group compared to change in scores in the control group was significant for global cognitive z-score, MMSE with ‘World’ spelled backwards, executive function, and learning and memory domain scores (Table 2). There was a significant worsening (t = 3.9, p < 0.001) of the UHDRS motor score over 12 months in the HD group but no change in behavioural score (Table 2).

Usefulness of scores for measuring change over time

There are several considerations to take into account when determining which brief cognitive test has greatest utility for measuring cognition over time. This includes how well the score reflects overall global cognition, whether there is any ceiling effect, and how noisy (variability of score residuals) the score is after taking systematic changes into consideration.

Simple linear models confirmed that scores of all three brief cognitive screening tests, as judged by their R2 values, were significantly correlated with the scores of the full neuropsychological test battery at baseline (Figure 2) and 12-month (not shown). Bootstrap procedures confirmed that there were no significant differences between the three brief cognitive screening tests in extent of correlation with overall global cognition (Table 3). Thus all brief cognitive tests provided a reasonable cross-sectional measure of global cognition.

Figure 2
figure 2

Correlations between brief cognitive screening tests scores and global cognitive z-scores at baseline. The control group is shown in the top row and the HD group in the bottom row. The R2 and the p values are shown for each of the brief cognitive screening tests.

Full size image
Table 3 Comparison of R 2 differences of relationships between brief cognitive tests and full cognitive battery
Full size table

To determine the variability of score residuals over time, simple linear models were fitted to the baseline and 12-month scores of overall global cognition and brief cognitive tests. The correlations within a test over time were evaluated by examining the R2 values of the model fits (Figure 3). In the control group, the range of scores in MMSE and MoCA was narrow due to a ceiling effect hence contributing to R2 values (R2 < 0.36). In contrast, the global z-score and UHDRS cognitive component showed greater utility in the control group, with a wider range of values together and small deviations from the linear fit, resulting in high R2 values. In the HD group the baseline scores were well correlated (R2 > 0.67) with 12-month scores for overall global cognition and all three brief cognitive tests (Figure 3). The comprehensive neuropsychological test battery and UHDRS cognitive component, as confirmed by bootstrap procedures, had smaller deviations from the linear fit than the two versions of the MMSE but not the MoCA (Table 4).This finding indicates that the two versions of MMSE had higher measurement noise (i.e. greater score variability over time), compared to overall global cognition and UHDRS cognitive assessment after 12 months.

Figure 3
figure 3

Correlations between baseline and 12-month scores of the five cognitive measures. Control group is shown in the top row and HD group in the bottom row. The R2, 95% CI (in square brackets) and p values of the relationship are shown for each of the cognitive measures.

Full size image
Table 4 Comparison of R 2 differences of relationships between baseline and 12-month scores in cognitive measures
Full size table

In summary, the combination of showing a significant decline of cognition in HD (Table 2), high correlation with global cognitive z-score (Figure 2), and lower variance of score residuals over time (Figure 3) compared to other brief cognitive assessments, indicated the UHDRS cognitive component performed the best of the brief cognitive tests in assessing and monitoring cognition in HD patients over a 12-month period.

Discussion

This study attempted to evaluate the usefulness of two widely used brief cognitive assessment tools (MMSE and MoCA) simultaneously with UHDRS cognitive component for monitoring cognitive changes in manifest HD patients over a 12-month interval by comparing them to a comprehensive neuropsychological test battery. In the process of evaluating the usefulness of these brief cognitive tests, we demonstrated that there was no significant change in overall global cognition in the presence of significant decline in the executive function domain in manifest HD patients after 12 months. Relative to the control group, which showed an increase in overall global cognitive z-score and learning and memory domain score over a 12-month period, there was significantly less change in domain-specific scores in the HD group over that period. The MMSE and MoCA were less effective than the UHDRS cognitive assessment for monitoring cognitive changes in manifest HD patients over 12 months.

Domain-specific cognitive performance

Cognitive decline, which has been shown to assume a relatively slow course especially in the early stages of HD [30–33], is a well-established hallmark of HD. Overall, our findings corroborated with other longitudinal studies on pre-manifest and early manifest HD patients wherein, relative to a control group, cognitive decline were evident after a 12-month interval in the HD group [30] and, similar conclusions were made at 24-month follow-up [31]. The significant decline in executive function domain score in the HD score was consistent to a study by Bachoud-Lévi et al. [32], which demonstrated that cognitive deterioration in early stages patients is limited to attention and executive functions. However, unlike their study which also showed significant changes in visuospatial and language functions over time, such changes were not evident in our study suggesting that such changes were limited in early HD and not across different stages of HD. Executive function domain has always been recognized to be the most vulnerable in HD [34] with progressive impairment evident not just in early stages of HD [32, 35] but also in pre-manifest HD patients [36].

In contrast to the HD group, which had minimal change in overall global cognition over time, the control group had a significant improvement in their scores over time. This suggested that controls had in general benefitted from practice effect on repeated testing of measures in the comprehensive neuropsychological test battery. This was consistent to previous works on cognitive performance in healthy controls in longitudinal studies [37, 38]. Practice effect in healthy controls is most apparent in the early phases of repetitive testing, with performance scores tending to plateau on subsequent testing [39, 40], or after changing to low frequency testing [41]. Atrophy of the caudate nucleus, a structure involved in learning process, is found in normal aging process but this process when compared to healthy controls, occurs at an expedited rate in HD patients as demonstrated through serial radio-imaging studies [31, 42]. Nevertheless, as already reported in a 2 to 4 year longitudinal study on cognition in early HD patients, practice effect was evident in certain executive function and memory related tests between first and second assessments [32]. Although it was reported in that same study [32] that practice effect was not observed in language performance, our HD cohort had actually shown a significant improvement in language domain score after 12 months. These findings suggested that HD patients indeed could benefit from practice effect over a relative short time interval and also showed that underlying disease progression may not be translated to measurable cognitive performance changes over short time interval.

Our findings reaffirmed the general slow progression of cognitive deterioration in HD patients over short time interval which inevitably create great difficulty in monitoring cognitive changes in HD patients on routine follow-up in clinic settings. Longitudinal monitoring of disease progression is generally conducted to evaluate potential interventions for delaying phenoconversion in HD thus the generally accepted view is that it is more meaningful to serially evaluate disease progression of pre-manifest HD patients. However, understanding short-term changes and the utility of various cognitive tools in manifest HD patients are also important for multi-disciplinary health team in planning and modifying disease management plans which consists of currently available pharmacological and non-pharmacological interventions aimed at improving patient’s quality of life. Nevertheless, our findings have implications for clinical practice and research. Cognitive decline in HD appeared to be specific in executive function and learning and memory domains after 12 months. Therefore in the clinic, cognitive deterioration in HD over 12-month should not be determined by changes in overall global cognitive score of comprehensive neuropsychological test battery but by detailed analysis of cognitive domain-specific performance. Due to practice effect, it is important in short to medium term longitudinal clinical research to include a control group when assessing the cognition of HD patients.

Usefulness of brief cognitive tests for longitudinal assessment

As expected, the MMSE, MoCA and the UHDRS cognitive component scores correlated well with overall global cognition, as determined through the comprehensive neuropsychological test battery, in the HD group. These findings support the utility of the three brief cognitive assessment tools in cross-sectional detection of cognitive deficits in manifest HD patients. Furthermore, our findings showed that there were no significant differences between the three brief cognitive tests in reflecting overall global cognition in HD patients, providing no evidence that one test is better than the other in this respect.

However, the baseline scores of comprehensive neuropsychological test battery (overall global cognition) and UHDRS cognitive assessment were highly correlated with their 12-month scores and as judged by the 95% CI of R2 values, both types of assessment had minimal deviations from the linear fit indicated that both tests were reliable and had low score variability over time. The reliability of the MMSE in the HD group, though reasonable, was significantly lower than that for the full neuropsychological test battery and the UHDRS cognitive component. Deficiencies in reliability of MMSE were highlighted in a study by Bowie et al. [43], which inferred that the test was inadequate in detecting small cognitive changes. Moreover, large score variance on annual assessment was another weakness of MMSE as shown in a study on patients with Alzheimer’s disease [44], which further limits its value in assessing disease progression. Similarly in our HD sample, the two versions of MMSE were found to have greater score variance than the comprehensive neuropsychological test battery and the UHDRS cognitive component. Even though the present study demonstrated that there were significant within-group changes after a 12-month period in the MMSE (with ‘World’ spelled backwards) in our HD patients, its use in routine follow-up in clinical practice should be interpreted with caution because of its tendency to vary from one assessment to the next. On the contrary, the MoCA and UHDRS cognitive component, as judged by the differences of R2 values from linear fit models using bootstrap procedure, had comparable performance to the comprehensive neuropsychological battery. Such findings are likely to be attributed by the nature of short-term cognitive progression in HD which is specific to executive function and also the overall design of the tests. The MoCA, which was claimed to have superior sensitivity for detecting MCI compared to MMSE, contains more demanding tasks for assessing executive and memory functions [45] while the UHDRS cognitive component essentially assesses the executive function domain. However, MoCA is an assessment tool that examines multiple cognitive domains hence similar to the overall global cognitive score of comprehensive neuropsychological battery, short-term cognitive decline in HD patients could be masked by practice effect in other domains within the test.

On the basis of high correlation to comprehensive neuropsychological battery and low variance across time, the UHDRS cognitive component is a good brief substitute for comprehensive neuropsychological testing and a sensitive cognitive measure to assess short-term cognitive changes in HD patients compared to MMSE and MoCA. However, the MoCA and MMSE, in that order, might be considered as reasonable alternatives to the ‘gold standard’ for use in clinic setting in circumstances where the UHDRS cognitive component is unavailable but secondary to the limitations of MMSE and MoCA, their results shall be interpreted discretely.

Other disease measures

There was no significant worsening in the UHDRS behavioural score within our HD group at follow-up, similar to prior observations [18]. Behavioural abnormalities in HD are heterogeneous in nature and without clear temporal progression [46]. Furthermore, psychiatric interventions are often effective in managing behavioural disturbances of HD patients [47] so such features are less likely to exhibit progressive deterioration over time. Thus, the UHDRS behavioural index is not particularly useful as a measure of short to medium term disease progression in HD. In contrast to the absence of measurable change in the behavioural measure, there was a significant increase in the UHDRS motor score over 12 months. This is consistent with the Huntington Study Group’s [18] report of an average three points increase in motor score over six months in manifest HD patients. The ability to demonstrate increase in the UHDRS motor score is not exclusive to manifest HD patients, with another study on pre-manifest patients showing that while the change was minimal after one year, there was significant increase in motor scores over five years [48]. These observations combined suggest that motor deterioration is possibly more aggressive in the short term than cognitive and behavioural changes in HD patients. However, the interpretation of the study findings was undoubtedly constrained by its small sample size and also limited number of patients in different disease stages.

Concluding remarks

Although MMSE and MoCA have been evaluated in previous cross-sectional HD studies [14–16], the utility of these brief cognitive tests has not been appraised longitudinally. This study provided a new perspective on the utility of two widely used brief cognitive assessment tools (MMSE and MoCA) in comparison to UHDRS cognitive assessment and other measures for monitoring cognitive changes in manifest HD patients over a 12-month period. MMSE and MoCA may be effective for describing global cognition in HD patients in cross-sectional analysis but they are less useful for monitoring longitudinal cognitive changes over short time interval. The UHDRS cognitive assessment, which focuses on testing executive function, is sensitive to short-term cognitive changes in HD and a more reliable brief assessment tool compared to MMSE and MoCA over 12 months. Nevertheless, our findings on the utility of these assessment tools in a restricted cohort of HD patients should be interpreted discretely and further studies on these brief cognitive tests are warranted in the future.