Background

Patients with persistent symptoms attributed to Lyme borreliosis often report a variety of cognitive symptoms. However, subjective cognitive symptoms are not always due to underlying cognitive impairments. Previously, subjective ratings of memory capabilities and objective memory performance were only weakly correlated in patients with post-treatment Lyme disease [1]. Most previous studies that compared cognitive performance of Lyme patients to performance of healthy controls found a worse performance in Lyme patients at group level [2,3,4,5,6,7,8,9,10,11,12]. The most affected cognitive domain was episodic memory. Findings concerning the domains verbal fluency and processing speed were less consistent. However, most studies were relatively small (n < 80), originated from the US, and used diverse inclusion criteria and methods. In the 4 larger US studies, mild to no cognitive abnormalities were identified [1, 2, 12, 13]. As both the Borrelia species and the clinical presentation of Lyme disease in Europe and the US differ [14], cognitive function in European Lyme patients requires separate assessment. Furthermore, most previous studies have not taken performance validity into account. This is crucial, as a suboptimal performance results in poor tests scores not reflecting an individual’s actual cognitive status. Very recently, the study by Touradji et al. in a group of US patients with post-treatment Lyme disease showed that 24% of the sample displayed suboptimal effort on measures of performance validity [12]. Hence, suboptimal performance affects the validity and reliability of neuropsychological outcomes, resulting in false positive results (i.e., patients incorrectly labelled as having a cognitive impairment) [15]. This stresses the need to take performance validity testing into account when cognitively assessing patients with persistent symptoms attributed to Lyme disease.

The aim of the present study was to objectively assess cognitive performance using sensitive tests in a large cohort of patients with persistent symptoms attributed to Lyme borreliosis, while taking performance validity into account, and to compare cognitive performance outcomes with subjective symptoms.

Methods

The current study uses baseline data collected between 2010 and 2013 as part of the Persistent Lyme Empiric Antibiotic Study Europe (PLEASE). Previously, we reported the primary and secondary outcome measures of this multicenter, placebo-controlled, double-blind randomized controlled trial from the Netherlands (ClinicalTrials.gov NCT01207739) [16, 17]. The local Institutional Review Board approved the PLEASE protocol, and informed consent was obtained from each participant. Here we provide a detailed report of the baseline cognitive and self-report questionnaire data. The study population comprises adult patients (n = 280) referred with persistent symptoms attributed to Lyme borreliosis, preceded by confirmed symptomatic Lyme disease or accompanied by positive B. burgdorferi IgG or IgM antibodies, as confirmed by means of immunoblot assay. Patients were not required to have received antibiotic treatment before study entry. Major symptoms included musculoskeletal pain, cognitive disturbances and/or fatigue. Details about inclusion and exclusion criteria have been published previously [16].

Outcomes

Cognitive performance was assessed using an extensive neuropsychological test battery covering five major cognitive domains: episodic memory, working memory / attention, verbal fluency, information-processing speed and executive function. Episodic memory was assessed using the Rey Auditory Verbal Learning Test (RAVLT), working memory / attention with the Digit Span, verbal fluency with the Category Fluency Test (animal/profession naming), and information-processing speed with the Trail Making Test Part A and the mean response time of cards I and II from the Stroop Color-Word Test, and the Symbol-Digit Substitution Test. Executive function was measured using the Interference Score of the Trail Making Test (Part B/Part A) and the Stroop Interference Score (card III/mean of cards I and II). Assessment details have been published previously [17].

To identify participants with insufficient performance validity, the Amsterdam Short Term Memory test (ASTM) was administered [18]. A poor performance on this task indicates suboptimal mental effort. The recommended cut-off score is 85 (maximum score = 90), with 86% sensitivity and 87% specificity [19]. However, since our goal was to prioritize optimal specificity (> 90%), adopting a conservative approach that reduces the risk of false alarms on performance validity tests (i.e., incorrectly labelling a participant as someone displaying suboptimal effort), we used a cut-off score of < 83 (which has a specificity of 95% and a sensitivity of 76%).

Subjective measurement of cognitive function was assessed with the Cognitive Failures Questionnaire (CFQ) [20], fatigue by the Checklist Individual Strength (CIS) [16], anxiety and depressive symptoms by the Hospital Anxiety and Depression Scale (HADS) [21], self-efficacy by a modified version of the Arthritis Self-Efficacy Scale (i.e., ‘pain’ replaced by ‘physical symptoms’) [22], illness cognitions by the Illness Cognition Questionnaire (ICQ) [23], worrying by the Penn-State Worry Questionnaire (PSWQ) [24], neuroticism and extraversion by the Eysenck Personality Questionnaire (EPQ) [25], and fear of body sensations by the Body Sensations Questionnaire (BSQ) [26].

Statistical analysis

First, we investigated which demographic/psychological factors were associated with poor performance validity. For patients with sufficient performance validity, we determined whether their cognitive performance was impaired by comparing individual test performances to an extensive normative sample (n = 26,939) from the Advanced Neuropsychological Diagnostics Infrastructure (ANDI) [27]. We performed a multivariate normative comparison (MNC) on each patient’s neuropsychological test profile, applying corrections for age, sex and education level. The MNC provides an individual classification based on the profile of tests as either ‘cognitively impaired’ or ‘cognitively unimpaired’ [27].

We explored performance on individual cognitive domains, averaging age-, sex- and education-adjusted z-scores per domain. A domain was classified as ‘impaired’ if z < − 1.5 (i.e., more than 1.5 SD below the normative mean). The relation between objective cognitive functioning and subjective complaints was analyzed with Pearson correlation coefficients.

Alpha was set at 0.05 throughout (two-tailed), and 95% confidence intervals are reported when appropriate. Benjamini-Hochberg correction was used to reduce the false discovery rate for multiple comparisons, accepting a false discovery rate of 0.10.

Results

Of the 280 patients included, one was unable to perform several neuropsychological tests due to visual impairment unrelated to Lyme disease. Of the 279 patients fully examined, 239 (85.4%) displayed sufficient performance validity.

Table 1 shows patient characteristics stratified by performance validity status.

Table 1 Demographic and psychosocial factors stratified by performance validitya
Full size table

Patients with insufficient performance validity had significantly lower education levels and less often paid work. They also reported significantly more feelings of helplessness (ICQ subscale helplessness, F(1,275) = 9.77), experienced more physical fatigue (CIS Activity subscale, F(1,276) = 6.79), and reported more problems in daily orientation (CFQ Orientation subscale, F(1,276) = 8.40).

Compared to the normative sample, 2.9% of patients (7/239) were cognitively impaired. For the separate domains, 2.1% were impaired on episodic memory (5/239), 5.4% on working memory / attention (13/239), 0.8% on verbal fluency (2/239), 2.1% on information-processing speed (5/239), and 0.4% on executive function (1/239). Table 2 shows the raw neuropsychological test scores as well as mean z-scores, and percentage of individuals with a cognitive decrement (z-score < 1.0 SD below the age, sex and education-adjusted normative mean).

Table 2 Neuropsychological test scores per domain and per testa
Full size table

No significant correlation between overall cognitive performance and subjective cognitive complaints (CFQ total score) was found in patients with sufficient performance validity (r = 0.120, p = 0.064). A significant correlation was found between objective performance and problems in orientation (CFQ subscale orientation; r = 0.246 (p < 0.001). Performance and other CFQ subscales did not show any correlations (p = 0.10–0.98).

Discussion

We have assessed neurocognitive function in the largest sample of patients with persistent symptoms attributed to Lyme disease so far, using sensitive tests and extensive normative data. Furthermore, this study was the first European study to take performance validity into account. In our study, 15% of patients displayed insufficient performance validity using a conservative cut-off that prioritizes specificity over sensitivity (i.e., reducing the chance of incorrectly labelling an individuals as displaying poor performance validity), indicating that their neuropsychological test scores cannot be interpreted reliably as they might not reflect the patients’ actual cognitive abilities. Of note, there remains a possibility that this group with insufficient performance validity contains legitimate poor performers. For example, they may have been too fatigued to perform sufficiently. Our percentage of insufficient performance is considerably lower than in the study of Touradji et al. [12] who found that 24% of their sample of post-treatment Lyme disease patients showed suboptimal effort, which may have been due to our conservative approach in assessing suboptimal effort. These substantial proportions of patients displaying suboptimal effort, however, illustrate the need for performance validity testing when assessing cognition in patients with persistent neuropsychological symptoms [15]. Patients who displayed insufficient effort reported more fatigue, memory-orientation difficulties, and more feelings of helplessness than the optimal performers. Additionally, these patients were more often without paid work and had lower education levels. It should also be noted that only 22.5% (9/40) of the individuals who displayed suboptimal effort would have been classified as ‘cognitively impaired’ based on the performance on all other neuropsychological tests. After exclusion of patients with poor performance validity, only 2.9% had impaired cognitive function. This rate is low compared to the high level of cognitive complaints reported by patients (on average about 1 SD above the normative mean on the CFQ) [20]. The lack of correlation between objective and subjective cognitive functioning, was also reported in another large study in patients with Lyme-associated symptoms [1]. This lack of correlation between objective cognitive performance and subjective cognitive complaints is not specific for Lyme, but has been demonstrated in other disorders as well (including HIV, dementia, and rheumatoid arthritis) [28,29,30]. Subjective cognitive complaints are often associated with depressive symptoms [29]. The percentage of 2.9% cognitively impaired patients is comparable to what is found in the normal population (i.e., by definition 2.3% of a normative sample performs worse than 2 SD below the normative mean). Similar to our results, the study by Kaplan et al. [1] found only a small percentage of cognitively impaired individuals. That study also examined personality characteristics, albeit with a smaller sample size and a less extensive test battery than the present study. Another large study did not find any differences between patients and healthy controls in cognitive function either [2]. However, a very recent large study, which did take performance validity into account, found a much higher percentage [12]. Furthermore, two studies by Keilp et al. found distinctive cognitive difference between patients with symptoms attributed to Lyme disease and healthy controls [6, 7]. We can only speculate on an explanation for the difference in impairment for the patients with sufficient performance. Possibly, differences between Borrelia species in the US and Europe may play a role. In addition, differences in recruitment bias across the various studies may also have played a role. For instance, in the paper by Touradji et al. [12] it is stated that participants partially were self-referred, whereas our patients were all referred to the study centers by a primary care physician or medical specialty.

In addition to the low prevalence of cognitive impairments in our study, the pathogenesis of impaired cognition in relation to Lyme disease is still unclear, with scarce evidence for underlying central nervous system pathology [31].

A potential limitation of the present study is the absence of a contemporaneous control group of healthy individuals. However, we compared the individuals’ performances to a substantially larger normative sample, with specific adjustments for age, education and sex, than would have been ever possible with recruiting our own controls. Additionally, the fact that our study population was more heterogenous than previous studies could be seen as a limitation, and not all patients received previous treatment with antibiotics.

Conclusions

The present study, taking performance validity into account in a large, well-defined cohort of patients with persistent symptoms attributed to Lyme borreliosis, demonstrates that only a small percentage of patients can be classified as cognitively impaired. Furthermore, self-reported symptoms of cognitive problems are unrelated to performance on neuropsychological tests in patients with Lyme-associated symptoms.