Introduction

Alzheimer’s disease (AD) is characterized by the accumulation of amyloid-β plaques, which has been shown to occur decades before symptom onset [1, 2]. Amyloid-β pathology can be detected in vivo by positron emission tomography (PET) using amyloid-β radiotracers such as [11C]Pittsburgh compound-B (PIB), [18F]Florbetapir, [18F]Florbetaben, or [18F]Flutemetamol allows to directly visualize fibrillary amyloid-β deposits in brain tissue [3,4,5,6]. Alternatively, Aβ42 levels in cerebrospinal fluid (CSF) reflect the concentration of soluble amyloid-β, which correlates with cerebral amyloid-β depositions [7]. PET and CSF have been included as equal alternatives into diagnostic criteria for both research [2, 8, 9] and clinical practice [10,11,12], although they measure amyloid in different pools (i.e., CSF and cortical brain tissue). In addition, it has been repeatedly shown in memory clinic cohorts that in 10–20% of patients these modalities yield conflicting results [13,14,15]. In our previous work, we showed that PET/CSF discordance also inflicts patient prognosis and thus has potential clinical consequences [16]. This discordance may include valuable information on underlying clinical or neuropathological differences [17].

A combination of various patient features has previously been demonstrated to predict amyloid-β positivity based on PET and/or CSF [18, 19]. In particular, a combination of demographic information, APOE ε4 carriership, neuropsychological tests, and magnetic resonance imaging (MRI) measures was effective in predicting amyloid-β status [20]. Additionally, CSF tau and p-tau have been shown to be predictive of amyloid PET status [21]. So far it has not been investigated whether the predictive ability of patient features for amyloid-β pathology differs when detected by PET or by CSF. We hypothesized that if there are significant differences in the predictive patterns of the two modalities, they must convey partially independent information. Additionally, as it has been suggested that CSF might be able to detect amyloid-β depositions earlier [22], it is possible that the relative predictive contribution of a patient feature changes throughout the course of Alzheimer’s disease. Therefore, in this exploratory study, we investigate the unique information provided by the PET-CSF discordant population using the predictive patterns for amyloid PET and CSF in (i) the total patient group and (ii) stratifying by syndrome diagnosis. Exploring this allows us to gain insight in the clinical and neurobiological factors related to discordant results between amyloid-β PET and CSF and ultimately about the underlying neuropathological processes during the disease course of AD.

Methods

Study population

We retrospectively included 777 patients, who had visited our tertiary memory clinic between 2005 and 2017 and had undergone both CSF Aβ42 analysis and amyloid-β PET within 1 year. We excluded nine patients that did not pass PET imaging quality control. Patients were screened according to the standardized protocol of the Amsterdam Dementia Cohort [23, 24]. This includes a clinical and neuropsychological evaluation, APOE genotyping, MR imaging, and lumbar puncture for CSF analysis. Patient diagnosis was determined during a multidisciplinary meeting, according to international guidelines [10, 11, 25,26,27,28,29,30,31,32,33].

Neuropsychological testing

Subjects underwent extensive neuropsychological testing as part of their diagnostic process. Mini-Mental State Examination (MMSE) scores were used to measure global cognition. In addition, five cognitive domains were assessed [34]. We used the visual association test (VAT), total immediate recall, and the Dutch version of the Rey Auditory Verbal Learning test (delayed recall) to assess memory. Language was assessed by VAT naming and category fluency (animals). The Trail-Making Test (TMT) part A, Digit Span forwards, and the Stroop test I and II were used for attention. Executive functioning was assessed by TMT B, Digit Span backwards, Stroop test III, the Frontal Assessment Battery, and the Dutch version of the Controlled Oral Word Association Test (letter fluency). Finally, we assessed visuospatial functioning by Visual Object and Space Perception battery: tests incomplete letters, dot counting, and number location.

For every test, we derived z-scores using the mean and standard deviation values from a group of healthy controls (n = 360) [34]. TMT A, TMT B, and Stroop Test scores were log-transformed to account for the non-normal distribution of the data and multiplied by − 1 so that lower scores would indicate worse performance. In case TMT B was aborted and TMT A was available (n = 132), we estimated the TMT B score using the multiplication of TMT A score with mean TMT B/A score ratio from the respective diagnostic group [35]. Thereafter, based on available tests, we used z-scores to compile a composite score for each of the five cognitive domains.

CSF

CSF was obtained by lumbar puncture between L3/4, L4/5, or L5/S1 intervertebral space, using a 25-gauge needle and a syringe [36]. The samples were collected in polypropylene microtubes and centrifuged at 1800g for 10 min at 4 °C. Thereafter, the samples were frozen at − 20 °C until manual analyses of Ab42, tau, and p-tau were performed using sandwich ELISAs [Innotest assays: β-amyloid1-42, tTAU-Ag, and PhosphoTAU-181p; Fujirebio (formerly Innogenetics)] at the Neurochemistry Laboratory of the Department of Clinical Chemistry of VUmc. As the median CSF Aβ42 values of our cohort have been gradually increasing over the years [37], we determined CSF amyloid-β status using Aβ42 values that had been adjusted for the longitudinal upward drift. We used a uniform cut-off of 813 pg/mL to dichotomize CSF data [38].

PET

Amyloid-β PET scanning is not part of standard diagnostic process in the Amsterdam Dementia Cohort. Patients underwent an amyloid-β PET for research purposes in the vast majority [39,40,41,42,43,44] or otherwise in case of a diagnostic dilemma. Amyloid-β PET scans were performed using the following PET scanners: ECAT EXACT HR+ scanner (Siemens Healthcare, Germany) and Gemini TF PET/CT, Ingenuity TF PET-CT and Ingenuity PET/MRI (Philips Medical Systems, the Netherlands). We included PET scans using four different radiotracers: [18F]Florbetaben [39, 44] (n = 322, 42%), [11C]PIB [41,42,43] (n = 271, 35%), [18F]Flutemetamol [45] (n = 151, 20%), and [18F]Florbetapir [40] (n = 24, 3%). PET scans were rated as positive or negative based on visual read by an expert nuclear medicine physician (BvB). PET scans were performed, on average, within 54 (± 75) days of the lumbar puncture.

MRI

The acquisition of MRI scans has been extensively described previously [24]. During the period of 2005 to 2017, the following scanners have been used: Discovery MR750 and Signa HDXT (both GE Medical Systems, USA), Ingenuity TF PET/MR (Philips Medical Systems, The Netherlands), Titan (Toshiba Medical Systems, Japan), and Magnetom Impact and Sonata (Siemens Healthcare, Germany). The MRI protocol included 3D T1-weighted, T2-weighted, fluid-attenuated inversion recovery (FLAIR), gradient-echo T2*, and/or susceptibility-weighted imaging sequences. The scans were visually assessed by a neuroradiologist on three different image planes. Parietal atrophy was rated using the posterior cortical atrophy (PCA) scale [46], medial temporal atrophy using the medial temporal lobe atrophy (MTA) scale [47], and the extent of white matter hyperintensities according to the Fazekas scale [48]. MTA and PCA scores were scored separately for right and left and averaged thereafter. In addition, the scans were assessed for the existence of lacunes and microbleeds.

Patient groups

We stratified the patients based on syndrome diagnosis: subjective cognitive decline (SCD, n = 194 (29%)) [49], mild cognitive impairment (MCI, n = 127 (17%)), and dementia (n = 447 (58%)). Within the dementia group, 309 (69%) patients had the diagnosis of Alzheimer’s disease, 66 (15%) a diagnosis within the frontotemporal dementia spectrum, 22 (5%) dementia with Lewy bodies, 6 (1%) vascular dementia, and 44 (10%) other dementia syndromes. Patient diagnosis was determined without knowledge of PET or CSF status. To reflect the information provided to the models in our analysis, we present patient group characteristics based on the binarized amyloid-β status on PET and CSF: concordantly positive (PET+/CSF+) or negative (PET−/CSF− for amyloid-β pathology, or discordantly positive amyloid-β status based on PET (PET+/CSF−) or CSF (PET/CSF+).

Statistical analysis

Statistical analysis was performed using R software (version 3.4.4) [50]. When presenting our study population by binarized PET/CSF status groups, we compared patient features using chi-squared tests, two samples t tests, Wilcoxon rank-sum tests, and linear regression models with Bonferroni correction for group-wise testing. Cognitive scores were compared while adjusting for age, sex, education, and syndrome diagnosis.

All subsequent analyses were performed in the total patient group as well as in the syndrome diagnosis groups of SCD, MCI, and dementia. We first summarized the relative predictive power of every variable in predicting PET and CSF amyloid-β status using random forest modeling. We performed random forest modeling to (i) get an estimate of the predictive power of variables in a setting, where all variables are present in the model; (ii) compare the importance of variables between models predicting PET and CSF amyloid-β status; and (iii) select patient features for multivariable logistic regression models. As classifier models are affected by missing data, we accounted for missing values using multiple imputations (using the mice library [51] including only the 17 predictor variables later used for analysis; with 25 imputations and 5 iterations) (Additional file 1: Table S1). For each of the imputed dataset, we ran two conditional random forest models (ntree = 1001, mtry = 5) [52, 53], predicting separately PET and CSF status using various patient features associated with Alzheimer’s disease [18,19,20]. As predictors, we selected demographic information (age, sex, education), biomarkers (APOE ε4 positivity, CSF tau, and p-tau), cognitive measures (MMSE; z-scores for memory, language, attention, executive, visuospatial), and MRI scores (MTA, PCA, Fazekas scale, the presence of lacunes and microbleeds). Accuracy, sensitivity, and specificity of the random forest models were evaluated using the mean out-of-bag (OOB) error estimates. Using this method, the performance of every tree in the random forest model is evaluated on the approximately 37% of observations that are not used for its training, allowing a means to train the model and perform analysis in the same dataset [54].

We used the area-under-the-curve (AUC)-based permutation variable importance measure (VIM) to estimate the relative predictive power for every patient feature. This measure was selected because of its higher accuracy in datasets with an unbalanced outcome class [55] and we expected this to be especially helpful in the SCD group with a low prevalence of amyloid-β positivity. The AUC-based permutation variable VIM is calculated as follows:

$$ {\mathrm{VI}}_j^{\left(\mathrm{AUC}\right)}=\frac{1}{\mathrm{ntree}}\ {\sum}_{t=1}^{\mathrm{ntree}}\ \left({\mathrm{AUC}}_{tj}-{\mathrm{AUC}}_{tj}^{\sim}\right) $$

where (1) ntree denotes the number of trees in the forest whose OOB observations include observations from both outcome classes, (2) AUCtj denotes the area under the curve computed in the OOB observations in the selected tree before permuting predictor j, and (3) \( {\mathrm{AUC}}_{tj}^{\sim } \) denotes the area under the curve computed from the OOB observations in tree t after randomly permuting predictor j [55]. As the variable is indirectly dependent on the size of population, these variables cannot be reliably compared between populations of different size. We preferred this VIM measure over several alternative VIM measures, including the Gini impurity criterion (which might show bias when predictors vary in their number of categories or scale of measurement), the error-rate-based permutation mutation (which might falsely identify the importance of highly correlated variables), or error-rate-based conditional permutation (which performs best in balanced datasets, while our dataset is unbalanced) [53, 55, 56].

For the second stage of the analysis, we selected patient features based on their predictive value in the random forest models. Similar to a previous study [20], we included patient features when their median VIM over the 25 random forests models for predicting either PET or CSF was higher than the median VIM of all the features for the patient group. Firstly, using Wilcoxon signed-rank tests for paired data in 1000x bootstrapped samples with replacement, we compared the VIM of every selected patient feature between the parallel random forest models predicting amyloid-β PET and CSF status. Secondly, to determine the unadjusted predictive power of these patient features, we performed bivariate logistic regression models with either PET or CSF positivity as the outcome and the selected patient features as predictors. Thirdly, to investigate the added predictive value of a patient feature to the other amyloid-β modality, we performed multivariable logistic regression models, with either PET or CSF positivity as the outcome and the selected patient feature with the status of the other amyloid-β modality as predictors. For these models, we assumed that if PET and CSF would truly provide equal information about amyloid status, additional patient features should never be significant predictors in these models, as the other amyloid status would already provide sufficient predictive power. However, if a patient feature added significant information, this would show a stronger association between the feature and the predicted amyloid-β modality.

Finally, as confirmation for our main findings for APOE ε4 positivity, CSF tau, and p-tau, we compared these multivariable logistic regression models to a univariate logistic regression model, where PET or CSF status was predicted only by the status of the other amyloid modality. We calculated the difference in Akaike Information Criterion (AIC) between the two models to investigate the change in model fit. A decrease in AIC between models can be interpreted as some (0–2), considerable (4–7), or strong (> 10) evidence for gain in model fit in favor of the second model [57].

We calculated the odds ratios (OR) with corresponding 95% confidence intervals for every patient feature both in the original dataset and in the 25× imputed datasets. Non-overlapping confidence intervals were considered significantly different. We used the false discovery rate (FDR) correction with a significance level of 0.05 to account for multiple testing [58].

Results

PET/CSF discordance

In total, 32 patients (4%) were discordantly amyloid-β positive based on PET and 65 (8%) based on CSF. The proportion of PET/CSF discordance was 15% in SCD (n = 30), 13% in MCI (n = 17), and 11% in dementia (n = 50). Of the discordant group, 67% (n = 20/30) of SCD, 53% (n = 9/17) of MCI, and 72% (n = 36/50) of dementia were PET−CSF+.

Overview of features

Patient characteristics grouped by PET/CSF status are summarized in Table 1 and CSF Aβ42 levels shown in Fig. 1. In general, the PET+CSF+ group showed a higher proportion of APOE ε4 carriers, more AD-like CSF markers, MRI features, and lower cognitive scores compared to PET−CSF− group. CSF tau and p-tau were lower in both PET−CSF− and PET−CSF+ groups, compared to PET+CSF− and PET+CSF+. The PET−CSF− group contained a lower proportion of APOE ε4 carriers and better cognitive scores than patients in the discordant groups.

Table 1 Patient groups by PET/CSF amyloid status
Full size table
Fig. 1
figure 1

CSF Aβ42 values by PET/CSF amyloid status groups in SCD, MCI, and dementia. The horizontal line indicates the cut-off of 813 pg/mL used for dichotomization of CSF-amyloid

Full size image

Patient feature selection

Out-of-bag accuracy, sensitivity, and specificity rates for the random forest models are reported in Additional file 1: Table S2.

VIM values over the 25 random forest models (one with each set of imputed data) for the total group are shown in Fig. 2a. APOE ε4 positivity was the most important predictor for amyloid-β positivity in the total patient group for both PET and CSF. CSF tau was similarly important when predicting PET or CSF, but CSF p-tau was a more important predictor for PET compared to CSF. Subsequently, we stratified for syndrome diagnosis (Fig. 2b–d). In SCD, APOE ε4 positivity was a stronger predictor for CSF than PET, whereas CSF p-tau was more associated with PET than CSF amyloid-β status. Additionally, MMSE and memory score had a stronger association with CSF than PET. CSF tau was equally important for predicting PET or CSF amyloid-β status. In contrast to the findings in SCD, in MCI, APOE ε4 carriership was a stronger predictor for PET than for CSF. Moreover, CSF tau and p-tau were more important for predicting PET than for CSF amyloid-β status. In dementia, CSF p-tau was more predictive of PET than CSF, but CSF tau was a stronger predictor for CSF than for PET amyloid-β status. Both PET and CSF had a strong association to APOE ε4 carriership. Finally, visuospatial and memory scores were more important for predicting PET positivity.

Fig. 2
figure 2

a–d Relative predictive power of patient features for amyloid PET and CSF status. AUC-based variable importance (VIM) from 25 random forest models predicting PET status and 25 models from predicting CSF status are plotted. p values (***p < 0.001, **p < 0.01, *p < 0.05, ns non-significant) indicate the bootstrapped difference of VIM values between models predicting PET and CSF status using Wilcoxon signed-rank tests

Full size image

Additionally, in a subanalysis in the total patient group excluding patients with concordantly negative amyloid status and MCI/dementia, CSF p-tau was the most important predictor for PET but not for CSF (n = 589, Additional file 1: Figure S1).

Univariate logistic regression models

We verified the predictive ability of the selected patient features with bivariate logistic regression models for PET and CSF status (Table 2; all possible models in Additional file 1: Table S3). The bivariate models largely confirmed the feature selection of the random forest procedure, as APOE ε4, CSF tau, and CSF p-tau were consistently significant predictors in all groups. In the total group and dementia, most of the patient features selected based on the random forest models were significant predictors.

Table 2 Predictive value of patient features for amyloid status based on PET or CSF
Full size table

Amyloid-adjusted multivariable logistic regression models

We investigated the added predictive value of the selected patient features to the other amyloid-β modality with multivariable logistic regression models (odds ratios and p values are shown in Table 3; all possible models in Additional file 1: Table S4). In the total group, increased levels of CSF p-tau and were more strongly associated with PET than CSF. In SCD, increased levels of CSF p-tau and tau were predictive of only PET, but not CSF positivity. APOE ε4 carriership and lower MMSE scores showed a predictive trend towards amyloid-β status based on CSF, but not on PET. In MCI, a positive PET scan was more strongly predicted by APOE ε4 and by increased levels of CSF p-tau and tau. Finally, in dementia, PET status had a stronger association with increased levels of CSF p-tau and tau and with a worse performance in memory and visuospatial ability than CSF amyloid-β status. APOE ε4 carriership was similarly associated with both PET and CSF. No patient feature showed a higher association with CSF in dementia.

Table 3 Amyloid-adjusted predictive value of patient features for amyloid status based on PET or CSF
Full size table

AIC change between multivariable and univariate models including amyloid status only

Multivariable logistic regression models including APOE ε4 carriership, CSF tau, and CSF p-tau as predictors usually showed significant (> 2) decrease of AIC compared to univariate logistic regression models, where PET or CSF status was predicted only by the status of the other amyloid modality (Table 4). Overall, differences between change of AIC when predicting PET or CSF were similar to findings from previous random forest and multivariate logistic regression models, indicating consistent results across multiple statistical approaches.

Table 4 Information gain of multivariable logistic regression models compared to univariate logistic regression including only amyloid modalities
Full size table

Discussion

We investigated the predictive patterns of various patient features for amyloid-β status based on PET or CSF to determine (i) whether these features have a different association with PET or CSF and (ii) whether this differs per disease stage. We found significant differences in the predictive strength of patient features for amyloid-β status based on PET or CSF. For example, CSF tau and especially CSF p-tau consistently showed a stronger association with amyloid-β status on PET. Additionally, the differential predictive pattern was influenced by the extent of cognitive impairment, as CSF tau was more important in SCD and MCI, while CSF p-tau became more important in the stage of dementia. Moreover, APOE ε4 carriership was more predictive towards CSF status in SCD, whereas it was more predictive towards PET in MCI. These findings suggest that PET and CSF do not provide identical information about the stage of Alzheimer’s disease.

The idea to study differences in the predictive strength of patient features for PET/CSF amyloid-β status was based on the differences in characteristics of patients with discordant amyloid-β biomarkers, which have been theorized to be caused by various factors. Possible explanations for the discordance include individual variances in CSF Aβ42 production [59], the composition of amyloid-β plaques [60], differences in the structure of Aβ fibrils [61], or a variety of technical issues [62, 63], including the variability in cut-off values for CSF Aβ42 [14]. It has also been proposed that in the earliest stages of amyloid-β accumulation CSF Aβ42 analysis might be more sensitive, as the decrease in the concentration of soluble isoforms might precede fibrillar amyloid-β plaque deposition detectable by PET [22]. Overall, we found significant differences in the relation between amyloid PET and CSF status and other biological variables, such as APOE genotype and (p)tau concentrations. The existence of differing predictive patterns between the two modalities implies that PET/CSF discordance may not only be explained by technical variation, but reflect differences in biological substrate between the modalities. In our previous work, we already showed that PET/CSF discordance has potential clinical consequences [16]. These results could also have an effect for future practice in AD research as well as patient care, as the two modalities are currently used as equal alternatives [2, 11].

Our main finding was that CSF p-tau and tau had a stronger association to amyloid-β based on PET compared to CSF. If we assume that CSF is a more sensitive modality for amyloid-β pathology, then the weaker association with tau could be explained by CSF Aβ42 capturing an earlier stage amyloid-β preceding tau pathology. This was reflected by the predictive patterns in the multivariable logistic regression models: when predicting PET status by CSF status, CSF (p)tau adds information about the added burden of disease (including advancing from CSF+PET− to CSF+PET+). When predicting CSF amyloid-β positivity, however, the existence of amyloid-β pathology on PET already provides sufficient predictive power, of subjects already having reached a later stage in amyloid deposition. Overall, although the exact cause of this finding remains unclear, it supports the notion that PET detects more advanced stages of AD pathology, being in accordance with previous work by others [64]. Although CSF tau and p-tau have been shown to be highly correlated [65], the results of the random forest models imply that CSF tau is more predictive towards amyloid-β pathology in SCD and MCI, whereas CSF p-tau is more predictive in dementia. This finding might be caused by wider neuronal death preceding the release of phosphorylated tau, although previous work seems to suggest that levels of CSF p-tau decrease in the later stages of AD [66,67,68]. Another possible explanation is that this finding is caused by the greater specificity of p-tau for AD pathology [69], as our cohort also included amyloid-positive patients diagnosed with non-AD dementia, likely due to secondary amyloid pathology.

Although we focus on the relative differences between PET and CSF, it should be emphasized that in the majority of cases these two modalities contain similar information. This was demonstrated by our finding that many of the selected patient features had similarly some predictive power for amyloid-β pathology for both PET and CSF. Of them, the biological factors APOE ε4 carriership, CSF tau, and p-tau were most consistent in having significant predictive ability amyloid-β status irrespective of the modality. These findings are not unexpected, as APOE ε4 carriership [18, 70, 71] and tau pathology [2, 72] are widely known to have a strong connection to amyloid-β pathology in Alzheimer’s disease. Cognitive measures and MRI visual reads showed overall a smaller predictive value towards amyloid-β status, being in concordance with the theory that they show changes downstream of amyloid and tau pathology [73].

The main strength of our study is the large number of patients with both amyloid-β modalities from a well-characterized cohort. Nevertheless, there were still a limited number of patients with discordant amyloid status, which could influence the reliability of our findings, especially when performing subgroup analysis. Another limitation is that due to the stratification by syndrome diagnoses, the outcome of amyloid-β positivity was not equally prevalent. Our results in the multivariable logistic regression models might be influenced by the high concordance rate between PET and CSF status, although the results are supported by similar findings in the random forest models and by the decrease in AIC compared to models using only the other amyloid modality as predictors. Additionally, the included patients underwent amyloid-β PET scans with four different radiotracers, allowing for variability in thresholds for amyloid-β positivity. However, this effect is likely reduced by all of the PET scans being visually rated by the same experienced nuclear medicine physician. As continuous measures for PET imaging were not available, we dichotomized CSF Aβ42 values, causing some loss of information, which could influence our results. Finally, this patient group did not have CSF Aβ40 values available, which have been shown to correct for the individual variation in the production of amyloid-β [74, 75].

Our findings can be summarized by a hypothetical model highlighting the relative predictive power of patient features towards amyloid-β status based on PET and CSF (Fig. 3). This model supports previous work, suggesting that CSF might be more sensitive in the early stages of amyloid-β pathology, whereas PET status might be more specific to later stages of amyloid-β accumulation. Although the modalities show similar information in the majority of cases, this could have implications for future research and clinical trials. For example, if aiming to capture the earliest stage of amyloid-β pathology, CSF might be preferred over PET. On the contrary, if high confidence of significant amyloid-β pathology is required, PET could be the modality of choice. Future work in other patient cohorts with a higher number of discordant PET/CSF cases is necessary to replicate these findings.

Fig. 3
figure 3

Hypothetical model for relative predictive strength of patient features towards PET and CSF amyloid status. The line location on the y-axis indicates the relative strength of the association between the patient feature and status of the amyloid-β modality. The line thickness indicates the overall predictive strength of the patient feature for amyloid status based on both PET and CSF

Full size image

Conclusion

In this exploratory work, we demonstrated that although various patient features have general predictive value towards amyloid-β status, there are finer differences revealed by discordant cases between the predictive pattern for amyloid-β status based on PET and CSF. This indicates that PET-CSF discordance might include valuable information on underlying clinical and neuropathological differences.