BACKGROUND

On 28 February 2012, the United States Food and Drug Administration (FDA) issued a new warning for the labeling of statin drugs regarding potential adverse effects on cognition,1 based on post-marketing surveillance reports, case reports, observational studies,2–11 and randomized controlled trials (RCTs).12–15 Post-marketing reports (case series of 60 to 171 individuals) have described ill-defined memory impairment, reversible upon statin discontinuation,1 and some observational studies have described adverse cognitive effects that recurred with re-challenge.5,8,16

Other reviewers examining RCT and observational study data reported that there is no conclusive evidence that statins cause or contribute to clinically meaningful cognitive impairment,17–19 and may actually provide a slight benefit in dementia prevention.20,21 A recent systematic review that included RCT data also found no statistically significant effects on cognition, but deemed the evidence to be of low or moderate strength, and called for additional larger and better-designed studies to settle the question.22 The FDA advisory selected four of several pertinent RCTs in its review, without being explicit about how these were selected, and when it weighed such information along with other observational data, the advisory came to different conclusions than other reviewers.

Overall, the use of RCT data in quantitative analyses has been limited: meta-analyses of RCT data were performed in only two reports, which included a total of three RCTs each.21,22 These reviews used narrow definitions of cognitive outcomes and were not designed to detect signals of adverse cognitive effects of statins. Furthermore, prior reviews often omitted short-term studies that could have captured more immediate side effects on cognition, similar to those noted in case reports.5,8 To address these limitations, we performed a systematic review and meta-analysis of cognitive test results and adverse event reports from RCTs of statin treatment in cognitively healthy and cognitively impaired individuals in the short as well as the long term.

METHODS

Study Aims

We aimed to synthesize current evidence on causal associations between statin therapy and impaired cognition from RCTs. Secondary goals were to determine whether cognitively impaired individuals are particularly vulnerable to any adverse effects on cognition, and whether blood–brain barrier penetrability differentiated among statins as to which were more likely to be associated with adverse cognitive effects.

The search, study selection, and analytic methods of this review were pre-specified in the review protocol provided in Appendix 1, available online. We focused our systematic review on RCTs of statins, because it is more straightforward to make causal claims based on randomized designs.

Data Sources and Searches

We used four complementary approaches to identify relevant studies (see Fig. 1). First, we recorded all statin RCTs in the inclusion and exclusion lists of three Cochrane reviews. Two examined the effectiveness of statins for the prevention23 and treatment of dementia,24 covering literature through 2007 and 2008, respectively, and one examined primary prevention of cardiovascular outcomes,25 covering literature through 2007. Second, we conducted electronic searches for RCTs of statins crossed with search terms for neurocognitive outcomes through 2008, designed to identify additional RCTs that the Cochrane reviews might have excluded at the abstract level. Third, we used a sensitive search strategy to identify RCTs of statins, irrespective of reported outcomes, between 2008 and December 2012. Finally, we reviewed the full text of all studies included in a recent large network meta-analysis of statins (and their adverse effects),26 and perused the reference lists of RCTs identified through our other approaches.

Figure 1
figure 1

PRISMA flow diagram for literature search and article selection.

Full size image

Our electronic database searches covered MEDLINE, EMBASE, and the Cochrane Central registry of trials. The complete search strategies are provided in Appendix 2, available online. We included only articles published in English during full text review. To standardize our application of the screening criteria, all authors completed a pilot round of the same 300 abstracts using Abstrackr software.27 After consensus agreement was established, we continued with single screening of abstracts. All articles selected in the abstract screening phase were retrieved and examined in full text for eligibility.

Study Selection

We sought to identify RCTs that compared statin therapy with or without other lipid-lowering agents versus no statins (standard therapy, no therapy, or placebo), and reported cognitive outcomes in individuals with a baseline diagnosis of normal cognition, or people with abnormal cognition (e.g., Alzheimer’s disease, traumatic brain injury, neurofibromatosis). We considered only statins approved for use in the United Sates or Europe (Table 1). We included RCTs with a mean number of participants per arm greater than ten, irrespective of follow-up duration, because we were also interested in short-term and transient effects. We considered the following outcomes: dichotomous classifications of cognitive impairment (e.g., dementia vs. no dementia); cognition measured by validated scales; and test performance in the cognitive domains of executive function, attention, processing speed, memory, working memory, or global metrics combining the above. We also recorded information on cognition-related adverse events, including memory loss, forgetfulness, amnesia, memory impairment, and confusion.

Table 1. Statin Medications Included in the Systematic Review
Full size table

Data Extraction

We abstracted the following information from all eligible studies: participant characteristics, study characteristics including objectives, year of publication, sample size, setting, country, funding mechanism, duration of follow-up, randomization method, reporting of dropouts, intervention and comparator details, and cognitive outcomes and harms. For categorical outcomes, we extracted the number experiencing the outcome in each arm. For continuous outcomes, we extracted test scores to calculate the mean differences at end of follow-up. We used differences in net changes (i.e., differences in the final minus baseline values between groups) only when mean differences of final values could not be calculated.28 Non-numerical data were extracted by a single reviewer, and checked by another. Reviewers experienced in meta-analysis extracted numerical data in duplicate. The analysis data set was uploaded on the Systematic Review Data Repository (http://ahrq-srdr-prod-347362009.us-east-1.elb.amazonaws.com/); a list of cognitive tests used in the included studies is provided in Appendix 3.

Risk of Bias of Individual Studies

We examined the following methodological items: adequacy of random sequence generation, allocation concealment, blinding of participants, personnel, and outcome assessors to treatment assignment, completeness of follow-up data (defined as loss to follow-up and treatment discontinuation rates lower than 20 %), and lack of differential loss to follow-up or discontinuation (defined as Fisher’s exact p value > 0.05 or difference in absolute rates < 5 %, across study groups). These correspond to the domains examined by the Cochrane Risk of Bias tool.29 Two independent reviewers rated risk of bias as high, low, or unclear. Discrepancies were resolved by consensus.

Evidence Synthesis

The results of the systematic review are described narratively. Meta-analyses were performed when information existed from two or more RCTs. Separate meta-analyses were performed for RCTs in cognitively normal subjects, and in patients with Alzheimer’s disease (AD). No meta-analyses were done for RCTs in patients with other types of cognitive impairment, because of the small number of available studies for each condition (two or fewer).

Continuous outcomes reported on the same scale (e.g., ADAS-Cog,30 MMSE31) were summarized as weighted mean differences at the end of follow-up. Continuous outcomes reported on different scales were summarized as standardized mean differences (SMDs), using the Hedges g metric.32 SMDs express a difference as a fraction of the pooled standard deviation of the measurements, and allow comparisons across measurements with different instruments, provided that studies come from populations that would have comparable variability in the various measurement scales. While this assumption is often hard to assess, it is frequently used in the psychometrics, education, and psychology literature. SMD values of ±0.2 imply small to modest differences.33 To use all available information and avoid double counting (i.e., to properly account for within-study correlation), we performed meta-analyses using the robust variance estimator proposed by Hedges.34 This approach adjusts the variance of the meta-analysis summary to account for model mis-specification.

All meta-analyses were done with a random effects model, because substantial between-study diversity was expected a priori. Models were fit with weighted least-squares methods. We assessed the extent of between-study heterogeneity using the estimated between-study standard deviation (i.e., τ, the square root of the between-study variance). Larger estimated τ values indicate the presence of greater heterogeneity.

Assessment of Heterogeneity and Meta-Regression Analyses

Random effects meta-regression with weighted least squares was used to investigate the contribution of the degree of blood brain barrier penetrability to between-study differences for cognitive effects. For this analysis, statins were divided into two categories, according to higher (simvastatin only, since no studies with fluvastatin are included) vs. lower (all others) propensity to cross the blood–brain barrier.35 Additional analyses were performed for study duration, study sample size, and risk of bias items, when the data allowed (i.e., when a characteristic varied across studies and six or more studies were available for analysis).

RESULTS

Study Characteristics

Figure 1 shows the literature search and article selection flow chart. We screened 5,823 citations and evaluated the full text of 1,827 articles, and included 25 RCTs (Table 2) reported in 33 publications.12–15,36–64 All had placebo rather than standard care comparators. RCTs (reported in 23 publications13–15,36–48,50–52,55,56,62,64) enrolled subjects with normal cognition at baseline, four RCTs (reported in seven publications12,54,57–61) enrolled patients with AD, and three RCTs enrolled other cognitively impaired subjects (traumatic brain injury,53,63 and neurofibromatosis type 149). Simvastatin (eight trials), pravastatin (eight trials), and lovastatin (six trials) were most frequently utilized in the statin RCTs. There were no RCTs that assessed neurocognitive outcomes for fluvastatin and pitavastatin.

Table 2. Included Randomized Controlled Trials and Reports of Adverse Cognitive Outcomes
Full size table

Overall, we found that studies were at low to moderate risk of bias. Procedures for randomized sequence generation and allocation concealment were deemed adequate in only 13 (52 %) and nine (36 %) studies, respectively, of the 25 studies included in the review. In contrast, blinding of participants, personnel, and outcome assessors was adequate in the majority of studies (> 90 % for all three types of blinding). Losses-to-follow-up and dropouts totaled less than 20 % of the enrolled population in 21 (84 %) of the studies. Differential dropout between compared groups was deemed significant or impossible to assess in seven studies (28 %). Study-level risk-of-bias information is presented in Table 3.

Table 3. Assessment of Study Validity
Full size table

Studies of Cognitively Normal Participants

The 18 RCTs of cognitively normal individuals included heterogeneous populations. Twelve RCTs (1,301 patients) enrolled generally healthy participants (there was no index medical condition), while six (44,263 patients) enrolled patients at elevated risk for cardiovascular disease. Nine out of 18 RCTs also included patients aged 65 years or older, with the remainder including younger patients.

Dementia, Confusion, and Other Cognitive Adverse Events

Development of dementia, confusion and other cognitive adverse events was reported in three out of 18 RCTs in cognitively healthy participants. The JUPITER trial [Justification for the Use of Statins in Primary Prevention: An Intervention Trial Evaluating Rosuvastatin] (n = 17,802)42 comparing rosuvastatin vs. placebo reported no statistically significant differences for the incidence of dementia (12 vs. nine patients, respectively), confusion (18 vs. four patients, respectively), or adverse nervous system events (69 vs. 76 patients, respectively). The absolute event rate was less than 1 % for all these outcomes. The HPS trial [Heart Protection Study] (n = 20,536)36 comparing simvastatin versus placebo found no statistically significant differences in the incidence of dementia (31 vs. 31 patients, espectively) or the proportion of patients classified as cognitively impaired by telephone-administered cognitive assessment at end of study (2,433 or 24 % vs. 2,485 or 24 %, respectively). Finally, a phase 1 dose-escalation study of atorvastatin vs. placebo in 22 healthy young adults reported dose-limiting central nervous system adverse effects in one person receiving a solution of 120 mg of atorvastatin, including mild, transient restlessness, euphoria, and mental confusion.50

Cognitive Test Outcomes

Cognitive test outcomes were assessed in 16 RCTs (27,693 patients total). Various tests were employed. “Global” measures included test scores that covered multiple cognitive domains, while “all effects” in Table 4 refers to a calculated overall measure of cognition, combining SMDs across all cognitive domains. Two of the 16 studies found no statistically significant effect of statins on cognitive tests, but did not report adequate data for inclusion in meta-analyses. Meta-analyses of the remaining 14 studies found no statistically significant differences between statin and no statin groups (Table 4 and Fig. 2), for the global, attention, executive, memory, processing speed and working memory domains. Summary effect sizes ranged between −0.35 and 0.10. With the exception of the working memory domain, which was rather imprecise, 95 % CIs excluded small to moderate effect sizes (e.g., effect sizes as large as 0.20 in either direction, and often much smaller, were not within the 95 % CIs of summary estimates). Across all cognitive domains, we found little evidence of an adverse impact of statins on cognitive outcomes: SMD = 0.01; 95 % CI −0.01 to 0.03).

Table 4. Meta-Analysis of Treatment Effect of Statins on Global Cognition and Cognitive Domains According to Subject Characteristics
Full size table
Figure 2
figure 2

Forest plot of treatment effect of statins on global cognition and cognitive domains in cognitively normal subjects.

Full size image

Between-study heterogeneity was low overall and for specific cognitive domains, with the exception of attention and working memory, where heterogeneity was more pronounced. Subgroup analyses according to blood–brain barrier penetrating statins suggested no differences in effect size. Similarly, risk-of-bias items for which meta-regression analyses were possible did not reveal any associations with the estimated treatment effects. Results from regression analyses are shown in Table 5.

Table 5. Meta-Analysis of Treatment Effect (Standardized Mean Difference) of Statins on Cognition by Statin Characteristics in Populations with Normal Cognition at Baseline
Full size table

Cognitively Impaired Populations

The four RCTs in 1,153 patients with an established diagnosis of AD included primarily older adults (mean ages above 68 years). The two largest RCTs that contributed the majority of patients (1,046/1,153 or 91 %) were deemed to have low risk of bias.12,54 Table 3 summarizes study-level assessments for risk of bias. In a random effects meta-analysis, the weighted mean differences for the ADAS-cog and MMSE instruments were higher in the statin compared to no statin arms, suggesting a trend toward benefit, but the 95 % confidence intervals were broad (see Table 4). Based on the confidence interval boundaries, the meta-analysis practically excludes a clinically important difference in ADAS-cog, which is approximately three to four points.65 Although the summary result for the MMSE suggested the presence of a beneficial statin effect (p < 0.05), the association was no longer significant after adjustment for multiple comparisons or when analyses were repeated using net changes (instead of differences in final values).

Finally, the effect of 10 days of statin treatment on global cognitive function in subjects with acute traumatic brain injury was investigated in two small RCTs, from the same research team. One showed no statistically significant cognitive benefit over 6 months,53 and the other showed positive benefit over 4 months.63 A small trial of 62 children with neurofibromatosis type 1, expected to have a high prevalence of learning disability, showed no significant effects for statin treatment.49

DISCUSSION

The FDA warning about potential adverse cognitive effects of statins has important public health implications; therefore, a thorough assessment of the effects of statins should rely on information from all available RCTs. To this end, we conducted meta-analyses designed to detect signals for adverse neurocognitive outcomes. We found no statistically significant effects of statin treatment on cognition. If anything, for cognitive domains such as memory, where the cumulative sample size is substantial, confidence intervals of the summary effect exclude even very small differences (e.g., SMDs of 0.05) in either direction. By contrast, when data was limited, such as for the working memory domain, confidence intervals were wide, and we could not exclude large beneficial or harmful effects. Between-study heterogeneity was higher for attention and working memory than other domains. In terms of an overall signal however, our analyses of SMDs for all cognitive outcomes do not confirm the signal detected from spontaneous adverse event reports. We also sought to explore the impact of study-level characteristics on the association of statin treatment with cognitive outcomes. We found no significant effects or modification of effects within or across neurocognitive domains such as memory or attention, by whether the drug penetrates the blood–brain barrier or not, study duration, sample size, location, or cognitive health status.

Our results suggest that the FDA warning about potential adverse effects of statins on cognition merits re-evaluation. The risks for cognition, if any, are likely outweighed by the beneficial effects of adherence to statin therapy on cardiovascular and cerebrovascular disease.66 In addition, the reasons for the discordance between trial results and reports from the Adverse Events Reporting System (or published case series) should be explored further. One explanation might be that cognitive adverse effects are more likely to occur at high statin dosages. Only five RCTs employed doses at the upper limit of current guidelines, so we cannot exclude possible adverse effects from high dosage. However, three of the four AD trials did use high doses, and significant negative effects were not seen in this particularly vulnerable population. In one study included in this review, a patient experienced confusion after receiving 120 mg of atorvastatin, a dose that is 40 mg higher than the recommended daily dose for treating hyperlipidemia.50 Neurotoxic effects associated with high dose statins have been reported in laboratory animals67–69 and a phase 1 study in humans.70 In the less controlled environment of home use, it is possible that some individuals reporting cognitive adverse events may have unintentionally overdosed. In addition, other factors (grapefruit juice ingestion,71,72 drug interactions,73,74 or constitutional differences in drug metabolism) can result in higher-than-expected statin plasma levels and subsequent adverse effects.

Compared to previous reviews on this topic, our approach is more comprehensive (it includes data from over 20 RCTs), and employs well-studied analysis methods. A previous systematic review summarized evidence for each cognitive test separately, and employed a “vote counting” approach for detecting a signal in RCT data.22 Evaluating each cognitive test in isolation does not allow learning across trials that assess similar cognitive outcomes measured on different scales, and does not facilitate signal detection for adverse events. Furthermore, vote counting also fails to account for the magnitude of observed effects or their precision.

To detect signals of small magnitude, one must maximize the use of the available information. We made methodological choices to address three challenges to using all available information. The first challenge was that the RCTs measured a variety of cognitive outcomes. For example, among the 18 RCTs in people with normal cognition, any instrument was used in three RCTs at most, and, often, in only one. We opted to use standardized effect sizes to synthesize information across distinct instruments measuring the same construct.

The second challenge was that many RCTs used two or more cognitive tests from the same cognitive domain. Such outcomes are correlated, because they are measured in the same patients, and are in some sense “partially redundant.” At the same time, each provides potentially useful information. Instead of arbitrarily choosing one of several tests from a RCT, we synthesized all of them and accounted for their correlations by calculating robust variance estimates.34

The third challenge was integrating results from the various cognitive domains for detecting an overall signal regarding the cognitive effects of statins. To this end, we calculated an omnibus overall measure of cognition across all cognitive domains. This measure does not have a straightforward interpretation, in that it does not stand for a particular outcome or cognitive domain. However, it does address the high level question of whether a “signal” with adequate statistical power exists for potential cognitive effects of statin treatments.

Our approach has limitations. First, publication bias and selective outcome reporting threaten the validity of all meta-analyses. When these biases operate, statistically significant findings are more likely to be reported in full, compared to findings of no difference and statistically nonsignificant results. Second, standardized effect sizes, especially when combining across cognition domains, are difficult to interpret. However, standardized effects may well suffice for detecting the presence or absence of a signal. In our case, the summary for the overall outcome was practically zero, and the confidence intervals were narrow enough to exclude important signals. Third, we interpreted the meta-analysis for overall cognition to suggest that statins do not have important effects on cognition, yet this approach could obscure some domain-specific cognitive effects of statins. However, meta-analyses per cognitive domain were not suggestive of differences by statin treatment, and we are not aware of a postulated pathophysiological mechanism predicting domain-specific statin effects. Fourth, poor reporting in RCTs may explain why we found no evidence for an effect of statins on adverse events.75 However, we drew our main conclusion on the basis of pre-specified, sensitive measurements of cognition. Another possible limitation is that the results of the meta-analysis (although unlikely to differ greatly) apply to the statins for which there were trials with cognitive test outcomes—which would exclude fluvastatin, rosuvastatin, and pitavastatin.

CONCLUSION

Given these results, it is questionable whether the FDA class warning about potential cognitive adverse effects of statins is still warranted. Future post marketing surveillance efforts should focus on critical analysis of effects of re-challenge, as well as factors not addressed in statin clinical trials, such as excessively high dosage and compliance with guidelines. At the present time, our findings, combined with individual information about dosing and compliance, will allow physicians to more effectively counsel patients about their cognitive health concerns. Our findings are congruent with the recent 2013 American College of Cardiology/American Heart Association Cholesterol Guideline safety statement that, “for individuals presenting with a confusional state or memory impairment while on statin therapy, it may be reasonable to evaluate the patient for non-statin causes, such as exposure to other drugs, as well as for systemic and neuropsychiatric causes, in addition to the possibility of adverse effects associated with statin drug therapy.”76