Background

Schizophrenia is among the 25 leading causes of years lived with disability in the age group 25–49 years, according to the 2019 Global Burden of Disease Study [1]. Besides the well-known positive and negative symptoms, cognitive deficits are another independent core part of schizophrenia. Areas of impairment include memory, attention, processing speed, and executive function, as well as higher cognitive functions such as reasoning and social cognition. These deficits not only affect a large proportion of patients [2, 3]—often, the cognitive symptoms precede the onset of positive symptoms for several years and persist long after the acute positive symptoms have subsided [2, 4]. It has been shown that cognitive dysfunction is a strong, discrete predictor of functional outcomes and quality of life in schizophrenic patients [5]. Apart from negative symptoms, the cognitive deficits present in many patients with schizophrenia pose the main obstacle for keeping up employment and social life alike [6].

While antipsychotic drugs constitute a central part of the treatment of schizophrenia in acute phases and relapse prevention [7,8,9], we are only beginning to understand which substance works best for which patient and for which facet of their symptoms. Antipsychotics can differ in several aspects, including in their mechanism of action, receptor-binding profile, efficacy, and side effects. While doctors are often able to control the positive symptoms relatively quickly with antipsychotic drugs, negative and cognitive symptoms are often not as straightforward to address. Evidence on the differential effect of individual antipsychotic substances is available but inconclusive. It has been claimed that the newer, traditionally called “second-generation” antipsychotics are superior to older, “first-generation” antipsychotics in terms of cognitive function [10].

There are multiple randomized controlled trials on this issue as well as a few meta-analyses on the comparative effects of antipsychotic drugs on cognition, but none of them incorporates the latest evidence and adheres to the highest quality standards at the same time [10, 11]. Moreover, the only available bigger meta-analysis [11] did not take into account the most recent principles for the assessment of cognitive symptoms in schizophrenia trials as suggested by the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative. The MATRICS initiative set out to improve the treatment of cognitive symptoms in schizophrenia and related disorders, identifying the cognitive domains of interest and developing a comprehensive test battery (MATRICS Consensus Cognitive Battery (MCCB)) for use in clinical trials. The MCCB is now considered the international gold standard for the measurement of cognition in this setting [12,13,14].

Objective

In summary, high-quality meta-analyses on the effects of antipsychotics on cognition are needed, but not available. To overcome this gap, we will conduct a network meta-analysis examining the effects of different antipsychotic drugs on cognitive function in patients with schizophrenia. Network meta-analysis combines the direct evidence observed in clinical trials with indirect evidence, therefore investigating every antipsychotic in comparison with every other antipsychotic and in consequence producing hierarchies between the antipsychotics according to their effect on overall cognitive function and the specific cognitive domains. In doing so, we aim to summarize the available evidence in an accessible and comprehensible form and thus enable patients and providers to make better, evidence-based treatment choices together.

Methods/design

This systematic review and network meta-analysis protocol will follow the PRISMA guidelines, extension for network meta-analysis [15]. The PRISMA-P Checklist can be found in Additional file 1. This protocol has been registered with PROSPERO [16]. We will update the report in PROSPERO with any necessary amendments.

Eligibility criteria

Characteristics of studies

We will include randomized controlled trials comparing one antipsychotic agent to at least one other antipsychotic or placebo for the treatment of schizophrenia or schizophrenia-like psychoses. Study arms using combinations of antipsychotics or combinations with other drugs will be excluded. We will include double-blind, single-blind, and open-label studies with an intervention phase of at least 3 weeks. Cluster-randomized trials will be excluded due to unit of analysis problems and to avoid violating the transitivity assumption [17]. In cross-over studies, we will only use the first intervention phase to avoid carry-over effects.

Characteristics of participants

We will include studies in which at least 80% of the patient sample has a primary diagnosis of schizophrenia or schizophrenia-like psychosis. We will include trials irrespective of the diagnostic criteria used. Studies in which all patients by inclusion criteria had a comorbid medical or psychiatric illness (including comorbid substance abuse) will be excluded, as these comorbidities or their treatment can have an influence on cognition and could bias the results. There will not be any restrictions in terms of gender, ethnicity, age, phase or severity of illness (first episode, acute episode of chronic illness, stable phase, treatment-resistant, predominant or prominent negative symptoms), or setting (in- and outpatients), but some of these factors will be examined in subgroup and meta-regression analyses.

Interventions

We will include a wide range of antipsychotic agents in oral or injectable form with the exception of short-acting injectables, because these are generally only used for emergency treatment. Our choice of included first-generation antipsychotic drugs is based on a systematic survey of schizophrenia experts [18]. The newer antipsychotic drugs are nowadays the most prescribed medication for schizophrenia and are therefore an obvious choice for inclusion. We included all “second generation” drugs currently available in Europe or the USA. Fixed-dose studies will only be included if the doses are within the range of the International Consensus Study on Antipsychotic Dosing [19]. This restriction will not apply to certain participant subgroups (children, elderly, first episode, or treatment-resistant patients) where different dose ranges seem to be more adequate. We will include all flexible-dose studies, as these allow the investigators to titrate to the adequate dose for the individual patient.

Outcome measures

We will exclusively consider studies that use validated psychometric tests to assess participants’ performance in at least one of the seven domains determined by the MATRICS initiative or that provide an adequate composite score for all domains. These seven domains, identified as particularly important for people with schizophrenia, are speed of processing, attention/vigilance, working memory, verbal learning, visual learning, reasoning and problem solving, and social cognition [20].

As the MATRICS consensus was developed relatively recently (the main publications came out in 2008), and has since also not been adopted in every study on cognition in schizophrenia, we will include tests outside the MCCB, based on their equivalence to the original MCCB tests for the composite score or one of the seven domains. The selection of these tests and the assignment to the respective domain will be made independently by two reviewers and will be supervised by a neuropsychological expert.

If multiple tests for one domain are reported, we will select the most appropriate of them following the principles outlined in this paragraph. We decided to avoid summarizing different tests for the same domain, as this could cause psychometric problems that can affect the validity of the combined scores and their comparability with the results of single test scores [21]. Exceptions will be made for congregate domain scores as defined by MATRICS for the domains speed of processing and working memory, which require multiple tests in the MCCB as well. We will prioritize these over the single MATRICS tests. Other congregate scores per domain or for overall cognition will be judged on a case-by-case basis and will only be considered if they are part of an established neuropsychological test battery.

For determining the most appropriate test within the domain, we will prioritize tests from the final MCCB over tests from the MATRICS beta battery over other tests. In case of multiple MATRICS tests (MCCB or beta battery) for one domain, we will choose the one with the highest intra-class correlation and the highest accordance with the quality criteria used in the development of the MCCB [14]; in case of multiple non-MATRICS tests, we will choose the one with the highest similarity to the MCCB. All decisions regarding the measurement hierarchy will be made independently by two experts in psychometry (RRE, RSEK); in case of disagreement, we will resolve it by discussion.

Primary outcome

Our primary outcome will be the composite score for the seven cognitive domains, preferably based on the MCCB [12,13,14]. Other composite scores of overall cognition reported in the trials will be evaluated on a case-by-case basis.

Secondary outcomes

Our secondary outcomes will include measures of one of the seven cognitive domains, as well as rating scales for functioning or quality of life (QoL). MATRICS recommends a selection of co-primary measures to assess functioning but does not include more direct measures. Hence, we will use them only if the suggested tests are not available [12], examples include the Global Assessment of Functioning Scale (GAF), the Social and Occupational Functioning Assessment Scale (SOFAS), or the Personal and Social Performance Scale (PSP). For QoL scales, MATRICS provides no recommendations. In case of multiple tests on the quality of life, we will include quality criteria and how well-known the test is, for our selection. We will follow especially the recommendations of the COSMIN initiative, that aims to improve the selection of health measurement instruments based on their psychometric properties [22]. They propose the following superior domains to evaluate the quality of measurement, if available: reliability, validity, responsiveness, and interpretability [22]. These criteria should help us in our selection process in case of multiple quality of life scales (Table 1).

Table 1 MCCB tests and co-primary measurements of functioning
Full size table

Search strategy

We have searched the Cochrane Schizophrenia Group Registry of Trials [23] for published and unpublished reports of randomized controlled studies relevant to our research question up to April 27, 2020 (see also Additional file 2) and are planning to conduct an update search. The detailed search strategy for the Cochrane Schizophrenia Group Trials Registry can be found in Additional file 3 [24]. No date or language restrictions will be applied. In addition, we will search the reference lists of previous reviews on the effect of antipsychotic drugs on cognitive function. In case of missing outcome information from included studies, we will try to retrieve it from the corresponding author or responsible drug company.

Identification and selection of studies

Two reviewers from our team will independently screen the search results for general inclusion criteria in Citavi [25] and in a second step (LF, NP) for the availability of cognition measurements. Disagreement will be resolved by discussion or consulting a third, experienced reviewer (SL/JST). As the third step, the measurement of cognition is considered in more detail and references with only inappropriate tests will be excluded (e.g., self-reported or interviewer-assessed cognitive impairment, not separately validated modifications of established tests). Disagreement in the evaluation of appropriate measurements will be resolved by discussion with neuropsychological experts (RRE, RSEK).

Data extraction

The two reviewers (LF, NP) will extract data from all selected trials in a Microsoft Access database. Our database is specifically developed for studies on schizophrenia and allows a standardized process. The software will automatically detect discrepancies between the two reviewers. When disagreement arises, we will resolve it by discussion and, if needed, by involving a third, senior reviewer. Information on the following points will be extracted:

  • General study information (e.g., author name, year, treatment arms)

  • Information on methodology (e.g., duration, blinding, diagnostic criteria used)

  • Characteristics of the study participants (e.g., subgroup, age, number of men/women, race/ethnicity)

  • Characteristics of the used antipsychotics (e.g., doses)

  • Outcome measures

Measurement of treatment effect

We will use the mean differences for the same scales and standardized mean differences (SMD) for different scales of the same outcome parameter. We will prefer the results obtained with imputation methods to handle missing data over completers’ data and results from mixed models of repeated measurement (MMRM) or multiple imputations over last observation carried forward (LOCF). We will extract standard deviations (SD) as a measure of imprecision, and standard errors (SEs) will be converted to SDs. If both are missing, we will estimate SDs from confidence intervals, t-values, or p-values as described in the Cochrane Handbook for Systematic Reviews [26]. If none of the options is viable, we will contact the original study authors. In case of no information on SDs, we will use the SD from another study using the same test. If there are many studies using the same test, we will build a weighted mean of the given SDs.

Risk of bias assessment

Two reviewers will independently assess the risk of bias using the Cochrane Risk of Bias tool version 2 (RoB 2.0) [27]. We will assess the risk of bias in respect of our primary outcome. In case of disagreement between the two reviewers, we will discuss it with a third, senior reviewer. We will use the RoB-MEN framework to evaluate the risk of bias due to missing evidence in network meta-analysis [28].

Data analysis

Conventional pairwise meta-analyses

We will use a random effects frequentist model for our pairwise meta-analyses.

Assessment of heterogeneity

We will measure heterogeneity with tau-squared (the between-study variance). The heterogeneity variance will be assumed common across the various treatment comparisons, and the empirical distributions will be used to characterize the amount of heterogeneity as low, moderate, or high using the first and third quantiles [29, 30]. We will explore the potential reasons for heterogeneity by subgroup and meta-regression analyses.

Network meta-analysis

Network meta-analysis combines direct and indirect evidence for all relative treatment effects and can therefore provide estimates with maximum power and increased precision [31]. We will depict all available direct comparisons in network plots for each outcome. We will test the transitivity assumption by investigating if clinical and methodological variables—which can act as effect moderators—are similarly distributed across studies grouped by comparison.

We will perform a statistical evaluation of the assumption of transitivity (often termed consistency) using the design-by-treatment interaction test that evaluates inconsistency from all possible sources in the network jointly [32], as well as by the SIDE test (separating indirect evidence from direct evidence) [33], assessing the agreement of indirect and direct evidence for every possible comparison in the network. In case of evidence of inconsistency or intransitivity, we will investigate possible sources (mistakes in data entry, differences in study characteristics). Small or moderate amounts of inconsistency will be further explored by network meta-regression and subgroup analyses using the potential effect modifiers listed below. We will estimate the probability for each intervention to be ranked at each possible place, given the relative effect sizes as estimated in NMA. We will present the results for all cognition outcomes ranking the interventions by their P-Score [34].

If the requirements for network meta-analysis are not met, we will present only the findings of the pairwise syntheses.

Subgroup analysis and meta-regression

We plan to investigate the impact of potential effect modifiers via network meta-regressions or subgroup analyses. We will examine the following characteristics: baseline severity of symptoms, inclusion of acutely ill or stable patients, age, co-medication with anticholinergics, co-medication with benzodiazepines, and dose of antipsychotics. We will examine the confidence intervals of the regression coefficients and compare the heterogeneity and inconsistency of the unadjusted and adjusted (network meta-regression) models to infer about the impact of the effect modifiers. Results will be interpreted with caution, given the observational nature of the examined associations.

Sensitivity analyses

We plan to conduct the following sensitivity analyses: excluding (a) unblinded (open-label) studies, (b) overall high risk of bias studies, (c) studies that did not use operationalized criteria for diagnosis, (d) studies with a duration shorter than 12 weeks, and (e) studies with specific patient characteristics (e.g., treatment-resistant, predominant negative symptoms).

Small study effects and reporting bias

We will explore the association between study size and effect size with a comparison-adjusted funnel plot [35,36,37]. Comparisons with more than 10 studies will be plotted in a contour-enhanced funnel plot. The possibility of reporting bias in the entire network will be assessed using the RoB-MEN framework [28].

Evaluating the confidence in estimates

The confidence in estimates of the main outcome will be evaluated with the framework Confidence in Network Meta-Analysis (CINeMA) [38], an adaptation of the Grading of Recommendations Assessment, Development, and Evaluation framework (GRADE) specifically developed for NMA. Within this framework, we will create tables with Summary of Confidence for the primary outcome “cognition composite score” and for the seven cognitive domains [38].

Statistical software

We will perform all analyses using the frequentist software R [39] (packages meta and netmeta [40, 41]). Network meta-regression will be performed in a Bayesian framework using self-programmed routines in JAGS [42].

Discussion

This network meta-analysis will examine the effect of antipsychotic drugs on cognitive function in individuals with schizophrenia. Considering the large number of available substances, conventional pairwise meta-analysis is not capable of providing a sufficient overview. The analysis will benefit from maximal statistical power by combining direct and indirect comparisons in NMA, measuring the relative effects of the different antipsychotics on cognition. We will derive evidence-based hierarchies showing which antipsychotic has the largest effect in each cognitive domain.

Network meta-analysis currently presents the most advanced way to summarize evidence from multiple (in theory interchangeable) treatments, the meaningfulness of the obtained results is highly dependent on the quality of included studies. We expect high heterogeneity in the reporting of cognition measurements and a low percentage of studies that follow the actual MATRICS recommendations, given that we plan to include studies without restriction in terms of publication date. Including studies conducted over such a long timeframe, with differing methodologies and different patient populations such as children, individuals with treatment-resistant schizophrenia, or first-episode patients, poses a challenge to the assumption of transitivity needed for network meta-analysis. While we do consider it meaningful to include a variety of different study settings and participants, we will meticulously explore the sources of heterogeneity in the network. We will also be very strict in selecting the eligible test measures and matching them to the MATRICS domains to make the obtained values more comparable.

While addressing cognitive dysfunction in schizophrenia should not be restricted to antipsychotic drug choice, and cognitive remediation programs did show some effect on the cognitive symptoms of schizophrenia [43], we think it is of utmost importance to evaluate the differences between the antipsychotic substances in their effect on cognitive functioning—not only to inform drug choice, but also to identify possible links between the mechanism of action and effect on cognition of the various substances based on their receptor profiles.