Psychostimulant medications, such as methylphenidate and amphetamine, are commonly prescribed for attention-deficit/hyperactivity disorder (ADHD) and have been a first-line medication for this disorder for over 60 years. However, a recent Cochrane review has raised doubts about the benefit of methylphenidate in the treatment of childhood ADHD (Storebø et al. 2015). Although the authors demonstrated that methylphenidate was associated with improved teacher-rated ADHD symptoms (standardized mean difference [SMD], −0.77 [95% CI −0.90 to −0.64] in 19 parallel group trials and first phase of crossover trials which included a total of 1698 subjects, the authors concluded that the true magnitude and relevance of this effect is uncertain because of the “very low quality” of most of the included studies. They assessed that 96.8% of all 185 trials included in the broader review were at high risk of bias trials according to their interpretation of the evidence. This assumption contrasts with previously published systematic reviews and meta-analyses, including the comprehensive review of the National Institute for Health and Care Excellence (for example, Faraone and Buitelaar 2010; King et al. 2006; National Institute of Health and Clinical Excellence 2008; van der Oord et al. 2008), which all judged study quality more favourably. Interestingly, in the Cochrane review 13 out of the 19 studies were graded as having a high risk of bias due to a vested interest bias. This domain of bias was introduced by the authors on top of and in addition to the standard Cochrane bias domains. It describes a trial as of high risk of bias when the trial was either funded by parties that might have had a conflict of interest (such as a manufacturer of methylphenidate) or where there were potential conflicts of interest among authors, for example, because they had been received remuneration from companies producing or selling ADHD medications.
This Cochrane review led to a fierce discussion and several commentaries that have questioned whether inaccuracies, inconsistencies and misinterpretations of trial data included in the review lead to false results and inadequate conclusions (Banaschewski et al. 2016a, b; Hoekstra and Buitelaar 2016a, b; Romanos et al. 2016a, b; Shaw 2016). Here, we focus on several issues that have not been addressed adequately by the authors of the Cochrane review.
Incorrect selection of studies for estimation of the effect size
The effect size estimate in the Cochrane review was based on 19 of 185 included studies. However, four studies did not compare methylphenidate with placebo but with a co-treatment (i.e. behavioural therapy or clonidine) in both the placebo or non-treatment arm as well as the active methylphenidate arm. This will almost certainly have resulted in a reduction in the reported effect sizes compared to what would be expected with a placebo comparison. Even more surprising, the multimodal treatment of ADHD study was included in the meta-analyses even though it lacked a placebo-arm or non-treatment condition and instead compared medication to treatment as usual (with the majority of those in this arm receiving medication). Finally, one of the included studies was conducted in children aged 3–6 years in whom its use is off-label and in an age range where medication treatments are generally agreed not to be first line and to be associated with lower effect sizes. Accordingly, to enable firm conclusions on the efficacy of methylphenidate in children and adolescents, these five studies should have been excluded from the analysis. The effect size based on teacher reports, after excluding these studies, increases from 0.77 (moderate) to 0.89 (high) (Banaschewski et al. 2016a, b). The authors excluded 56 studies that used a crossover design and provided only endpoint data. The rational for exclusion is that carry-over effects may have inflated the reported effect sizes. Whilst this is possible, including these studies had minimal impact on the effect size which with all 75 eligible studies was 0.91 (Banaschewski et al. 2016a, b).
Judgement of risk of bias that is not justified by “The cochrane collaboration’s risk of bias” tool
It is of utmost importance to critically evaluate the findings of trials in the light of potential bias that may have influenced both study design and conduct or the stated conclusions. This should, however, be supported through empirical testing of whether bias has really distorted the analysis of data, reporting of results and drawing conclusions.
The authors considered seven sources of bias: randomization methods, the concealment of allocation, blinding of participants and personnel, blinding of the outcome assessment, incomplete outcome data, selective reporting, and vested interest bias. Studies were judged as having a high risk of bias if any one source of bias was present, in other words, each type of bias was considered of equal importance. Originally, “The Cochrane Collaboration’s risk of bias” tool was developed on the basis of a combination of empirical and theoretical considerations, leading to a focus on risk of bias rather than study “quality” and a division of assessments into six bias domains (Sterne 2013). Including the funding source as a standard item in the tool is an adaptation that has caused particular controversy. Indeed, there is no evidence that the funding source has really distorted the analysis of data. Previous analysis gave indication, that the principle bias associated with industry funding is associated with the interpretation of the results rather than study design and conduct (Sterne 2013; Lundh et al. 2012). This potential bias of interpretation is, however, not relevant when including study data to calculate effect sizes in meta-analyses.
In the Cochrane review, even those remaining seven studies that were categorized as having overall low risk of bias were still described by the authors as probably biased because in their view frequent adverse events such as loss of appetite or trouble falling asleep may have unmasked the intervention status. This assumption which suggests that the authors entered the review with a prior notion that all studies were probably biased which is an important context for the conduct of the review. There is however no empirical evidence presented that supports bias due to possible de-blinding caused by adverse events. In fact, from a practical and clinical perspective school teachers, whose rating were the basis of the effect sizes, are less likely than parents to detect changes in eating or sleeping patterns in school children and attribute these to adverse drug reactions and therefore less likely than parents to be biased in their assessment of the core symptoms of ADHD.
In summary, incorrect inclusion and exclusion of studies and unjustified judgement of risk of bias has led to results that contradict decades of evidence from randomized clinical trials. Since long-term randomized trials are impossible and unavailable due to ethical limitations, recent analyses from registries provide the most convincing evidence on the long-term benefits of methylphenidate. These studies indicate, for example, that treatment with methylphenidate has beneficial effects in terms of important distal outcomes that go beyond symptom control such as a reduction in comorbid depression, substance-use and dependency, trauma-related visits to the emergency room and mortality (Chang et al. 2014, 2016; Dalsgaard et al. 2015; Man et al. 2015). Furthermore, in the absence of clinically relevant effects of non-pharmacological treatment options on the core symptoms of ADHD (Sonuga-Barke et al. 2013), a dogmatically biased devaluation of the currently most effective treatment is neither helpful for science nor patients.
Banaschewski T, Buitelaar J, Chui CS et al (2016a) Methylphenidate for ADHD in children and adolescents: throwing the baby out with the bathwater. Evid Based Ment Health 19:97–99
Banaschewski T, Gerlach M, Becker K et al (2016b) Trust, but verify. The errors and misinterpretations in the Cochrane analysis by O.J. Storebo and colleagues on the efficacy and safety of methylphenidate for the treatment of children and adolescents with ADHD. Z Kinder-Jugendpsychiatr Psychother 44:307–314
Chang Z, D’Onofrio BM, Quinn PD, Lichtenstein P, Larsson H (2016) Medication for attention-deficit/hyperactivity disorder and risk for depression: a Nationwide Longitudinal Cohort Study. Biol Psychiatry 80:916–922
Chang Z, Lichtenstein P, Halldner L et al (2014) Stimulant ADHD medication and risk for substance abuse. J Child Psychol Psychiatry 55:878–885
Dalsgaard S, Østergaard SD, Leckman JF, Mortensen PB, Pedersen MG (2015) Mortality in children, adolescents, and adults with attention deficit hyperactivity disorder: a nationwide cohort study. Lancet 385:2190–2196
Faraone SV, Buitelaar J (2010) Comparing the efficacy of stimulants for ADHD in children and adolescents using meta-analysis. Eur Child Adolesc Psychiatry 19:353–364
Hoekstra PJ, Buitelaar JK (2016a) Is the evidence of methylphenidate for children and adolescents with attention-deficit/hyperactivity disorder flawed? Eur Child Adolesc Psychiatry 25:339–340
Hoekstra PJ, Buitelaar JK (2016b) Response to: the evidence base of methylphenidate for children and adolescents with attention-deficit/hyperactivity disorder is in fact flawed. Eur Child Adolesc Psychiatry 25:1039–1040
King S, Griffin S, Hodges Z et al (2006) A systematic review and economic model of the effectiveness and cost-effectiveness of methylphenidate, dexamfetamine and atomoxetine for the treatment of attention deficit hyperactivity disorder in children and adolescents. Health Technol Assess 10: iii–iv, xiii-146
Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L (2012) Industry sponsorship and research outcome. Cochrane Database Syst Rev. doi:10.1002/14651858.MR000033.pub2
Man KK, Chan EW, Coghill D et al (2015) Methylphenidate and the risk of trauma. Pediatrics 135:40–48
National Institute of Health and Clinical Excellence (2008) Attention deficit hyperactivity disorder: diagnosis and management of ADHD in children, young people and adults. National Institute of Health and Clinical Excellence, London
Romanos M, Goghill D, Gerlach M et al (2016a) Check and double check—the Cochrane review by Storebo et al (2015) is indeed flawed. Z Kinder-Jugendpsychiatr Psychother 44:336–337
Romanos M, Reif A, Banaschewski T (2016b) Methylphenidate for attention-deficit/hyperactivity disorder. JAMA 316:994–995
Shaw P (2016) Quantifying the benefits and risks of methylphenidate as treatment for childhood attention-deficit/hyperactivity disorder. JAMA 315:1953–1955
Sonuga-Barke EJS, Brandeis D, Cortese S et al (2013) Nonpharmacological interventions for ADHD: systematic review and meta-analyses of randomized controlled trials of dietary and psychological treatments. Am J Psychiatry 170:275–289
Sterne J (2013) Why the Cochrane risk of bias tool should not include funding source as a standard item [editorial]. Cochrane Database Syst Rev. doi:10.1002/14651858.ED000076
Storebø OJ, Krogh HB, Ramstad E et al (2015) Methylphenidate for attention-deficit/hyperactivity disorder in children and adolescents: Cochrane systematic review with meta-analysis and trial sequential analysis of randomized clinical trials. Br Med J 351:h5203
Van der Oord S, Prins PJ, Oosterlaan J et al (2008) Efficacy of methylphenidate, psychosocial treatments and their combination in school-aged children with ADHD: a meta-analysis. Clin Psychol Rev 28:783–800
On behalf of the World Federation of ADHD and EUNETHYDIS.
About this article
Cite this article
Gerlach, M., Banaschewski, T., Coghill, D. et al. What are the benefits of methylphenidate as a treatment for children and adolescents with attention-deficit/hyperactivity disorder?. ADHD Atten Def Hyp Disord 9, 1–3 (2017). https://doi.org/10.1007/s12402-017-0220-2