A new approach to outliers in meta-analysis

Health Care Management Science volume 11, pages121–131(2008)Cite this article

Abstract

The synthesis of evidence from trials and medical studies using meta-analysis is essential for Evidence Based Medicine. However, problematical outlying results often occur even under the random-effects model. We propose a model that allows a long-tailed distribution for the random effect, which removes the necessity for an arbitrary decision to include or exclude outliers. In this approach, they are included, but with a reduced weight. We also introduce a modification of the forest plot to show the downweighting of outliers. We illustrate the methodology and its usefulness by carrying out both frequentist and Bayesian meta-analyses using data sets from the Cochrane Collaboration.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. 1.

    Copas J, Jackson D (2004) A bound for publication bias based on the fraction of unpublished studies. Biometrics 60:146–153

    Article  Google Scholar 

  2. 2.

    Baker RD, Jackson D (2006) Using journal impact factors to correct for the publication bias of medical studies. Biometrics 62:785–792

    Article  Google Scholar 

  3. 3.

    Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F (2000) Methods for meta-analysis in medical research. Wiley, Chichester

    Google Scholar 

  4. 4.

    Biggerstaff BJ, Tweedie RL (1997) Incorporating variability of estimates of heterogeneity in the random effects model in meta-analysis. Stat Med 16:753–768

    Article  Google Scholar 

  5. 5.

    Simonian R, Laird N (1986) Meta-analysis in clinical trials. Control Clin Trials 7:177–188

    Article  Google Scholar 

  6. 6.

    Hardy RJ, Thompson SG (1996) A likelihood approach to meta-analysis with random effects. Stat Med 15:619–629

    Article  Google Scholar 

  7. 7.

    Barnett V (1978) The study of outliers: purpose and model. Appl Stat 27:242–250

    Article  Google Scholar 

  8. 8.

    Fleiss JL (1993) The statistical basis of meta analysis. Stat Methods Med Res 2:121–145

    Article  Google Scholar 

  9. 9.

    Hardy RJ, Thompson SG (1998) Detecting and describing heterogeneity in meta-analysis. Stat Med 17:841–856

    Article  Google Scholar 

  10. 10.

    Lee KJ, Thompson SG (2008) Flexible parametric models for random effects distributions. Stat Med 27:418–434. Available online at http://www3.interscience.wiley.com/cgi-bin/abstract/114240283/ABSTRACT

    Article  Google Scholar 

  11. 11.

    Smith TC, Spiegelhalter DJ, Thomas A (1995) Bayesian approaches to random-effects meta-analysis: a comparative study. Stat Med 14:2685–2699

    Article  Google Scholar 

  12. 12.

    Van Houwelingen HC, Zwinderman KH, Stijnen T (1993) A bivariate approach to meta-analysis. Stat Med 12:2273–2284

    Article  Google Scholar 

  13. 13.

    Aitkin M (1999) Meta-analysis by random effect modelling in generalized linear models. Stat Med 18:2343–2351

    Article  Google Scholar 

  14. 14.

    Johnson NL, Kotz S, Balakrishnan N (1994) Continuous univariate distributions. Wiley, New York

    Google Scholar 

  15. 15.

    Burnham KP, Anderson DR (1998) Model Selection and Inference: a practical information-theoretic approach. Springer, New York

    Google Scholar 

  16. 16.

    Tarone RE (1985) Score tests. In: Kotz S, Johnson NL (eds) Encyclopaedia of Statistical Sciences. Wiley, New York

    Google Scholar 

  17. 17.

    Subbotin MT (1923) On the law of frequency of errors. Mathematicheskii Sbornik 31:296–301

    Google Scholar 

  18. 18.

    Fisher RA (1925) Expansion of “Student’s” integral in powers of n  − 1. Metron 5:109–120

    Google Scholar 

  19. 19.

    Geweke J (1991) Genereric, algorithmic approaches to Monte-Carlo integration in Bayesian inference. In: Flournoy N, Tsutakawa RK (eds) Statistical Multiple Integration, pp 117–135

  20. 20.

    Fioravanti M, Yanagi M (2005) Cytidinediphosphocholine (CDP-choline) for cognitive and behavioural disturbances associated with chronic cerebral disorders in the elderly (Review). The Cochrane Collaboration, www.cochrane.org/reviews/en/ab000269.html

  21. 21.

    Marinho VCC, Higgins JPT, Logan S, Sheiham A (2002) Fluoride toothpastes for preventing dental caries in children and adolescents. The Cochrane Collaboration, www.cochrane.org/reviews/en/ab002278.html

  22. 22.

    Edwards JE, Oldman A, Smith L, Collins SL, Carroll D, Wiffen PJ, McQuay HJ, Moore RA (2008) Single dose oral aspirin for acute pain (meta-analysis). The Cochrane Collaboration, www.cochrane.org/reviews/en/ab002067.html

Download references

Author information

Affiliations

  1. Centre for Operational Research and Applied Statistics, University of Salford, Salford, UK

    Rose Baker

  2. MRC Biostatistics Unit, Institute of Public Health, Cambridge, UK

    Dan Jackson

Authors
  1. Rose Baker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dan Jackson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rose Baker.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Baker, R., Jackson, D. A new approach to outliers in meta-analysis. Health Care Manage Sci 11, 121–131 (2008). https://doi.org/10.1007/s10729-007-9041-8

Download citation

Keywords

  • Subbotin distribution
  • Arcsinh transformation
  • Score test
  • Importance sampling