Skip to main content

Methodological issues and advances in biological meta-analysis

Evolutionary Ecology volume 26, pages 1253–1274 (2012)Cite this article

Abstract

Meta-analysis has changed the way researchers conduct literature reviews not only in medical and social sciences but also in biological sciences. Meta-analysis in biological sciences, especially in ecology and evolution (which we refer to as ‘biological’ meta-analysis) faces somewhat different methodological problems from its counterparts in medical and social sciences, where meta-analytic techniques were originally developed. The main reason for such differences is that biological meta-analysis often integrates complex data composed of multiple strata with, for example, different measurements and a variety of species. Here, we review methodological issues and advancements in biological meta-analysis, focusing on three topics: (1) non-independence arising from multiple effect sizes obtained in single studies and from phylogenetic relatedness, (2) detecting and accounting for heterogeneity, and (3) identifying publication bias and measuring its impact. We show how the marriage between mixed-effects (hierarchical/multilevel) models and phylogenetic comparative methods has resolved most of the issues under discussion. Furthermore, we introduce the concept of across-study and within-study meta-analysis, and propose how the use of within-study meta-analysis can improve many empirical studies typical of ecology and evolution.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

  • Adams DC (2008) Phylogenetic meta-analysis. Evolution 62:567–572

    Article  PubMed  Google Scholar 

  • Arnqvist G, Wooster D (1995) Meta-analysis: synthesizing research findings in ecology and evolution. Trends Ecol Evol 10:236–240

    Article  PubMed  CAS  Google Scholar 

  • Barto EK, Rillig MC (2012) Dissemination biases in ecology: effect sizes matter more than quality. Oikos 121:228–235

    Google Scholar 

  • Becker BJ (2005) Fail safe N or file-drawer number. In: Rothstein H, Sutton AJ, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester, pp 111–125

    Google Scholar 

  • Begg CB, Mazumdar M (1994) Operating characteristics of a rank correlation test for publication bias. Biometrics 50:1088–1101

    Article  PubMed  CAS  Google Scholar 

  • Blomberg SP, Garland T, Ives AR (2003) Testing for phylogenetic signal in comparative data: behavioral traits are more labile. Evolution 57:717–745

    PubMed  Google Scholar 

  • Bolker BM, Brooks ME, Clark CJ, Geange SW, Poulsen JR, Stevens MHH, White JSS (2009) Generalized linear mixed models: a practical guide for ecology and evolution. Trends Ecol Evol 24:127–135

    Article  PubMed  Google Scholar 

  • Borenstein M (2005) Software for publication bias. In: Rothstein H, Sutton AJ, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester, pp 193–220

    Google Scholar 

  • Borenstein M, Hedges LV, Higgins JPI, Rothstein HR (2009) Introduction to meta-analysis. Wiley, Chichester

    Book  Google Scholar 

  • Burnham KP, Anderson DR (2002) Model selection and multimodel inference: a practical information-theoretic approach, 2nd edn. Springer, Berlin

    Google Scholar 

  • Carmona D, Lajeunesse MJ, Johnson MTJ (2011) Plant traits that predict resistance to herbivores. Funct Ecol 25:358–367

    Article  Google Scholar 

  • Cheung MWL (2008) A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. Psychol Methods 13:182–202

    Article  PubMed  Google Scholar 

  • Cleasby IR, Nakagawa S (2012) The influence of male age on within-pair and extra-pair paternity in passerines. Ibis. doi:10.1111/j.1474-919X.2011.01209.x

  • Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum, Hillsdale

    Google Scholar 

  • Congdon P (2003) Applied Bayesian modelling. Wiley, Chichester

    Book  Google Scholar 

  • Cooper H, Hedges LV, Valentine JC (2009) The handbook of research synthesis and meta-analysis, 2nd edn. Russell Sage Foundation, New York

    Google Scholar 

  • Cornwallis CK, West SA, Davis KE, Griffin AS (2010) Promiscuity and the evolutionary transition to complex societies. Nature 466:969–972

    Article  PubMed  CAS  Google Scholar 

  • Cumming G, Finch S (2001) A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educ Psychol Measurement 61:532–584

    Google Scholar 

  • Davidson AM, Jennions M, Nicotra AB (2011) Do invasive species show higher phenotypic plasticity than native species and, if so, is it adaptive? A meta-analysis. Ecol Lett 14:419–431

    Article  PubMed  Google Scholar 

  • Dubois F, Cezilly F (2002) Breeding success and mate retention in birds: a meta-analysis. Behav Ecol Sociobiol 52:357–364

    Article  Google Scholar 

  • Duval S (2005) The trim and fill method. In: Rothstein H, Sutton AJ, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester, pp 127–144

    Google Scholar 

  • Duval S, Tweedie R (2000a) A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. J Am Stat Assoc 95:89–98

    Google Scholar 

  • Duval S, Tweedie R (2000b) Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56:455–463

    Article  PubMed  CAS  Google Scholar 

  • Egger M, Smith GD, Schneider M, Minder C (1997) Bias in meta-analysis detected by a simple, graphical test. Br Med J 315:629–634

    Article  CAS  Google Scholar 

  • Egger M, Smith GD, Altman DG (2001) Systematic reviews in health care: meta-analysis in context, 2nd edn. BMJ, London

    Book  Google Scholar 

  • Evans SR, Hinks AE, Wilkin TA, Sheldon BC (2010) Age, sex and beauty: methodological dependence of age- and sex-dichromatism in the great tit Parus major. Biol J Linn Soc 101:777–796

    Article  Google Scholar 

  • Felsenstein J (1985) Phylogenies and the comparative method. Am Nat 125:1–15

    Article  Google Scholar 

  • Felsenstein J (2008) Comparative methods with sampling error and within-species variation: contrasts revisited and revised. Am Nat 171:713–725

    Article  PubMed  Google Scholar 

  • Freckleton RP, Harvey PH, Pagel M (2002) Phylogenetic analysis and comparative data: a test and review of evidence. Am Nat 160:712–726

    Article  PubMed  CAS  Google Scholar 

  • Garamszegi LZ (2006) Comparing effect sizes across variables: generalization without the need for Bonferroni correction. Behav Ecol 17:682–687

    Article  Google Scholar 

  • Gasparrini A (2011) Multivariate meta-analysis and meta-regression: package ‘mvmeta’. In: R package version 0.2.3 edn. http://cran.r-project.org/web/packages/mvmeta/index.html

  • Gates S (2002) Review of methodology of quantitative reviews using meta-analysis in ecology. J Anim Ecol 71:547–557

    Article  Google Scholar 

  • Gelman A, Hill J (2007) Data analysis using regression and multilevel/hierarchical models. Cambridge University Press, Cambridge

    Google Scholar 

  • Gilmour AR, Gogel BJ, Cullis BR, Welham SJ, Thompson R (2002) ASReml user guide release 1.0. VSN Internationl Ltd, Hemel Hempstead, UK

  • Griffin AS, West SA (2003) Kin discrimination and the benefit of helping in cooperatively breeding vertebrates. Science 302:634–636

    Article  PubMed  CAS  Google Scholar 

  • Griffin AS, Sheldon BC, West SA (2005) Cooperative breeders adjust offspring sex ratios to produce helpful helpers. Am Nat 166:628–632

    Google Scholar 

  • Grueber CE, Nakagawa S, Laws RJ, Jamieson IG (2011) Multimodel inference in ecology and evolution: challenges and solutions. J Evol Biol 24:699–711

    Article  PubMed  CAS  Google Scholar 

  • Gurevitch J, Hedges LV (1999) Statistical issues in ecological meta-analyses. Ecology 80:1142–1149

    Article  Google Scholar 

  • Gurevitch J, Curtis PS, Jones MH (2001) Meta-analysis in ecology. Adv Ecol Res 32:199–247

    Article  CAS  Google Scholar 

  • Hadfield JD (2010) MCMC methods for multi-response generalised linear mixed models: the MCMCglmm R package. J Stat Softw 33:1–22

    Google Scholar 

  • Hadfield JD, Nakagawa S (2010) General quantitative genetic methods for comparative biology: phylogenies, taxonomies and multi-trait models for continuous and categorical characters. J Evol Biol 23:494–508

    Article  PubMed  CAS  Google Scholar 

  • Hamilton WD (1964) The genetical evolution of social behaviour. I and II. J Theor Biol 7:1–52

    Article  PubMed  CAS  Google Scholar 

  • Harrison F (2011) Getting started with meta-analysis. Methods Ecol Evol 2:1–10

    Article  Google Scholar 

  • Harvey PH, Pagel MD (1991) The comparative method in evolutionary biology. Oxford University Press, Oxford

    Google Scholar 

  • Hatchwell BJ (1999) Investment strategies of breeders in avian cooperative breeding systems. Am Nat 154:205–219

    Article  Google Scholar 

  • Hector KL, Nakagawa S (2012) Quantitative analysis of compensatory and catch-up growth in diverse taxa. J Anim Ecol. doi:10.1111/j.1365-2656.2011.01942.x

  • Hedges L, Olkin I (1985) Statistical methods for meta-analysis. Academic Press, New York

    Google Scholar 

  • Hedges LV, Vevea JL (1998) Fixed- and random-effects models in meta-analysis. Psychol Methods 3:486–504

    Article  Google Scholar 

  • Hedges LV, Vevea JL (2005) Selection method approaches. In: Rothstein H, Sutton AJ, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester, pp 145–174

    Google Scholar 

  • Hedges LV, Gurevitch J, Curtis PS (1999) The meta-analysis of response ratios in experimental ecology. Ecology 80:1150–1156

    Article  Google Scholar 

  • Higgins JPT, Thompson SG (2002) Quantifying heterogeneity in a meta-analysis. Stat Med 21:1539–1558

    Article  PubMed  Google Scholar 

  • Higgins JPT, Thompson SG, Deeks JJ, Altman DG (2003) Measuring inconsistency in meta-analyses. Br Med J 327:557–560

    Article  Google Scholar 

  • Higgins JPT, Thompson SG, Spiegelhalter DJ (2009) A re-evaluation of random-effects meta-analysis. J R Stat Soc A 172:137–159

    Article  Google Scholar 

  • Horváthová T, Nakagawa S, Uller T (2012) Strategic female reproductive investment in response to male attractiveness in birds. Proc R Soc B 279:163–170

    Google Scholar 

  • Hunt M (1997) How science takes stock: the story of meta-analysis. Russell Sage, New York

    Google Scholar 

  • Ives AR, Zhu J (2006) Statistics for correlated data: phylogenies, space, and time. Ecol Appl 16:20–32

    Article  PubMed  Google Scholar 

  • Ives AR, Midford PE, Garland T (2007) Within-species variation and measurement error in phylogenetic comparative methods. Syst Biol 56:252–270

    Article  PubMed  Google Scholar 

  • Jackson D, Riley R, White IR (2011) Multivariate meta-analysis: potential and promise. Stat Med 30:2481–2498

    Article  Google Scholar 

  • Jennions MD, Møller AP (2002) Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution. Proc R Soc B 269:43–48

    Article  PubMed  Google Scholar 

  • Jennions MD, Lorite C, Rosenberg M, Rothstein H (2012) Publication and related biases. In: Koricheva J, Gurevitch J, Mengersen K (eds) The handbook of meta-analysis in ecology and evolution. Princeton University Press, Princeton (in press)

  • Jones KS, Nakagawa S, Sheldon BC (2009) Environmental sensitivity in relation to size and sex in birds: meta-regression analysis. Am Nat 174:122–133

    Article  PubMed  Google Scholar 

  • Kelly CD, Jennions MD (2011) Sexual selection and sperm quantity: meta-analyses of strategic ejaculation. Biol Rev 86:863–884

    Article  PubMed  Google Scholar 

  • Knowles SCL, Nakagawa S, Sheldon BC (2009) Elevated reproductive effort increases blood parasitaemia and decreases immune function in birds: a meta-regression approach. Funct Ecol 23:405–415

    Article  Google Scholar 

  • Koricheva J, Gurevitch J, Mengersen K (2012) The handbook of meta-analysis in ecology and evolution. Princeton University Press, Princeton

    Google Scholar 

  • Kueffer C, Niinemets U, Drenovsky RE, Kattge J, Milberg P, Poorter H, Reich PB, Werner C, Westoby M, Wright IJ (2011) Fame, glory and neglect in meta-analyses. Trends Ecol Evol 26:493–494

    Article  PubMed  Google Scholar 

  • Lajeunesse MJ (2009) Meta-analysis and the comparative phylogenetic method. Am Nat 174:369–381

    Article  PubMed  Google Scholar 

  • Lajeunesse MJ (2010) Achieving synthesis with meta-analysis by combining and comparing all available studies. Ecology 91:2561–2564

    Article  PubMed  Google Scholar 

  • Lajeunesse MJ (2011a) On the meta-analysis of response ratios for studies with correlated and multi-group designs. Ecology 92:2049–2055

    Article  Google Scholar 

  • Lajeunesse MJ (2011b) phyloMeta: a program for phylogenetic comparative analyses with meta-analysis. Bioinformatics 27:2603–2604

    PubMed  CAS  Google Scholar 

  • Lajeunesse MJ, Forbes MR (2003) Variable reporting and quantitative reviews: a comparison of three meta-analytical techniques. Ecol Lett 6:448–454

    Article  Google Scholar 

  • Leimu R, Koricheva J (2004) Cumulative meta-analysis: a new tool for detection of temporal trends and publication bias in ecology. Proc R Soc B 271:1961–1966

    Article  PubMed  Google Scholar 

  • Leimu R, Koricheva J (2005) What determines the citation frequency of ecological papers? Trends Ecol Evol 20:28–32

    Article  PubMed  Google Scholar 

  • Liermann M, Hilborn R (1997) Depensation in fish stocks: a hierarchic Bayesian meta-analysis. Can J Fish Aquat Sci 54:1976–1984

    Google Scholar 

  • Lipsey MW, Wilson DB (2001) Practical meta-analysis. Sage, Beverly Hills

    Google Scholar 

  • Lortie CJ, Aarssen LW, Budden AE, Koricheva JK, Leimu R, Tregenza T (2007) Publication bias and merit in ecology. Oikos 116:1247–1253

    Article  Google Scholar 

  • Lunn DJ, Thomas A, Best N, Spiegelhalter D (2000) WinBUGS—a Bayesian modelling framework: concepts, structure, and extensibility. Stat Computing 10:325–337

    Article  Google Scholar 

  • Lynch M (1991) Methods for the analysis of comparative data in evolutionary biology. Evolution 45:1065–1080

    Article  Google Scholar 

  • Martins EP, Hansen TF (1997) Phylogenies and the comparative method: a general approach to incorporating phylogenetic information into the analysis of interspecific data. Am Nat 149:646–667

    Article  Google Scholar 

  • Meunier J, Pinto SF, Burri R, Roulin A (2011) Eumelanin-based coloration and fitness parameters in birds: a meta-analysis. Behav Ecol Sociobiol 65:559–567

    Article  Google Scholar 

  • Milner RNC, Detto T, Jennions MD, Backwell PRY (2010) Experimental evidence for a seasonal shift in the strength of a female mating preference. Behav Ecol 21:311–316

    Article  Google Scholar 

  • Møller AP, Jennions MD (2001) Testing and adjusting for publication bias. Trends Ecol Evol 16:580–586

    Article  Google Scholar 

  • Nakagawa S, Cuthill IC (2007) Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol Rev 82:591–605

    Article  PubMed  Google Scholar 

  • Nakagawa S, Freckleton RP (2008) Missing inaction: the dangers of ignoring missing data. Trends Ecol Evol 23:592–596

    Article  PubMed  Google Scholar 

  • Nakagawa S, Hauber ME (2010) Great challenges with few subjects: statistical strategies for neuroscientists. Neurosci Biobehav Rev 35:462–473

    Article  PubMed  Google Scholar 

  • Osenberg CW, Sarnelle O, Cooper SD, Holt RD (1999) Resolving ecological questions through meta-analysis: goals, metrics, and models. Ecology 80:1105–1117

    Article  Google Scholar 

  • Pagel M (1999) Inferring the historical patterns of biological evolution. Nature 401:877–884

    Article  PubMed  CAS  Google Scholar 

  • Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L (2007) Performance of the trim and fill method in the presence of publication bias and between-study heterogeneity. Stat Med 26:4544–4562

    Article  PubMed  Google Scholar 

  • Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L (2008) Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol 61:991–996

    Article  PubMed  Google Scholar 

  • Pigott TD (2009) Handling missing data. In: Cooper H, Hedges LV, Valentine JC (eds) The handbook of research synthesis and meta-analysis. Russell Sage Foundation, New York, pp 399–416

    Google Scholar 

  • Pinheiro JC, Bates DM (2000) Mixed-effects models in S and S-Plus. Springer, New York

    Book  Google Scholar 

  • Poulin R (2000) Manipulation of host behaviour by parasites: a weakening paradigm? Proc R Soc B 267:787–792

    Article  PubMed  CAS  Google Scholar 

  • Roberts CJ, Stanley TD (2005) Meta-regression analysis: issues of publication bias in economics. Blackwell, Malden

    Google Scholar 

  • Rosenberg MS (2005) The file-drawer problem revisited: a general weighted method for calculating fail-safe numbers in meta-analysis. Evolution 59:464–468

    PubMed  Google Scholar 

  • Rosenberg MS, Adams DC, Gurevitch J (2000) MetaWin: statistical software for meta-analysis, 2nd edn. Sinauer, Sunderland

    Google Scholar 

  • Rosenthal R (1979) The “file drawer problem” and tolerance for null results. Psychol Bull 86:638–641

    Article  Google Scholar 

  • Rothstein H, Sutton AJ, Borenstein M (2005) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester

    Book  Google Scholar 

  • Santos ESA, Macedo RH (2011) Load lightening in southern lapwings: group-living mothers lay smaller eggs than pair-living mothers. Ethology 117:547–555

    Article  Google Scholar 

  • Santos ESA, Scheck D, Nakagawa S (2011) Dominance and plumage traits: meta-analysis and metaregression analysis. Anim Behav 82:3–19

    Article  Google Scholar 

  • Schielzeth H (2010) Simple means to improve the interpretability of regression coefficients. Methods Ecol and Evol 1:103–113

    Article  Google Scholar 

  • Schwarzer G (2010) Meta-analysis with R: package ‘meta’. In: R package version 1.6-1 edn. http://cran.r-project.org/web/packages/meta/index.html

  • Schwarzer G, Carpenter J, Rucker G (2010) Empirical evaluation suggests Copas selection model preferable to trim and fill method for selection bias in meta-analysis. J Clin Epidemiol 63:282–288

    Article  PubMed  Google Scholar 

  • Slatyer RA, Mautz BS, Backwell PR, Jennions MD (2011) Estimating genetic benefits of polyandry from experimental studies: a meta-analysis. Biol Rev 86:863–884

    Article  Google Scholar 

  • Smith BR, Blumstein DT (2008) Fitness consequences of personality: a meta-analysis. Behav Ecol 19:448–455

    Article  Google Scholar 

  • Sterne JAC, Becker BJ, Egger M (2005) The funnel plot. In: Rothstein H, Sutton AJ, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester, pp 75–98

    Google Scholar 

  • Sutton AJ (2009) Publication bias. In: Cooper H, Hedges LV, Valentine JC (eds) The handbook of research synthesis and meta-analysis. Russell Sage Foundation, New York, pp 435–452

    Google Scholar 

  • Sutton AJ, Higgins JPI (2008) Recent developments in meta-analysis. Stat Med 27:625–650

    Article  PubMed  Google Scholar 

  • Sutton JT, Nakagawa S, Robertson BC, Jamieson IG (2011) Disentangling the roles of natural selection and genetic drift in shaping variation at MHC immunity genes. Mol Ecol 20:4408–4420

    Article  PubMed  Google Scholar 

  • Terrin N, Schmid CH, Lau J (2005) In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. J Clin Epidemiol 58:894–901

    Article  PubMed  Google Scholar 

  • Thompson B (2002) What future quantitative social science research could look like: confidence intervals for effect sizes. Educ Researcher 31:25–32

    Article  Google Scholar 

  • Trikalinos TA, Ioannidis JP (2005) Assessing the evolution of effect sizes over time. In: Rothstein H, Sutton AJ, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester, pp 241–259

    Google Scholar 

  • Van den Bergh B, Dewitte S (2006) Digit ratio (2D: 4D) moderates the impact of sexual cues on men’s decisions in ultimatum games. Proc R Soc B 273:2091–2095

    Article  PubMed  Google Scholar 

  • Verdu M, Traveset A (2004) Bridging meta-analysis and the comparative method: a test of seed size effect on germination after frugivores’ gut passage. Oecologia 138:414–418

    Article  PubMed  Google Scholar 

  • Verdu M, Traveset A (2005) Early emergence enhances plant fitness: a phylogenetically controlled meta-analysis. Ecology 86:1385–1394

    Article  Google Scholar 

  • Viechtbauer W (2010) Conducting meta-analyses in R with the meta for package. J Stat Softw 36:1–48

    Google Scholar 

  • Wang MC, Bushman BJ (1998) Using the normal quantile plot to explore meta-analytic data sets. Psychol Methods 3:46–54

    Article  Google Scholar 

  • Wehi P, Nakagawa S, Trewick S, Morgan-Richards M (2011) Does predation result in adult sex ratio skew in a sexually dimorphic insect genus? J Evol Biol 24:2321–2328

    Article  PubMed  CAS  Google Scholar 

  • Weir LK, Grant JWA, Hutchings JA (2011) The influence of operational sex ratio on the intensity of competition for mates. Am Nat 177:167–176

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors thank J. Endler for conceiving this special issue and E. Schlicht, S. Chamberlain, M. Lagisz and two anonymous reviewers for the comments on an early version of this paper. S.N. acknowledges the Humboldt Fellowship and Marsden Fund for support and also is grateful to B. Kempenaers for hosting and providing an excellent working environment at the MPIO. E.S.A.S. acknowledges the support of a University of Otago PhD Scholarship.

Author information

Authors and Affiliations

  1. Department of Zoology, University of Otago, 340 Great King Street, Dunedin, 9054, New Zealand

    Shinichi Nakagawa & Eduardo S. A. Santos

  2. Max Planck Institute for Ornithology, Seewiesen, Germany

    Shinichi Nakagawa

Authors
  1. Shinichi Nakagawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eduardo S. A. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shinichi Nakagawa.

Appendix: A survey of biological meta-analytic studies

Appendix: A survey of biological meta-analytic studies

We conducted a survey of the meta-analyses published in the last 3 years (2009–2011). We performed a search of the online database Scopus, by using the following keywords: “meta-analys*”, and “meta-regression” (last updated in October 2011); we restricted our search to the following journal titles: American Naturalist, Animal Behaviour, Behavioral Ecology, Behavioral Ecology and Sociobiology, Biological Reviews, Ecology, Ecological Application, Ecology Letters, Ecological Monographs, Evolution, Evolutionary Ecology, Functional Ecology, Journal of Evolutionary Biology, Journal of Animal Ecology, Journal of Applied Ecology, Journal of Ecology, Molecular Ecology, New Phytologist, Oecologia, Oikos, and Quarterly Reviews of Biology. Our search yielded a total of 390 original studies; we used the most recent 100 ‘biological’ meta-analytic studies as defined in the Introduction. From each study, we retrieved the following information: (1) the type of effect size used, (2) whether the study accounted for phylogenetic relatedness or not, (3) whether the study used any procedure to deal with publication bias, and (4) the software used to conduct the meta-analysis. In addition, we classified the effect sizes into the three main categories (i.e., means, correlation coefficient, and dichotomous; see section ‘Effect-size statistics in ecology and evolution’). All other types of effect sizes (e.g. repeatability and selection coefficients) were pooled in one category (named ‘others’). If the studies used any procedure for dealing with publication bias, we classified them into one of three categories: (1) studies that only identified publication bias, (2) studies that only assessed the impact of publication bias; or (3) studies that conducted both types of procedures (see the section ‘Publication bias’). In addition to the results reported in the main text, we found that most studies did not account (83%) for phylogenetic relatedness of the species included in the analyses. For this estimate, we excluded most of the ecological meta-analyses, because these studies mainly deal with effects on ecosystem or community scales, where phylogenetic non-independence does not apply. Most of the meta-analyses included in our survey used the software MetaWin to conduct their analyses (42.3% of studies). Eighteen studies did not report the software used; 19 different meta-analysis software packages were employed in the remaining studies.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Nakagawa, S., Santos, E.S.A. Methodological issues and advances in biological meta-analysis. Evol Ecol 26, 1253–1274 (2012). https://doi.org/10.1007/s10682-012-9555-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10682-012-9555-5

Keywords

  • Fixed-effect meta-analysis
  • Random-effects meta-analysis
  • Meta-regression
  • Egger’s regression
  • I 2
  • Heterogeneity
  • Multivariate meta-analysis
  • Trim and fill method