Abstract
Meta-analysis is the statistical method for synthesizing studies on the same topic and is often used in clinical psychology to quantify the efficacy of treatments. A major threat to the validity of meta-analysis is publication bias, which implies that some studies are less likely to be published and are therefore less often included in a meta-analysis. A consequence of publication bias is the overestimation of the meta-analytic effect size that may give a false impression with respect to the efficacy of a treatment, which might result in (avoidable) suffering of patients and waste of resources. Guidelines recommend to routinely assess publication bias in meta-analyses, but this is currently not common practice. This chapter describes popular and state-of-the-art methods to assess publication bias in a meta-analysis and summarizes recommendations for applying these methods. We also illustrate how these methods can be applied to two meta-analyses that are typical for clinical psychology such that psychologists can readily apply the methods in their own meta-analyses.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A vector is R terminology for a particular data structure that contains in our case seven numeric values with the studies’ standardized mean difference (yi) and corresponding sampling variance (vi).
- 2.
The funnel plot based on the data of the meta-analysis by Archer et al. (2012) is available in the annotated R codes (https://osf.io/qjk9b/)
- 3.
The study’s mean, sample size, and standard deviation of both groups are available on page 73 of Cowlishaw et al. (2012).
- 4.
Statistical power of the studies is computed using the estimate of the fixed-effect model as proxy for the true effect size and a two-tailed hypothesis with significance level 0.05 (Stanley et al., 2017).
- 5.
Moderate heterogeneity is defined in terms of the I2-statistic that is commonly used in meta-analysis to quantify the heterogeneity. The I2-statistic (Higgins & Thompson, 2002) indicates the proportion of total variance that can be attributed to heterogeneity in true effect size. Moderate heterogeneity is I2 = 0.5 according to the rules-of-thumb proposed in Higgins et al. (2003).
- 6.
Research is currently ongoing to study whether this assumption can be relaxed by not only weighing statistically significant and nonsignificant studies differently in p-uniform* but also allow more complex weighting schemes. For example, marginally significant studies (i.e., studies with p-values just above the significance threshold) may have a different probability of being published than other nonsignificant studies. Weighing these studies differently may improve estimation and drawing inferences.
References
Aguinis, H., Dalton, D. R., Bosco, F. A., Pierce, C. A., & Dalton, C. M. (2010). Meta-analytic choices and judgment calls: Implications for theory building and testing, obtained effect sizes, and scholarly impact. Journal of Management, 37(1), 5–38. https://doi.org/10.1177/0149206310377113
Aguinis, H., Gottfredson, R. K., & Wright, T. A. (2011). Best-practice recommendations for estimating interaction effects using meta-analysis. Journal of Organizational Behavior, 32(8), 1033–1043. https://doi.org/10.1002/job.719
Alinaghi, N., & Reed, W. R. (2018). Meta-analysis and publication bias: How well does the FAT-PET-PEESE procedure work? Research Synthesis Methods, 9(2), 285–311. https://doi.org/10.1002/jrsm.1298
Alonso, J., Angermeyer, M. C., Bernert, S., Bruffaerts, R., Brugha, T. S., Bryson, H., Girolamo, G., Graaf, R., Demyttenaere, K., Gasquet, I., Haro, J. M., Katz, S. J., Kessler, R. C., Kovess, V., Lépine, J. P., Ormel, J., Polidori, G., Russo, L. J., Vilagut, G., … Vollebergh, W. A. (2004). Prevalence of mental disorders in Europe: Results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatrica Scandinavica. Supplementum, 420, 21–27. https://doi.org/10.1111/j.1600-0047.2004.00327.x
Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. The American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191
Archer, J., Bower, P., Gilbody, S., Lovell, K., Richards, D., Gask, L., Dickens, C., & Coventry, P. (2012). Collaborative care for depression and anxiety problems. Cochrane Database of Systematic Reviews, 10. https://doi.org/10.1002/14651858.CD006525.pub2
Aytug, Z. G., Rothstein, H. R., Zhou, W., & Kern, M. C. (2012). Revealed or concealed? Transparency of procedures, decisions, and judgment calls in meta-analyses. Organizational Research Methods, 15(1), 103–133. https://doi.org/10.1177/1094428111403495
Bakker, M., Van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7(6), 543–554. https://doi.org/10.1177/1745691612459060
Banks, G. C., Kepes, S., & Banks, K. P. (2012). Publication bias: The antagonist of meta-analytic reviews and effective policymaking. Educational Evaluation and Policy Analysis, 34(3), 259–277. https://doi.org/10.3102/0162373712446144
Becker, B. J. (2005). Failsafe N or file-drawer number. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 111–125). Wiley.
Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50(4), 1088–1101.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Wiley.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2), 97–111. https://doi.org/10.1002/jrsm.12
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019). Correcting for bias in psychology: A comparison of meta-analytic methods. Advances in Methods and Practices in Psychological Science, 2(2), 115–144. https://doi.org/10.1177/2515245919847196
Coburn, K. M., & Vevea, J. L. (2015). Publication bias as a function of study characteristics. Psychological Methods, 20(3), 310–330. https://doi.org/10.1037/met0000046
Coburn, K. M., & Vevea, J. L. (2016). weightr: Estimating weight-function models for publication bias. https://cran.r-project.org/package=weightr
Cochran, W. G. (1954). The combination of estimates from different experiments. Biometrics, 10(1), 101–129.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.).
Cooper, H., DeNeve, K., & Charlton, K. (1997). Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychological Methods, 2(4), 447–452. https://doi.org/10.1037/1082-989X.2.4.447
Coursol, A., & Wagner, E. E. (1986). Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias. Professional Psychology: Research and Practice, 17(2), 136–137. https://doi.org/10.1037/0735-7028.17.2.136
Cowlishaw, S., Merkouris, S., Dowling, N., Anderson, C., Jackson, A., & Thomas, S. (2012). Psychological therapies for pathological and problem gambling. Cochrane Database of Systematic Reviews, 11. https://doi.org/10.1002/14651858.CD008937.pub2
David, D., Cotet, C., Matu, S., Mogoase, C., & Stefan, S. (2018). 50 years of rational-emotive and cognitive-behavioral therapy: A systematic review and meta-analysis. Journal of Clinical Psychology, 74(3), 304–318. https://doi.org/10.1002/jclp.22514
Deeks, J. J., Macaskill, P., & Irwig, L. (2005). The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. Journal of Clinical Epidemiology, 58(9), 882–893. https://doi.org/10.1016/j.jclinepi.2005.01.016
Deisenhofer, A., Delgadillo, J., Rubel, J. A., Böhnke, J. R., Zimmermann, D., Schwartz, B., & Lutz, W. (2018). Individual treatment selection for patients with posttraumatic stress disorder. Depression and Anxiety, 35(6), 541–550. https://doi.org/10.1002/da.22755
Driessen, E., Hollon, S. D., Bockting, C. L. H., Cuijpers, P., & Turner, E. H. (2015). Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of US National Institutes of Health-Funded Trials. PLoS One, 10(9), e0137864. https://doi.org/10.1371/journal.pone.0137864
Duval, S., & Tweedie, R. L. (2000a). A nonparametric "trim and fill" method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95(449), 89–98. https://doi.org/10.1080/01621459.2000.10473905
Duval, S., & Tweedie, R. L. (2000b). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634.
Ellis, P. D. (2010). The essential guide to effect sizes: An introduction to statistical power, meta-analysis, and the interpretation of research results. Cambridge University Press. https://doi.org/10.1017/CBO9780511761676
Fanelli, D. (2012). Negative results are disappearing from most disciplines and countries. Scientometrics, 90(3), 891–904. https://doi.org/10.1007/s11192-011-0494-7
Francis, G. (2013). Replication, statistical consistency, and publication bias. Journal of Mathematical Psychology, 57(5), 153–169. https://doi.org/10.1016/j.jmp.2013.02.003
Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505. https://doi.org/10.1126/science.1255484
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015). The extent and consequences of p-hacking in science. PLoS Biology, 13(3), e1002106. https://doi.org/10.1371/journal.pbio.1002106
Hedges, L. V. (1992). Modeling publication selection effects in meta-analysis. Statistical Science, 7(2), 246–255.
Hedges, L. V., & Vevea, J. L. (1996). Estimating effect size under publication bias: Small sample properties and robustness of a random effects selection model. Journal of Educational and Behavioral Statistics, 21(4), 299–332.
Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557–560. https://doi.org/10.1136/bmj.327.7414.557
Holma, K. M., Melartin, T. K., Haukka, J., Holma, I. A. K., Sokero, T. P., & Isometsä, E. T. (2010). Incidence and predictors of suicide attempts in DSM–IV major depressive disorder: A five-year prospective study. American Journal of Psychiatry, 167(7), 801–808. https://doi.org/10.1176/appi.ajp.2010.09050627
Ioannidis, J. P. A. (2013). Clarifications on the application and interpretation of the test for excess significance and its extensions. Journal of Mathematical Psychology, 57(5), 184–187. https://doi.org/10.1016/j.jmp.2013.03.002
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017). The power of bias in economics research. The Economic Journal, 127(605), F236–F265. https://doi.org/10.1111/ecoj.12461
Ioannidis, J. P., & Trikalinos, T. A. (2007). An exploratory test for an excess of significant findings. Clinical Trials, 4(3), 245–253. https://doi.org/10.1177/1740774507079441
Iyengar, S., & Greenhouse, J. B. (1988). Selection models and the file drawer problem: Rejoinder. Statistical Science, 3(1), 133–135. http://www.jstor.org/stable/2245932
Jaycox, L. H., & Foa, E. B. (1999). Cost-effectiveness issues in the treatment of posttraumatic stress disorder. In Cost-effectiveness of psychotherapy: A guide for practitioners, researchers, and policymakers (pp. 259–269). Oxford University Press.
Jin, Z. C., Zhou, X. H., & He, J. (2014). Statistical methods for dealing with publication bias in meta-analysis. Statistics in Medicine, 34(2), 343–360. https://doi.org/10.1002/sim.6342
Kepes, S., Banks, G. C., McDaniel, M., & Whetzel, D. L. (2012). Publication bias in the organizational sciences. Organizational Research Methods, 15(4), 624–662. https://doi.org/10.1177/1094428112452760
Kraemer, H. C., Gardner, C., Brooks, J., & Yesavage, J. A. (1998). Advantages of excluding underpowered studies in meta-analysis: Inclusionist versus exclusionist viewpoints. Psychological Methods, 3(1), 23–31. https://doi.org/10.1037/1082-989X.3.1.23
Lane, D. M., & Dunlap, W. P. (1978). Estimating effect size: Bias resulting from the significance criterion in editorial decisions. British Journal of Mathematical & Statistical Psychology, 31, 107–112.
Lau, J., Antman, E. M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., & Chalmers, T. C. (1992). Cumulative meta-analysis of therapeutic trials for myocardial infarction. New England Journal of Medicine, 327(4), 248–254. https://doi.org/10.1056/nejm199207233270406
Light, R. J., & Pillemer, D. B. (1984). Summing up: The science of reviewing research. Harvard University Press.
Macaskill, P., Walter, S. D., & Irwig, L. (2001). A comparison of methods to detect publication bias in meta-analysis. Statistics in Medicine, 20(4), 641–654.
Maljanen, T., Knekt, P., Lindfors, O., Virtala, E., Tillman, P., & Härkänen, T. (2016). The cost-effectiveness of short-term and long-term psychotherapy in the treatment of depressive and anxiety disorders during a 5-year follow-up. Journal of Affective Disorders, 190, 254–263. https://doi.org/10.1016/j.jad.2015.09.065
Margraf, J. (2009). Kosten und Nutzen der Psychotherapie. Springer Medizin. http://public.ebookcentral.proquest.com/choice/publicfullrecord.aspx?p=450836
Marks-Anglin, A., & Chen, Y. (2020). A historical review of publication bias. Research Synthesis Methods, 11(6), 725–742. https://doi.org/10.1002/jrsm.1452
Matloff, N. S. (2011). The art of R programming: Tour of statistical software design. No Starch Press. http://site.ebrary.com/id/10513550
McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016). Adjusting for publication bias in meta-analysis: An evaluation of selection methods and some cautionary notes. Perspectives on Psychological Science, 11(5), 730–749. https://doi.org/10.1177/1745691616662243
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
Moreno, S. G., Sutton, A. J., Ades, A. E., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009). Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study. BMC Medical Research Methodology, 9(2), 1–17. https://doi.org/10.1186/1471-2288-9-2
Niemeyer, H., Musch, J., & Pietrowsky, R. (2012). Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for schizophrenia. Schizophrenia Research, 138(2), 103–112. https://doi.org/10.1016/j.schres.2012.03.023
Niemeyer, H., Musch, J., & Pietrowsky, R. (2013). Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for depression. Journal of Consulting and Clinical Psychology, 81(1), 58–74. https://doi.org/10.1037/a0031152
Niemeyer, H., Van Aert, R. C. M., Schmid, S., Uelsmann, D., Knaevelsrud, C., & Schulte-Herbrueggen, O. (2020). Publication bias in meta-analyses of posttraumatic stress disorder interventions. Meta-Psychology, 4, 31.
Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8(2), 157–159.
Page, M. J., Higgins, J. P. T., & Sterne, J. A. C. (2019). Chapter 13: Assessing risk of bias due to missing results in a synthesis. In J. P. T. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, & V. A. Welch (Eds.), Cochrane handbook for systematic reviews of interventions version 6.0.
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2007). Performance of the trim and fill method in the presence of publication bias and between-study heterogeneity. Statistics in Medicine, 26(25), 4544–4562. https://doi.org/10.1002/sim.2889
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008). Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. Journal of Clinical Epidemiology, 61(10), 991–996. https://doi.org/10.1016/j.jclinepi.2007.11.010
Raudenbush, S. W. (2009). Analyzing effect sizes: Random-effects models. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (pp. 295–315). Russell Sage Foundation.
R Core Team. (2020). R: A language and environment for statistical computing. http://www.r-project.org/
Renkewitz, F., & Keiner, M. (2019). How to detect publication bias in psychological research. Zeitschrift Für Psychologie. https://doi.org/10.1027/2151-2604/a000386
Rhodes, K. M., Turner, R. M., & Higgins, J. P. (2015). Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data. Journal of Clinical Epidemiology, 68(1), 52–60. https://doi.org/10.1016/j.jclinepi.2014.08.012
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication bias in meta-analysis: Prevention, assessment and adjustments. Wiley.
Scargle, J. D. (2000). Publication bias: The "file-drawer problem" in scientific inference. Journal of Scientific Exploration, 14(1), 91–106. http://arxiv.org/abs/physics/9909033
Schonemann, P. H., & Scargle, J. D. (2008). A generalized publication bias model. Chinese Journal of Psychology, 50(1), 21–29.
Schumacher, S., Niemeyer, H., Engel, S., Cwik, J. C., & Knaevelsrud, C. (2018). Psychotherapeutic treatment and HPA axis regulation in posttraumatic stress disorder: A systematic review and meta-analysis. Psychoneuroendocrinology, 98, 186–201. https://doi.org/10.1016/j.psyneuen.2018.08.006
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve and effect size: Correcting for publication bias using only significant results. Perspectives on Psychological Science, 9(6), 666–681. https://doi.org/10.1177/1745691614553988
Stanley, T. D. (2017). Limitations of PET-PEESE and other meta-analysis methods. Social Psychological and Personality Science, 8(5), 581–591. https://doi.org/10.1177/1948550617693062
Stanley, T. D., & Doucouliagos, H. (2014). Meta-regression approximations to reduce publication selection bias. Research Synthesis Methods, 5(1), 60–78.
Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. (2017). Finding the power to reduce publication bias. Statistics in Medicine, 36(10), 1580–1598. https://doi.org/10.1002/sim.7228
Stanley, T. D., Jarrell, S. B., & Doucouliagos, H. (2010). Could it be better to discard 90% of the data? A statistical paradox. The American Statistician, 64(1), 70–77. https://doi.org/10.1198/tast.2009.08205
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000). Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology, 53(11), 1119–1129. https://doi.org/10.1016/S0895-4356(00)00242-0
Sterne, J. A. C., Harbord, R. M., Sutton, A. J., Jones, D. R., Ioannidis, J. P., Terrin, N., Lau, J., Schmid, C. H., Carpenter, J., Rucker, G., Schwarzer, G., Tetzlaff, J., Moher, D., Deeks, J. J., Peters, J., Macaskill, P., Duval, S., Altman, D. G., & Higgins, J. P. T. (2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. British Medical Journal, 343(7818), 1–8. https://doi.org/10.1136/bmj.d4002
Teetor, P. (2011). R cookbook. O’Reilly.
Terrin, N., Schmid, C. H., & Lau, J. (2005). In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. Journal of Clinical Epidemiology, 58(9), 894–901. https://doi.org/10.1016/j.jclinepi.2005.01.006
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003). Adjusting for publication bias in the presence of heterogeneity. Statistics in Medicine, 22(13), 2113–2126. https://doi.org/10.1002/sim.1461
Tolin, D. F., McKay, D., Forman, E. M., Klonsky, E. D., & Thombs, B. D. (2015). Empirically supported treatment: Recommendations for a new model. Clinical Psychology: Science and Practice, 22(4), 317–338. https://doi.org/10.1111/cpsp.12122
Turner, R. M., Jackson, D., Wei, Y., Thompson, S. G., & Higgins, J. P. T. (2015). Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis. Statistics in Medicine, 34(6), 984–998. https://doi.org/10.1002/sim.6381
Van Aert, R. C. M. (2020). Puniform: Meta-analysis methods correcting for publication bias. https://cran.r-project.org/package=puniform
Van Aert, R. C. M., & Van Assen, M. A. L. M. (2022). Correcting for publication bias in a meta-analysis with the p-uniform* method. Manuscript submitted for publication. https://doi.org/10.31222/osf.io/zqjr9
Van Aert, R. C. M., Wicherts, J. M., & Van Assen, M. A. L. M. (2016). Conducting meta-analyses on p-values: Reservations and recommendations for applying p-uniform and p-curve. Perspectives on Psychological Science, 11(5), 713–729. https://doi.org/10.1177/1745691616650874
Van Aert, R. C. M., Wicherts, J. M., & Van Assen, M. A. L. M. (2019). Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis. PLoS One, 14(4), e0215052. https://doi.org/10.1371/journal.pone.0215052
Van Assen, M. A. L. M., Van Aert, R. C. M., & Wicherts, J. M. (2015). Meta-analysis using effect size distributions of only statistically significant studies. Psychological Methods, 20(3), 293–309. https://doi.org/10.1037/met0000025
Van Assen, M. A. L. M., Van Den Akker, O. R., Augusteijn, H. E. M., Bakker, M., Nuijten, M. B., Olsson-Collentine, A., Stoevenbelt, A. H., Wicherts, J. M., & Van Aert, R. C. M. (2022). The meta-plot: A graphical tool for interpreting the results of a meta-analysis. Manuscript submitted for publication. https://doi.org/10.31234/osf.io/cwhnq
Vandekerckhove, J., Guan, M., & Styrcula, S. A. (2013). The consistency test may be too weak to be useful: Its systematic application would not improve effect size estimation in meta-analyses. Journal of Mathematical Psychology, 57(5), 170–173. https://doi.org/10.1016/j.jmp.2013.03.007
Vevea, J. L., Clements, N. C., & Hedges, L. V. (1993). Assessing the effects of selection bias on validity data for the General Aptitude Test Battery. Journal of Applied Psychology, 78(6), 981–987. https://doi.org/10.1037/0021-9010.78.6.981
Vevea, J. L., & Woods, C. M. (2005). Publication bias in research synthesis: Sensitivity analysis using a priori weight functions. Psychological Methods, 10(4), 428–443. https://doi.org/10.1037/1082-989X.10.4.428
Viechtbauer, W. (2007). Confidence intervals for the amount of heterogeneity in meta-analysis. Statistics in Medicine, 26(1), 37–52. https://doi.org/10.1002/sim.2514
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7, 1832. https://doi.org/10.3389/fpsyg.2016.01832
Author Note
We would like to thank Claudia Kapp, Manuel Heinrich, and Johannes Heekerens for commenting on a previous version of this chapter.
The authors made the following contributions. Robbie C.M. van Aert: Conceptualization, Formal analysis, Writing—Original Draft Preparation, Writing—Review & Editing; Helen Niemeyer: Conceptualization, Writing—Review & Editing.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this chapter
Cite this chapter
van Aert, R.C.M., Niemeyer, H. (2022). Publication Bias. In: O'Donohue, W., Masuda, A., Lilienfeld, S. (eds) Avoiding Questionable Research Practices in Applied Psychology. Springer, Cham. https://doi.org/10.1007/978-3-031-04968-2_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-04968-2_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04967-5
Online ISBN: 978-3-031-04968-2
eBook Packages: Behavioral Science and PsychologyBehavioral Science and Psychology (R0)