Skip to main content
SpringerLink
Log in

Avoiding Bias in Publication Bias Research: The Value of “Null” Findings

  • Published:
Journal of Business and Psychology Aims and scope Submit manuscript

Abstract

Meta-analytic reviews are an important tool for advancing science and guiding evidence-based practice. Publication bias is one of the greatest threats to meta-analytic reviews. This paper assesses the degree of publication bias in four previously published meta-analytic datasets from various fields of study in the organizational sciences. Of these datasets, one appears to be relatively unaffected by publication bias while the others seem to be noticeably influenced by this bias. Our “null” result (i.e., a prior meta-analytic estimate is unlikely to have been affected by publication bias) increases our confidence in the accuracy of our cumulative knowledge. Yet, our other findings suggest the presence of publication bias and point to the need for caution and further research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Among other factors, the failsafe N (Rosenthal 1979) rests on the improbable assumption that “missing” effect sizes are zero in magnitude (it focuses on p values rather than effect size magnitude). Thus, the failsafe N is inadequate to assess publication bias (Becker 2005; see also, e.g., Aguinis et al. 2011; Becker 1994, Evans 1996; Kepes et al. 2012).

  2. In the organizational sciences, sub-group analyses tend to compare samples published in journal articles to “unpublished” samples (e.g., dissertations, conference papers, completely unpublished samples). Thus, sub-group comparisons assess the extent to which the results from distinct sub-groups differ (Banks et al. 2010). Inherent in such a comparison are the assumptions that all published and unpublished samples have been identified or that the samples in each sub-group are representative of all completed samples within those sub-groups. Both assumptions are unlikely to hold (Hopewell et al. 2005).

  3. The accuracy of meta-analytic estimates depends on the number samples in the meta-analytic distribution, which depends on sample and study properties, including levels of statistical significance (i.e., the samples in the meta-analytic distribution are unlikely to be perfectly “true” representations of the population). This type of sampling error is called second-order sampling error (Hunter and Schmidt 2004). The smaller the number of samples in the meta-analytic distribution, the higher the chances that the meta-analytic results are influenced by this error.

  4. This does not affect the decision to use the random-effects estimation model for the meta-analytic procedures as the estimation models for the meta-analytic and trim and fill procedures are independent of each other.

  5. The random-effects trim and fill model did not support this finding as zero samples were imputed, yielding results that are identical to the meta-analytic ones (i.e., t&f adj. \( {\bar r_o} \) = .28; t&f adj. 95 % CI = .24–.33).

  6. The random-effects trim and fill model imputed one missing sample, yielding a trim and fill adjusted \( {\bar r_o} \) of .17 (t&f adj. 95 % CI = .03–.30). Given the size of the distribution (k = 8), we did not interpret the results from Egger’s test of the intercept and Begg and Mazumdar’s rank correlation test.

  7. The random-effects trim and fill imputed four samples at the right-hand side of the funnel plot, yielding a trim and fill adjusted observed mean of .26 (t&f adj. 95 % CI = .20–.32).

  8. However, the random-effects trim and fill did not impute any missing samples, leaving the trim and fill adjusted statistics identical to the meta-analytic ones (i.e., t&f adj. \( {\bar r_o} \) = .17; t&f adj. 95 % CI = .12–.22).

  9. The random-effects trim and fill indicated this as well. It imputed 13 samples at the right-hand side of the funnel plot, yielding a trim and fill adjusted observed mean of −.07 (t&f adj. 95 % CI = −.14 to −.01).

  10. The random-effects trim and fill model did not support this finding. This trim and fill model imputed zero samples, yielding results that are identical to the meta-analytic ones (i.e., t&f adj. \( {\bar r_o} \) = 1.09; t&f 95 % CI = .72–1.46).

  11. Samples in the social sciences, including the organizational sciences, typically contain more between-sample heterogeneity than samples in the medical sciences. For instance, the medical sciences tend to use randomized control trials, which are subject to substantially less heterogeneous influences than the study designs predominantly used in the organizational sciences (e.g., field studies). Similarly, heterogeneous influences due to measurement error tend to be substantially less in the medical sciences because variables are often dichotomous (e.g., drug treatment [yes/no], side-effects [yes/no], and death [yes/no]).

References

  • Aguinis, H., Pierce, C. A., Bosco, F. A., Dalton, D. R., & Dalton, C. M. (2011). Debunking myths and urban legends about meta-analysis. Organizational Research Methods, 14, 306–331. doi:10.1177/1094428110375720.

    Article  Google Scholar 

  • American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.

    Google Scholar 

  • Banks, G. C., Batchelor, J. H., & McDaniel, M. A. (2010). Smarter people are (a bit) more symmetrical: A meta-analysis of the relationship between intelligence and fluctuating asymmetry. Intelligence, 38, 393–401. doi:10.1016/j.intell.2010.04.003.

    Article  Google Scholar 

  • Banks, G. C., Kepes, S., & Banks, K. P. (2012a). Publication bias: The antagonist of meta-analytic reviews and effective policy making. Educational Evaluation and Policy Analysis, 34, 259–277. doi:10.3102/0162373712446144.

    Article  Google Scholar 

  • Banks, G. C., Kepes, S., & McDaniel, M. A. (2012b). Publication bias: A call for improved meta-analytic practice in the organizational sciences. International Journal of Selection and Assessment, 20, 182–196. doi:10.1111/j.1468-2389.2012.00591.x.

    Article  Google Scholar 

  • Banks, G. C., & McDaniel, M. A. (2011). The kryptonite of evidence-based I-O psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 4, 40–44. doi:10.1111/j.1754-9434.2010.01292.x.

    Article  Google Scholar 

  • Becker, B. J. (1994). Combining significance levels. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 215–230). New York: Russell Sage Foundation.

    Google Scholar 

  • Becker, B. J. (2005). The failsafe N or file-drawer number. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 111–126). West Sussex: Wiley.

    Google Scholar 

  • Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101. doi:10.2307/2533446.

    Article  PubMed  Google Scholar 

  • Berlin, J. A., & Ghersi, D. (2005). Preventing publication bias: Registries and prospective meta-analysis. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 35–48). West Sussex: Wiley.

    Google Scholar 

  • Bettencourt, B. A., & Miller, N. (1996). Gender differences in aggression as a function of provocation: A meta-analysis. Psychological Bulletin, 119, 422–447. doi:10.1037/0033-2909.119.3.422.

    Article  PubMed  Google Scholar 

  • Blackwell, S. C., Thompson, L., & Refuerzo, J. (2009). Full publication of clinical trials presented at a national maternal-fetal medicine meeting: Is there a publication bias? American Journal of Perinatology, 26, 679–682. doi:10.1055/s-0029-1220786.

    Article  PubMed  Google Scholar 

  • Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2005). Comprehensive meta-analysis (Version 2). Englewood: Biostat.

    Google Scholar 

  • Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester: Wiley.

    Book  Google Scholar 

  • Briner, R. B., & Rousseau, D. M. (2011). Evidence-based I-O psychology: Not there yet. Industrial and Organizational Psychology: Perspectives on Science and Practice, 4, 3–22. doi:10.1111/j.1754-9434.2010.01287.x.

    Article  Google Scholar 

  • Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. Journal of the American Medical Association, 291, 2457–2465. doi:10.1001/jama.291.20.2457.

    Article  PubMed  Google Scholar 

  • Cooper, H. (2003). Editorial. Psychological Bulletin, 129, 3–9. doi:10.1037/0033-2909.129.1.3.

    Article  Google Scholar 

  • Curfman, G. D., Morrissey, S., & Drazen, J. M. (2006). Expression of concern reaffirmed. New England Journal of Medicine, 354, 1193. doi:10.1056/NEJMe068054.

    Article  PubMed  Google Scholar 

  • Davis, M. S. (1971). That’s interesting! Towards a phenomenology of sociology and a sociology of phenomenology. Philosophy of the Social Sciences, 1, 309–344. doi:10.1177/004839317100100211.

    Article  Google Scholar 

  • Dickersin, K. (1990). The existence of publication bias and risk factors for its occurrence. Journal of the American Medical Association, 263, 1385–1389. doi:10.1001/jama.263.10.1385.

    Article  PubMed  Google Scholar 

  • Dickersin, K. (2005). Publication bias: Recognizing the problem, understandings its origins and scope, and preventing harm. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 11–34). West Sussex: Wiley.

    Google Scholar 

  • Duval, S. J. (2005). The “trim and fill” method. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 127–144). West Sussex: Wiley.

    Google Scholar 

  • Duval, S. J., & Tweedie, R. L. (2000a). A nonparametric “Trim and Fill” method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95, 89–98. doi:10.2307/2669529.

    Google Scholar 

  • Duval, S. J., & Tweedie, R. L. (2000b). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455–463. doi:10.1111/j.0006-341X.2000.00455.x.

    Article  PubMed  Google Scholar 

  • Dwan, K., et al. (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE, 3, e3081. doi:10.1371/journal.pone.00030.

    Article  PubMed Central  PubMed  Google Scholar 

  • Eagly, A. H., Johannesen-Schmidt, M. C., & van Engen, M. L. (2003). Transformational, transactional, and laissez-faire leadership styles: A meta-analysis comparing women and men. Psychological Bulletin, 129, 569–591. doi:10.1037/0033-2909.129.4.569.

    Article  PubMed  Google Scholar 

  • Egger, M., & Smith, G. D. (1998). Bias in location and selection of studies. British Medical Journal, 316, 61–66. doi:10.1136/bmj.316.7124.61.

    Article  PubMed Central  PubMed  Google Scholar 

  • Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634. doi:10.1136/bmj.315.7109.629.

    Article  PubMed Central  PubMed  Google Scholar 

  • Evangelou, E., Trikalinos, T. A., & Ioannidis, J. P. (2005). Unavailability of online supplementary scientific information from articles published in major journals. The FASEB Journal, 19, 1943–1944. doi:10.1096/fj.05-4784lsf.

    Article  Google Scholar 

  • Evans, S. (1996). Statistician’s comment (to Misleading meta-analysis: “Fail safe N” is a useful mathematical measure of the stability of results by R. Persaud). British Medical Journal, 312, 125.

    Article  PubMed Central  Google Scholar 

  • Ferguson, C. J., & Brannick, M. T. (2011). Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychological Methods, 17, 120–128. doi:10.1037/a0024445.

    Article  PubMed  Google Scholar 

  • Field, A. P., & Gillett, R. (2010). How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology, 63, 665–694. doi:10.1348/000711010X502733.

    Article  PubMed  Google Scholar 

  • Gigerenzer, G. (2004). Mindless statistics. Journal of Socio-Economics, 33, 587–606. doi:10.1016/j.socec.2004.09.033.

    Article  Google Scholar 

  • Greenhouse, J. B., & Iyengar, S. (2009). Sensitivity analysis and diagnostics. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 417–433). New York: Russell Sage Foundation.

    Google Scholar 

  • Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1–20. doi:10.1037/h0076157.

    Article  Google Scholar 

  • Hedges, L. V. (1992). Modeling publication selection effects in meta-analysis. Statistical Science, 7, 246–255. doi:10.1214/ss/1177011364.

    Article  Google Scholar 

  • Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic Press.

    Google Scholar 

  • Hedges, L. V., & Vevea, J. L. (2005). Selection methods approaches. In H. R. Rothstein, A. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 145–174). West Sussex: Wiley.

    Google Scholar 

  • Hermelin, E., Lievens, F., & Robertson, I. T. (2007). The validity of assessment centers for the prediction of supervisory performance ratings: A meta-analysis. International Journal of Selection and Assessment, 15, 405–411. doi:10.1111/j.1468-2389.2007.00399.x.

    Article  Google Scholar 

  • Higgins, J. P. T., & Green, S. (Eds.). (2009). Cochrane handbook for systematic reviews of interventions; Version 5.0.2 [updated September 2009]: The Cochrane Collaboration. www.cochrane-handbook.org.

  • Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539–1558. doi:10.1002/sim.1186.

    Article  PubMed  Google Scholar 

  • Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327, 557–560. doi:10.1136/bmj.327.7414.557.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hopewell, S., Clarke, M., & Mallett, S. (2005). Grey literature and systematic reviews. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 48–72). West Sussex: Wiley.

    Google Scholar 

  • Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A. D., & Dickersin, K. (2009). Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews. doi:10.1002/14651858.MR000006.pub3.

    Google Scholar 

  • Hopewell, S., McDonald, S., Clarke, M. J., & Egger, M. (2007). Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database of Systematic Reviews. doi:10.1002/14651858.MR000006.pub3.

    Google Scholar 

  • Hubbard, R., & Armstrong, J. S. (1997). Publication bias against null results. Psychological Reports, 80, 337–338. doi:10.2466/PR0.80.1.337-338.

    Article  Google Scholar 

  • Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park: Sage.

    Google Scholar 

  • Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. doi:10.1371/journal.pmed.0020124.

    Article  PubMed Central  PubMed  Google Scholar 

  • Jick, T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24, 602–611. doi:10.2307/2392366.

    Article  Google Scholar 

  • Kepes, S., Banks, G. C., McDaniel, M., & Whetzel, D. L. (2012). Publication bias in the organizational sciences. Organizational Research Methods,. doi:10.1177/1094428112452760.

    Google Scholar 

  • Kisamore, J. L., & Brannick, M. T. (2008). An illustration of the consequences of meta-analysis model choice. Organizational Research Methods, 11, 35–53. doi:10.1177/1094428106287393.

    Article  Google Scholar 

  • Le, H., Oh, I.-S., Shaffer, J., & Schmidt, F. L. (2007). Implications of methodological advances for the practice of personnel selection: How practitioners benefit from meta-analysis. Academy of Management Perspectives, 21, 6–15.

    Article  Google Scholar 

  • McDaniel, M. A. (2009, April). Cumulative meta-analysis as a publication bias method. Paper presented at the annual meeting of the Society for Industrial and Organizational Psychology, New Orleans, LA.

  • McDaniel, M. A., Rothstein, H. R., & Whetzel, D. L. (2006). Publication bias: A case study of four test vendors. Personnel Psychology, 59, 927–953. doi:10.1111/j.1744-6570.2006.00059.x.

    Article  Google Scholar 

  • McNatt, D. B. (2000). Ancient Pygmalion joins contemporary management: A meta-analysis of the result. Journal of Applied Psychology, 85, 314–322. doi:10.1037/0021-9010.85.2.314.

    Article  PubMed  Google Scholar 

  • Moreno, S. G., Sutton, A. J., Turner, E. H., Abrams, K. R., Cooper, N. J., Palmer, T. M., et al. (2009). Novel methods to deal with publication biases: Secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications. British Medical Journal, 339, b2981. doi:10.1136/bmj.b2981.

    Article  PubMed Central  PubMed  Google Scholar 

  • Palmer, T. M., Peters, J. L., Sutton, A. J., & Moreno, S. G. (2008). Contour-enhanced funnel plots for meta-analysis. Stata Journal, 8, 242–254.

    Google Scholar 

  • Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008). Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. Journal of Clinical Epidemiology, 61, 991–996. doi:10.1016/j.jclinepi.2007.11.010.

    Article  PubMed  Google Scholar 

  • Peters, J., Sutton, A., Jones, D. R., Abrams, K. R., Rushton, L., & Moreno, S. G. (2010). Assessing publication bias in meta-analyses in the presence of between-study heterogeneity. Journal of the Royal Statistical Society (Series A), 173, 575–591. doi:10.1111/j.1467-985X.2009.00629.x.

    Article  Google Scholar 

  • Quiñones, M. A., Ford, J. K., & Teachout, M. S. (1995). The relationship between work experience and job performance: A conceptual and meta-analytic review. Personnel Psychology, 48, 887–910. doi:10.1111/j.1744-6570.1995.tb01785.x.

    Article  Google Scholar 

  • Renkewitz, F., Fuchs, H. M., & Fiedler, S. (2011). Is there evidence of publication biases in JDM research? Judgment and Decision Making, 6, 870–881.

    Google Scholar 

  • Robbins, S., Oh, I.-S., Le, H., & Button, C. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: Integrated meta-analytic path-analysis. Journal of Applied Psychology, 94, 1163–1184. doi:10.1037/a0015738.

    Article  PubMed  Google Scholar 

  • Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638–641. doi:10.1037/0033-2909.86.3.638.

    Article  Google Scholar 

  • Rothstein, H. R. (2012). Accessing relevant literature. In H. Cooper (Ed.), APA handbook of research methods in psychology: Vol. 1. Foundations, planning, measures, and psychometrics (pp. 133–144). Washington, DC: American Psychological Association.

  • Rothstein, H. R., & Hopewell, S. (2009). Grey literature. In H. M. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 103–126). New York: Russell Sage Foundation.

    Google Scholar 

  • Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005a). Publication bias in meta-analyses. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 1–7). West Sussex: Wiley.

    Chapter  Google Scholar 

  • Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005b). Publication bias in meta-analysis: Prevention, assessment, and adjustments. West Sussex: Wiley.

    Book  Google Scholar 

  • Sackett, P. R., & Larson, J. R. (1990). Research strategies and tactics in industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 1, pp. 419–489). Palo Alto, CA: Consulting Psychologists Press.

    Google Scholar 

  • Scandura, T. A., & Williams, E. A. (2000). Research methodology in management: Current practices, trends, and implications for future research. The Academy of Management Journal, 43, 1248–1264. doi:10.2307/1556348.

    Article  Google Scholar 

  • Schmidt, F. L., & Le, H. (2005). Hunter & Schmidt’s Meta-analysis programs (Version 1.1). The University of Iowa, IA.

  • Schmidt, F. L., Oh, I.-S., & Hayes, T. (2009). Fixed versus random effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology, 62, 97–128. doi:10.1348/000711007X255327.

    Article  PubMed  Google Scholar 

  • Schopfel, J. (2006). Observations on the future of grey literature. Grey Journal, 2, 67–76.

    Google Scholar 

  • Schulze, R. (2004). Meta-analysis: A comparison of approaches. Cambridge: Hogrefe & Huber.

    Google Scholar 

  • Smith, G. D., & Egger, M. (1994). Who benefits from medical interventions? British Medical Journal, 308, 72–74. doi:10.1136/bmj.308.6921.72.

    Article  PubMed Central  PubMed  Google Scholar 

  • Song, F., Easterwood, A., Gilbody, S., Duley, L., & Sutton, A. J. (2000). Publication and other selection biases in systematic reviews. Health Technology, 4, 1–115. doi:10.3310/hta4100.

    Google Scholar 

  • Song, F., et al. (2010). Dissemination and publication of research findings: An updated review of related biases. Health Technology Assessment, 14, 1–220. doi:10.3310/hta14080.

    Google Scholar 

  • Sterling, T. D., & Rosenbaum, W. L. (1995). Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. American Statistician, 49, 108–112. doi:http://www.tandfonline.com/doi/abs/10.1080/00031305.1995.10476125.

    Google Scholar 

  • Sterne, J. A., & Egger, M. (2005). Regression methods to detect publication bias and other bias in meta-analysis. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 99–110). West Sussex: Wiley.

    Google Scholar 

  • Sterne, J. A., Gavaghan, D., & Egger, M. (2005). The funnel plot. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 75–98). West Sussex: Wiley.

    Google Scholar 

  • Sterne, J. A., et al. (2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. British Medical Journal, 342, d4002–d4010. doi:10.1136/bmj.d4002.

    Article  Google Scholar 

  • Sutton, A. J. (2005). Evidence concerning the consequences of publication and related biases. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 175–192). West Sussex: Wiley.

    Google Scholar 

  • Sutton, A. J. (2009). Publication bias. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 435–452). New York: Russell Sage Foundation.

    Google Scholar 

  • Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003). Adjusting for publication bias in the presence of heterogeneity. Statistics in Medicine, 22, 2113–2126. doi:10.1002/sim.1461.

    Article  PubMed  Google Scholar 

  • Torgerson, C. J. (2006). Publication bias: The Achilles’ heel of systematic reviews? British Journal of Educational Studies, 54, 89–102. doi:http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8527.2006.00332.x/abstract.

    Article  Google Scholar 

  • Trikalinos, T. A., & Ioannidis, J. P. A. (2005). Assessing the evolution of effect sizes over time. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 241–259). West Sussex: Wiley.

    Google Scholar 

  • Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine, 358, 252–260. doi:10.1056/NEJMsa065779.

    Article  PubMed  Google Scholar 

  • Vevea, J. L., & Woods, C. M. (2005). Publication bias in research synthesis: Sensitivity analysis using a priori weight functions. Psychological Methods, 10, 428–443. doi:10.1037/1082-989X.10.4.428.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management, School of Business, Virginia Commonwealth University, 301 West Main Street, PO Box 844000, Richmond, VA, 23284, USA

    Sven Kepes

  2. College of Business and Economics, Longwood University, Farmville, VA, USA

    George C. Banks

  3. Fox School of Business, Temple University, Philadelphia, PA, USA

    In-Sue Oh

Authors
  1. Sven Kepes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George C. Banks
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. In-Sue Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sven Kepes.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kepes, S., Banks, G.C. & Oh, IS. Avoiding Bias in Publication Bias Research: The Value of “Null” Findings. J Bus Psychol 29, 183–203 (2014). https://doi.org/10.1007/s10869-012-9279-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10869-012-9279-0

Keywords

Search

Navigation