Skip to main content
SpringerLink
Log in

Bias and Groupthink in Science’s Peer-Review System

  • Chapter
  • First Online:
Groupthink in Science

Abstract

Studies have shown that various types of biases can impact scientific peer review. These biases may contribute to a type of groupthink that can make it difficult to obtain funding or publish innovative or controversial research. The desire to achieve consensus and uniformity within a research group or scientific discipline can make it difficult for individuals to contradict the status quo. This chapter reviews the scientific literature regarding biases in the peer-review system, reflects on the potential impact of bias, and discusses approaches to minimize or control bias in peer review.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 99.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Our definition is loosely inspired by Irvin’s definition (1972) but has been modified so as to apply to the context of science and peer review.

References

  • Armstrong, J. S. (1997). Peer review for journals: Evidence on quality control, fairness, and innovation. Science and Engineering Ethics, 3(1), 63–84.

    Article  Google Scholar 

  • Baggs, J. G., Broome, M. E., Dougherty, M. C., Freda, M. C., & Kearney, M. H. (2008). Blinding in peer review: The preferences of reviewers for nursing journals. Journal of Advances in Nursing, 64(2), 131–138.

    Article  Google Scholar 

  • Barber, B. (1961). Resistance by scientists to scientific discovery. Science, 134(3479), 596–602.

    Article  Google Scholar 

  • Bornmann, L., Mutza, R., & Daniela, H. D. (2007). Gender differences in grant peer review: A meta-analysis. Journal of Informetrics, 1(3), 226–238.

    Article  Google Scholar 

  • Borsuk, R. M., Aarssen, L. W., Budden, A. E., Koricheva, J., Leimu, R., Tregenza, T., et al. (2009). To name or not to name: The effect of changing author gender on peer review. Bioscience, 59(11), 985–989.

    Article  Google Scholar 

  • Buchen, L. (2009). May 29, 1919: A major eclipse, relatively speaking. Wired, May 29, 2009. Available at: https://www.wired.com/2009/05/dayintech_0529/. Accessed 17 Apr 2017.

  • Campanario, J. M. (2009). Rejecting and resisting Nobel class discoveries: Accounts by Nobel Laureates. Scientometrics, 81(2), 549–565.

    Article  Google Scholar 

  • Card, R. F. (2005). Making sense of the diversity-based legal argument for affirmative action. Public Affairs Quarterly, 19(1), 11–24.

    Google Scholar 

  • Ceci, S. J., & Williams, W. M. (2011). Understanding current causes of women’s underrepresentation in science. Proceedings of the National Academy of Sciences of the United States of America, 108(8), 3157–3162.

    Article  Google Scholar 

  • Chubin, D., & Hackett, E. (1990). Peerless science: Peer review and U.S. science policy. Albany, NY: State University of New York Press.

    Google Scholar 

  • Committee on Science, Engineering, and Public Policy. (2010). Expanding underrepresented minority participation: America’s science and technology talent at the crossroads. Washington, DC: National Academies Press.

    Google Scholar 

  • Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & Reporting Bias Group. (2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias – An updated review. PLoS One, 8(7), e66844.

    Article  Google Scholar 

  • Easterbrook, P. J., Berlin, J. A., Gopalan, R., & Matthews, D. R. (1991). Publication bias in clinical research. Lancet, 337(8746), 867–872.

    Article  Google Scholar 

  • Ernst, E., & Kienbacher, T. (1991). Chauvinism. Nature, 352(6336), 560.

    Article  Google Scholar 

  • Fisher, M., Friedman, S. B., & Strauss, B. (1994). The effects of blinding on acceptance of research papers by peer review. Journal of the American Medical Association, 272(2), 143–146.

    Article  Google Scholar 

  • Garfield, E. (1989). Delayed recognition in scientific discovery: Citation frequency analyses aids the search for case histories. Current Contents, 23, 3–9.

    Google Scholar 

  • Garfunkel, J. M., Ulshen, M. H., Hamrick, H. J., & Lawson, E. E. (1994). Effect of institutional prestige on reviewers’ recommendations and editorial decisions. Journal of the American Medical Association, 272(2), 137–138.

    Article  Google Scholar 

  • Ginther, D. K., Haak, L. L., Schaffer, W. T., & Kington, R. (2012). Are race, ethnicity, and medical school affiliation associated with NIH R01 type 1 award probability for physician investigators? Academic Medicine, 87(11), 1516–1524.

    Article  Google Scholar 

  • Ginther, D. K., Kahn, S., & Schaffer, W. T. (2016). Gender, race/ethnicity, and national institutes of health r01 research awards: Is there evidence of a double bind for women of color? Academic Medicine, 91(8), 1098–1107.

    Article  Google Scholar 

  • Ginther, D. K., Schaffer, W., Schnel, J., Masimore, B., Liu, F., Haak, L. L., et al. (2011). Race, ethnicity, and NIH research awards. Science, 333(6045), 1015–1019.

    Article  Google Scholar 

  • Godlee, F., Gale, C. R., & Martyn, C. N. (1998). Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: A randomized controlled trial. Journal of the American Medical Association, 280(3), 237–240.

    Article  Google Scholar 

  • Grod, O. N., Budden, A. E., Tregenza, T., Koricheva, J., Leimu, R., Aarssen, L. W., et al. (2008). Systematic variation in reviewer practice according to country and gender in the field of ecology and evolution. PLoS One, 3(9), e3202.

    Article  Google Scholar 

  • Harding, S. (2004). Asocially relevant philosophy of science? Resources from standpoint theory’s controversiality. Hypatia, 19(1), 25–47.

    Article  Google Scholar 

  • Helmer, M., Schottdorf, M., Neef, A., & Battaglia, D. (2017). Gender bias in scholarly peer review. In: P. Rodgers (Ed.). eLife, 6, e21718.

    Article  Google Scholar 

  • Ho, R. C., Mak, K. K., Tao, R., Lu, Y., Day, J. R., & Pan, F. (2013). Views on the peer review system of biomedical journals: An online survey of academics from high-ranking universities. BMC Medical Research Methodology, 13, 74.

    Article  Google Scholar 

  • Hunter, J. (2012). Post-publication peer review: Opening up scientific conversation. Frontiers in Computational Neuroscience, 6, 63. (August 30).

    Article  Google Scholar 

  • Jang, J., Vannier, M. W., Wang, F., Deng, Y., Ou, F., Bennett, J., et al. (2013). A bibliometric analysis of academic publication and NIH funding. Journal of Informetrics, 7(2), 318–324.

    Article  Google Scholar 

  • Janis, I. L. (1972). Victims of groupthink: A psychological study of foreign policy decisions and fiascoes (2nd ed.). Boston: Houghton Mifflin Company.

    Google Scholar 

  • Janis, I. L. (1982). Groupthink: Psychological studies of policy decisions and fiascos. Boston: Cengage Learning.

    Google Scholar 

  • Justice, A. C., Cho, M. K., Winker, M. A., Berlin, J. A., & Rennie, D. (1998). Does masking author identity improve peer review quality? A randomized controlled trial. PEER investigators. Journal of the American Medical Association, 280(3), 240–242.

    Article  Google Scholar 

  • Kaatz, A., Lee, Y. G., Potvien, A., Magua, W., Filut, A., Bhattacharya, A., et al. (2016). Analysis of National Institutes of Health R01 application critiques, impact, and criteria scores: Does the sex of the principal investigator make a difference? Academic Medicine, 91(8), 1080–1088.

    Article  Google Scholar 

  • Kaatz, A., Magua, W., Zimmerman, D. R., & Carnes, M. (2015). A quantitative linguistic analysis of National Institutes of Health R01 application critiques from investigators at one institution. Academic Medicine, 90(1), 69–75.

    Article  Google Scholar 

  • Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press.

    Google Scholar 

  • Lamont, M. (2009). How professors think: Inside the curious world of academic judgment. Cambridge, MA: Harvard University Press.

    Book  Google Scholar 

  • Lane, J. A., & Linden, D. J. (2009). Is there gender bias in the peer review process at Journal of Neurophysiology? Journal of Neurophysiology, 101(5), 2195–2196.

    Article  Google Scholar 

  • Larivière, V., Ni, C., Gingras, Y., Cronin, B., & Sugimoto, C. (2013). Global gender disparities in science. Nature, 504(7479), 211–213.

    Article  Google Scholar 

  • Latour, B., & Woolgar, S. (1979). Laboratory life: The social construction of scientific facts. London: Sage.

    Google Scholar 

  • Lauer, M. S., Krumholz, H. M., & Topol, E. J. (2015). Time for a prepublication culture in clinical research? Lancet, 386(1012), 2447–2449.

    Article  Google Scholar 

  • Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2012). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.

    Article  Google Scholar 

  • Longino, H. (1990). Science as social knowledge. Princeton, NJ: Princeton University Press.

    Book  Google Scholar 

  • Lycan, W. G. (1988). Judgement and justification. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Mahoney, M. J. (1977). Publication preferences: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1(2), 161–175.

    Article  Google Scholar 

  • McNutt, R. A., Evans, A. T., Fletcher, R. H., & Fletcher, S. W. (1990). The effects of blinding on the quality of peer review. A randomized trial. Journal of the American Medical Association, 263(10), 1371–1376.

    Article  Google Scholar 

  • Murray, D. L., Morris, D., Lavoie, C., Leavitt, P. R., MacIsaac, H., Masson, M. E., et al. (2016). Bias in research grant evaluation has dire consequences for small universities. PLoS One, 11(6), e0155876.

    Article  Google Scholar 

  • Nelson, D. J. (2007). A national analysis of minorities in science and engineering faculties at research universities. Norman, OK: University of Oklahoma.

    Google Scholar 

  • Olson, C. M., Rennie, D., Cook, D., Dickersin, K., Flanagin, A., Hogan, J. W., et al. (2002). Publication bias in editorial decision making. Journal of the American Medical Association, 287(21), 2825–2828.

    Article  Google Scholar 

  • Planck, M. (1962). Quoted in Kuhn, T. S. 1962). The structure of scientific revolutions (pp. 33–34). Chicago: University of Chicago Press.

    Google Scholar 

  • Pulverer, B. (2016). Preparing for preprints. EMBO Journal, 35(24), 2617–2619.

    Article  Google Scholar 

  • Quine, W. V. (1961). From a logical point of view. New York: Harper and Rowe.

    Google Scholar 

  • Resch, K. I., Ernst, E., & Garrow, J. (2000). A randomized controlled study of reviewer bias against an unconventional therapy. Journal of the Royal Society of Medicine, 93(4), 164–167.

    Article  Google Scholar 

  • Resnik, D. B. (1994). Methodological conservatism and social epistemology. International Studies in the Philosophy of Science, 8(3), 247–264.

    Article  Google Scholar 

  • Resnik, D. B., & Elmore, S. A. (2016). Ensuring the quality, fairness, and integrity of journal peer review: A possible role for editors. Science and Engineering Ethics, 22(1), 169–188.

    Article  Google Scholar 

  • Resnik, D. B., Gutierrez-Ford, C., & Peddada, S. (2008). Perceptions of ethical problems with scientific journal peer review: An exploratory study. Science and Engineering Ethics, 14(3), 305–310.

    Article  Google Scholar 

  • Ross, J. S., Gross, C. P., Desai, M. M., Hong, Y., Grant, A. O., Daniels, S. R., et al. (2006). Effect of blinded peer review on abstract acceptance. Journal of the American Medical Association, 295(14), 1675–1680.

    Article  Google Scholar 

  • Shamoo, A. E., & Resnik, D. B. (2015). Responsible conduct of research (3rd ed.). New York: Oxford University Press.

    Google Scholar 

  • Shen, H. (2013). Mind the gender gap. Nature, 495(7439), 22–24.

    Article  Google Scholar 

  • Sklar, L. (1975). Methodological conservatism. Philosophical Review, 84(3), 374–400.

    Article  Google Scholar 

  • Smith, R. (2006). Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99(4), 178–182.

    Article  Google Scholar 

  • Stent, G. S. (1972). Prematurity and uniqueness in scientific discovery. Scientific American, 227(6), 84–93.

    Article  Google Scholar 

  • Stern, J. M., & Simes, R. J. (1997). Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. British Medical Journal, 315(7109), 640–645.

    Article  Google Scholar 

  • Teixeira da Silva, J. A., & Dobránszki, J. (2015). Problems with traditional science publishing and finding a wider niche for post-publication peer review. Accountability in Research, 22(1), 22–40.

    Article  Google Scholar 

  • Timmer, A., Hilsden, R. J., & Sutherland, L. R. (2001). Determinants of abstract acceptance for the Digestive Diseases Week--a cross sectional study. BMC Medical Research Methodology, 1, 13.

    Article  Google Scholar 

  • Tregenza, T. (2002). Gender bias in the refereeing process? Trends in Ecology & Evolution, 17(8), 349–350.

    Article  Google Scholar 

  • van Rooyen, S., Delamothe, T., & Evans, S. J. (2010). Effect on peer review of telling reviewers that their signed reviews might be posted on the web: Randomised controlled trial. British Medical Journal, 341, c5729.

    Article  Google Scholar 

  • van Rooyen, S., Godlee, F., Evans, S., Black, N., & Smith, R. (1999). Effect of open peer review on quality of reviews and on reviewers’ recommendations: A randomised trial. British Medical Journal, 318(7175), 23–27.

    Article  Google Scholar 

  • van Rooyen, S., Godlee, F., Evans, S., Smith, R., & Black, N. (1998). Effect of blinding and unmasking on the quality of peer review: A randomized trial. Journal of the American Medical Association, 280(3), 234–237.

    Article  Google Scholar 

  • Waisbren, S. E., Bowles, H., Hasan, T., Zou, K. H., Emans, S. J., Goldberg, C., et al. (2008). Gender differences in research grant applications and funding outcomes for medical school faculty. Journal of Women’s Health (Larchmont), 17(2), 207–214.

    Article  Google Scholar 

  • Walsh, E., Rooney, M., Appleby, L., & Wilkinson, G. (2000). Open peer review: A randomised controlled trial. British Journal of Psychiatry, 176, 47–51.

    Article  Google Scholar 

  • Wenneras, C., & Wold, A. (1997). Nepotism and sexism in peer-review. Nature, 387(6631), 341–343.

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported, in part, by the National Institute of Environmental Health Sciences (NIEHS), National Institutes of Health (NIH), and the Fonds de Recherche du Québec en Santé (FRQS). This paper does not represent the views of the NIEHS, NIH, the FRQS, or any governmental organization.

Author information

Authors and Affiliations

  1. National Institute of Environmental Health Sciences, National Institutes of Health, Research, Triangle Park, NC, USA

    David B. Resnik & Elise M. Smith

Authors
  1. David B. Resnik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elise M. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David B. Resnik .

Editor information

Editors and Affiliations

  1. Department of Psychiatry, University of Tennessee Health Science Center, Memphis, TN, USA

    David M. Allen  & James W. Howell  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2020 The Author(s)

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Resnik, D.B., Smith, E.M. (2020). Bias and Groupthink in Science’s Peer-Review System. In: Allen, D.M., Howell, J.W. (eds) Groupthink in Science. Springer, Cham. https://doi.org/10.1007/978-3-030-36822-7_9

Download citation

Publish with us

Policies and ethics

Search

Navigation