Advertisement

Preserving the Quality of Scientific Research: Peer Review of Research Articles

  • Pali U. K. De SilvaEmail author
  • Candace K. Vance
Chapter
Part of the Fascinating Life Sciences book series (FLS)

Abstract

Peer review of scholarly articles is a mechanism used to assess and preserve the trustworthiness of reporting of scientific findings. Since peer reviewing is a qualitative evaluation system that involves the judgment of experts in a field about the quality of research performed by their colleagues (and competitors), it inherently encompasses a strongly subjective element. Although this time-tested system, which has been evolving since the mid-eighteenth century, is being questioned and criticized for its deficiencies, it is still considered an integral part of the scholarly communication system, as no other procedure has been proposed to replace it. Therefore, to improve and strengthen the existing peer review process, it is important to understand its shortcomings and to continue the constructive deliberations of all participants within the scientific scholarly communication system . This chapter discusses the strengths, issues, and deficiencies of the peer review system, conventional closed models (single-blind and double-blind), and the new open peer review model and its variations that are being experimented with by some journals.

Keywords

Article peer review system Closed peer review Open peer review Scientific journal publishing Single blind peer reviewing Article retraction Nonselective review Post-publication review system Double blind peer reviewing 

References

  1. Alberts, B., Hanson, B., & Kelner, K. L. (2008). Reviewing peer review. Science, 321(5885), 15.CrossRefPubMedGoogle Scholar
  2. Ali, P. A., & Watson, R. (2016). Peer review and the publication process. Nursing Open. doi:  10.1002/nop2.51.
  3. Baggs, J. G., Broome, M. E., Dougherty, M. C., Freda, M. C., & Kearney, M. H. (2008). Blinding in peer review: the preferences of reviewers for nursing journals. Journal of Advanced Nursing, 64(2), 131–138.CrossRefPubMedGoogle Scholar
  4. Bjork, B.-C., Roos, A., & Lauri, M. (2009). Scientific journal publishing: yearly volume and open access availability. Information Research: An International Electronic Journal, 14(1).Google Scholar
  5. Bohannon, J. (2013). Who’s afraid of peer review. Science, 342(6154).Google Scholar
  6. Boldt, A. (2011). Extending ArXiv. org to achieve open peer review and publishing. Journal of Scholarly Publishing, 42(2), 238–242.CrossRefGoogle Scholar
  7. Bombardier, C., Laine, L., Reicin, A., Shapiro, D., Burgos-Vargas, R., Davis, B., … & Kvien, T. K. (2000). Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis. New England Journal of Medicine, 343(21), 1520–1528Google Scholar
  8. Bornmann, L. (2011). Scientific peer review. Annual Review of Information Science and Technology, 45(1), 197–245.CrossRefGoogle Scholar
  9. Bornmann, L., & Daniel, H.-D. (2009). Reviewer and editor biases in journal peer review: An investigation of manuscript refereeing at Angewandte Chemie International Edition. Research Evaluation, 18(4), 262–272.CrossRefGoogle Scholar
  10. Bornmann, L., & Daniel, H. D. (2010). Reliability of reviewers’ ratings when using public peer review: A case study. Learned Publishing, 23(2), 124–131.CrossRefGoogle Scholar
  11. Bornmann, L., Mutz, R., & Daniel, H.-D. (2007). Gender differences in grant peer review: A meta-analysis. Journal of Informetrics, 1(3), 226–238.CrossRefGoogle Scholar
  12. Borsuk, R. M., Aarssen, L. W., Budden, A. E., Koricheva, J., Leimu, R., Tregenza, T., et al. (2009). To name or not to name: The effect of changing author gender on peer review. BioScience, 59(11), 985–989.CrossRefGoogle Scholar
  13. Bosch, X., Pericas, J. M., Hernández, C., & Doti, P. (2013). Financial, nonfinancial and editors’ conflicts of interest in high-impact biomedical journals. European Journal of Clinical Investigation, 43(7), 660–667.CrossRefPubMedGoogle Scholar
  14. Brown, R. J. C. (2007). Double anonymity in peer review within the chemistry periodicals community. Learned Publishing, 20(2), 131–137.CrossRefGoogle Scholar
  15. Budden, A. E., Tregenza, T., Aarssen, L. W., Koricheva, J., Leimu, R., & Lortie, C. J. (2008). Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution, 23(1), 4–6.CrossRefGoogle Scholar
  16. Burnham, J. C. (1990). The evolution of editorial peer review. JAMA, 263(10), 1323–1329.CrossRefPubMedGoogle Scholar
  17. Callaham, M. L., & Tercier, J. (2007). The relationship of previous training and experience of journal peer reviewers to subsequent review quality. PLoS Med, 4(1), e40.CrossRefPubMedPubMedCentralGoogle Scholar
  18. Campbell, P. (2006). Peer Review Trial and Debate. Naturehttp://www.nature.com/nature/peerreview/debate/
  19. Campbell, P. (2008). Nature peer review trial and debate. Nature: International Weekly Journal of Science, 11 Google Scholar
  20. Campos-Arceiz, A., Primack, R. B., & Koh, L. P. (2015). Reviewer recommendations and editors’ decisions for a conservation journal: Is it just a crapshoot? And do Chinese authors get a fair shot? Biological Conservation, 186, 22–27.CrossRefGoogle Scholar
  21. Cantor, M., & Gero, S. (2015). The missing metric: Quantifying contributions of reviewers. Royal Society open science, 2(2), 140540.CrossRefPubMedPubMedCentralGoogle Scholar
  22. CDC. (2016). Measles: Cases and Outbreaks. Retrieved from http://www.cdc.gov/measles/cases-outbreaks.html
  23. Ceci, S. J., & Williams, W. M. (2011). Understanding current causes of women’s underrepresentation in science. Proceedings of the National Academy of Sciences, 108(8), 3157–3162.CrossRefGoogle Scholar
  24. Chan, A. W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA, 291(20), 2457–2465.CrossRefPubMedGoogle Scholar
  25. Charlton, B. G. (2004). Conflicts of interest in medical science: peer usage, peer review andCoI consultancy’. Medical Hypotheses, 63(2), 181–186.CrossRefPubMedGoogle Scholar
  26. Cressey, D. (2014). Journals weigh up double-blind peer review. Nature news.Google Scholar
  27. Dalton, R. (2001). Peers under pressure. Nature, 413(6852), 102–104.CrossRefPubMedGoogle Scholar
  28. DeVries, D. R., Marschall, E. A., & Stein, R. A. (2009). Exploring the peer review process: What is it, does it work, and can it be improved? Fisheries, 34(6), 270–279. doi: 10.1577/1548-8446-34.6.270 CrossRefGoogle Scholar
  29. Emerson, G. B., Warme, W. J., Wolf, F. M., Heckman, J. D., Brand, R. A., & Leopold, S. S. (2010). Testing for the presence of positive-outcome bias in peer review: A randomized controlled trial. Archives of Internal Medicine, 170(21), 1934–1939.CrossRefPubMedGoogle Scholar
  30. Fanelli, D. (2010). Do pressures to publish increase scientists’ bias? An empirical support from US States Data. PLoS ONE, 5(4), e10271.CrossRefPubMedPubMedCentralGoogle Scholar
  31. Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 109(42), 17028–17033. doi: 10.1073/pnas.1212247109 CrossRefGoogle Scholar
  32. Ferguson, C., Marcus, A., & Oransky, I. (2014). Publishing: The peer-review scam. Nature, 515(7528), 480.CrossRefPubMedGoogle Scholar
  33. Ford, E. (2015). Open peer review at four STEM journals: An observational overview. F1000Research, 4.Google Scholar
  34. Fountain, H. (2014). Science journal pulls 60 papers in peer-review fraud. Science, 3, 06.Google Scholar
  35. Freda, M. C., Kearney, M. H., Baggs, J. G., Broome, M. E., & Dougherty, M. (2009). Peer reviewer training and editor support: Results from an international survey of nursing peer reviewers. Journal of Professional Nursing, 25(2), 101–108.CrossRefPubMedGoogle Scholar
  36. Gillespie, G. W., Chubin, D. E., & Kurzon, G. M. (1985). Experience with NIH peer review: Researchers’ cynicism and desire for change. Science, Technology and Human Values, 10(3), 44–54.CrossRefGoogle Scholar
  37. Greaves, S., Scott, J., Clarke, M., Miller, L., Hannay, T., Thomas, A., et al. (2006). Overview: Nature’s peer review trial. Nature, 10. Google Scholar
  38. Grieneisen, M. L., & Zhang, M. (2012). A comprehensive survey of retracted articles from the scholarly literature. PLoS ONE, 7(10), e44118.CrossRefPubMedPubMedCentralGoogle Scholar
  39. Grivell, L. (2006). Through a glass darkly. EMBO Reports, 7(6), 567–570.CrossRefPubMedPubMedCentralGoogle Scholar
  40. Harrison, C. (2004). Peer review, politics and pluralism. Environmental Science & Policy, 7(5), 357–368.CrossRefGoogle Scholar
  41. Hartog, C. S., Kohl, M., & Reinhart, K. (2011). A systematic review of third-generation hydroxyethyl starch (HES 130/0.4) in resuscitation: Safety not adequately addressed. Anesthesia and Analgesia, 112(3), 635–645.CrossRefPubMedGoogle Scholar
  42. Hojat, M., Gonnella, J. S., & Caelleigh, A. S. (2003). Impartial judgment by the “gatekeepers” of science: Fallibility and accountability in the peer review process. Advances in Health Sciences Education, 8(1), 75–96.CrossRefPubMedGoogle Scholar
  43. Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Med, 2(8), e124.CrossRefPubMedPubMedCentralGoogle Scholar
  44. James, M. J., Cook-Johnson, R. J., & Cleland, L. G. (2007). Selective COX-2 inhibitors, eicosanoid synthesis and clinical outcomes: A case study of system failure. Lipids, 42(9), 779–785.CrossRefPubMedGoogle Scholar
  45. Janssen, S. J., Bredenoord, A. L., Dhert, W., de Kleuver, M., Oner, F. C., & Verlaan, J.-J. (2015). Potential conflicts of interest of editorial board members from five leading spine journals. PLoS ONE, 10(6), e0127362.CrossRefPubMedPubMedCentralGoogle Scholar
  46. Jefferson, T., Alderson, P., Wager, E., & Davidoff, F. (2002). Effects of editorial peer review: A systematic review. JAMA, 287(21), 2784–2786.CrossRefPubMedGoogle Scholar
  47. Jelicic, M., & Merckelbach, H. (2002). Peer-review: Let’s imitate the lawyers! Cortex, 38(3), 406–407.CrossRefPubMedGoogle Scholar
  48. Jinha, A. E. (2010). Article 50 million: An estimate of the number of scholarly articles in existence. Learned Publishing, 23(3), 258–263.CrossRefGoogle Scholar
  49. Khan, K. (2010). Is open peer review the fairest system? No. Bmj, 341, c6425.CrossRefGoogle Scholar
  50. Kilwein, J. H. (1999). Biases in medical literature. Journal of Clinical Pharmacy and Therapeutics, 24(6), 393–396.CrossRefPubMedGoogle Scholar
  51. Koonin, E. V., Landweber, L. F., & Lipman, D. J. (2013). Biology direct: Celebrating 7 years of open, published peer review. Biology direct, 8(1), 1.CrossRefGoogle Scholar
  52. Kozlowski, L. T. (2016). Coping with the conflict-of-interest pandemic by listening to and doubting everyone, including yourself. Science and Engineering Ethics, 22(2), 591–596.CrossRefPubMedGoogle Scholar
  53. Krebs, H. A., & Johnson, W. A. (1937). The role of citric acid in intermediate metabolism in animal tissues. Enzymologia, 4, 148–156.Google Scholar
  54. Kriegeskorte, N., Walther, A., & Deca, D. (2012). An emerging consensus for open evaluation: 18 visions for the future of scientific publishing. Beyond open access: Visions for open evaluation of scientific papers by post-publication peer review, 5.Google Scholar
  55. Langfeldt, L. (2006). The policy challenges of peer review: Managing bias, conflict of interests and interdisciplinary assessments. Research Evaluation, 15(1), 31–41.CrossRefGoogle Scholar
  56. Lawrence, P. A. (2003). The politics of publication. Nature, 422(6929), 259–261.CrossRefPubMedGoogle Scholar
  57. Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.CrossRefGoogle Scholar
  58. Lippert, S., Callaham, M. L., & Lo, B. (2011). Perceptions of conflict of interest disclosures among peer reviewers. PLoS ONE, 6(11), e26900.CrossRefPubMedPubMedCentralGoogle Scholar
  59. Link, A. M. (1998). US and non-US submissions: an analysis of reviewer bias. Jama, 280(3), 246–247. Google Scholar
  60. Lo, B., & Field, M. J. (Eds.). (2009). Conflict of interest in medical research, education, and practice. Washington, D.C.: National Academies Press.Google Scholar
  61. Loonen, M. P. J., Hage, J. J., & Kon, M. (2005). Who benefits from peer review? An analysis of the outcome of 100 requests for review by Plastic and Reconstructive Surgery. Plastic and Reconstructive Surgery, 116(5), 1461–1472.CrossRefPubMedGoogle Scholar
  62. Luukkonen, T. (2012). Conservatism and risk-taking in peer review: Emerging ERC practices. Research Evaluation, rvs001.Google Scholar
  63. McClintock, B. (1950). The origin and behavior of mutable loci in maize. Proceedings of the National Academy of Sciences, 36(6), 344–355.CrossRefGoogle Scholar
  64. McCullough, J. (1989). First comprehensive survey of NSF applicants focuses on their concerns about proposal review. Science, Technology and Human Values, 14(1), 78–88.CrossRefGoogle Scholar
  65. McIntyre, W. F., & Evans, G. (2014). The Vioxx® legacy: Enduring lessons from the not so distant past. Cardiology Journal, 21(2), 203–205.CrossRefPubMedGoogle Scholar
  66. Moylan, E. C., Harold, S., O’Neill, C., & Kowalczuk, M. K. (2014). Open, single-blind, double-blind: Which peer review process do you prefer? BMC Pharmacology and Toxicology, 15(1), 1.CrossRefGoogle Scholar
  67. Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing world: An international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology, 64(1), 132–161.CrossRefGoogle Scholar
  68. Nath, S. B., Marcus, S. C., & Druss, B. G. (2006). Retractions in the research literature: misconduct or mistakes? Medical Journal of Australia, 185(3), 152.PubMedGoogle Scholar
  69. Nature Editorial (2008). Working double-blind. Nature, 451, 605–606.Google Scholar
  70. Nature Neuroscience Editorial. (2006). Women in neuroscience: A numbers game. Nature Neuroscience, 9, 853.CrossRefGoogle Scholar
  71. Okike, K., Hug, K. T., Kocher, M. S., & Leopold, S. S. (2016). Single-blind vs double-blind peer review in the setting of author prestige. JAMA, 316(12), 1315–1316.CrossRefPubMedGoogle Scholar
  72. Olson, C. M., Rennie, D., Cook, D., Dickersin, K., Flanagin, A., Hogan, J. W., … & Pace, B. (2002). Publication bias in editorial decision making. JAMA, 287(21), 2825–2828.Google Scholar
  73. Palmer, A. R. (2000). Quasireplication and the contract of error: lessons from sex ratios, heritabilities and fluctuating asymmetry. Annual Review of Ecology and Systematics, 441–480.Google Scholar
  74. Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences, 5(02), 187–195.CrossRefGoogle Scholar
  75. PLOS MED Editors. (2008). Making sense of non-financial competing interests. PLOS Med, 5(9), e199.Google Scholar
  76. Pulverer, B. (2010). Transparency showcases strength of peer review. Nature, 468(7320), 29–31.CrossRefPubMedGoogle Scholar
  77. Pöschl, U., & Koop, T. (2008). Interactive open access publishing and collaborative peer review for improved scientific communication and quality assurance. Information Services & Use, 28(2), 105–107.Google Scholar
  78. Relman, A. S. (1985). Dealing with conflicts of interest. New England Journal of Medicine, 313(12), 749–751.CrossRefPubMedGoogle Scholar
  79. Rennie, J., & Chief, I. N. (2002). Misleading math about the Earth. Scientific American, 286(1), 61.CrossRefGoogle Scholar
  80. Resch, K. I., Ernst, E., & Garrow, J. (2000). A randomized controlled study of reviewer bias against an unconventional therapy. Journal of the Royal Society of Medicine, 93(4), 164–167.PubMedPubMedCentralGoogle Scholar
  81. Resnik, D. B., & Elmore, S. A. (2016). Ensuring the quality, fairness, and integrity of journal peer review: A possible role of editors. Science and Engineering Ethics, 22(1), 169–188.CrossRefPubMedGoogle Scholar
  82. Ross, J. S., Gross, C. P., Desai, M. M., Hong, Y., Grant, A. O., Daniels, S. R., et al. (2006). Effect of blinded peer review on abstract acceptance. JAMA, 295(14), 1675–1680.CrossRefPubMedGoogle Scholar
  83. Sandström, U. (2009, BRAZIL. JUL 14-17, 2009). Cognitive bias in peer review: A new approach. Paper presented at the 12th International Conference of the International-Society-for-Scientometrics-and-Informetrics. Google Scholar
  84. Shatz, D. (2004). Peer review: A critical inquiry. Lanham, MD: Rowman & Littlefield.Google Scholar
  85. Schneider, L. (2016, September 4). Beall-listed Frontiers empire strikes back. Retrieved from https://forbetterscience.wordpress.com/2016/09/14/beall-listed-frontiers-empire-strikes-back/
  86. Schroter, S., Black, N., Evans, S., Carpenter, J., Godlee, F., & Smith, R. (2004). Effects of training on quality of peer review: Randomised controlled trial. BMJ, 328(7441), 673.CrossRefPubMedPubMedCentralGoogle Scholar
  87. Service, R. F. (2002). Scientific misconduct. Bell Labs fires star physicist found guilty of forging data. Science (New York, NY), 298(5591), 30.CrossRefGoogle Scholar
  88. Shimp, C. P. (2004). Scientific peer review: A case study from local and global analyses. Journal of the Experimental Analysis of Behavior, 82(1), 103–116.CrossRefPubMedCentralGoogle Scholar
  89. Smith, R. (1999). Opening up BMJ peer review: A beginning that should lead to complete transparency. BMJ, 318, 4–5.CrossRefPubMedPubMedCentralGoogle Scholar
  90. Smith, R. (2006). Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99(4), 178–182. doi: 10.1258/jrsm.99.4.178 CrossRefPubMedPubMedCentralGoogle Scholar
  91. Souder, L. (2011). The ethics of scholarly peer review: A review of the literature. Learned Publishing, 24(1), 55–72.CrossRefGoogle Scholar
  92. Spielmans, G. I., Biehn, T. L., & Sawrey, D. L. (2009). A case study of salami slicing: pooled analyses of duloxetine for depression. Psychotherapy and Psychosomatics, 79(2), 97–106.CrossRefPubMedGoogle Scholar
  93. Spier, R. (2002). The history of the peer-review process. Trends in Biotechnology, 20(8), 357–358.CrossRefPubMedGoogle Scholar
  94. Squazzoni, F. (2010). Peering into peer review. Sociologica, 4(3).Google Scholar
  95. Squazzoni, F., & Gandelli, C. (2012). Saint Matthew strikes again: An agent-based model of peer review and the scientific community structure. Journal of Informetrics, 6(2), 265–275.CrossRefGoogle Scholar
  96. Steen, R. G. (2010). Retractions in the scientific literature: is the incidence of research fraud increasing? Journal of Medical Ethics, jme-2010.Google Scholar
  97. Stroebe, W., Postmes, T., & Spears, R. (2012). Scientific misconduct and the myth of self-correction in science. Perspectives on Psychological Science, 7(6), 670–688.CrossRefPubMedGoogle Scholar
  98. Tite, L., & Schroter, S. (2007). Why do peer reviewers decline to review? A survey. Journal of Epidemiology and Community Health, 61(1), 9–12.CrossRefPubMedPubMedCentralGoogle Scholar
  99. Travis, G. D. L., & Collins, H. M. (1991). New light on old boys: cognitive and institutional particularism in the peer review system. Science, Technology and Human Values, 16(3), 322–341.CrossRefGoogle Scholar
  100. Tregenza, T. (2002). Gender bias in the refereeing process? Trends in Ecology & Evolution, 17(8), 349–350.CrossRefGoogle Scholar
  101. Valkonen, L., & Brooks, J. (2011). Gender balance in Cortex acceptance rates. Cortex, 47(7), 763–770.CrossRefPubMedGoogle Scholar
  102. van Rooyen, S., Delamothe, T., & Evans, S. J. W. (2010). Effect on peer review of telling reviewers that their signed reviews might be posted on the web: Randomised controlled trial. BMJ, 341, c5729.CrossRefPubMedPubMedCentralGoogle Scholar
  103. van Rooyen, S., Godlee, F., Evans, S., Black, N., & Smith, R. (1999). Effect of open peer review on quality of reviews and on reviewers’ recommendations: A randomised trial. British Medical Journal, 318(7175), 23–27.CrossRefPubMedPubMedCentralGoogle Scholar
  104. Walker, R., & Rocha da Silva, P. (2014). Emerging trends in peer review—A survey. Frontiers in neuroscience, 9, 169.Google Scholar
  105. Walsh, E., Rooney, M., Appleby, L., & Wilkinson, G. (2000). Open peer review: A randomised controlled trial. The British Journal of Psychiatry, 176(1), 47–51.CrossRefPubMedGoogle Scholar
  106. Walters, W. P., & Bajorath, J. (2015). On the evolving open peer review culture for chemical information science. F1000Research, 4.Google Scholar
  107. Ware, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community-Results from an international study. Information Services and Use, 28(2), 109–112.Google Scholar
  108. Ware, M. (2011). Peer review: Recent experience and future directions. New Review of Information Networking, 16(1), 23–53.Google Scholar
  109. Webb, T. J., O’Hara, B., & Freckleton, R. P. (2008). Does double-blind review benefit female authors? Heredity, 77, 282–291.Google Scholar
  110. Wellington, J., & Nixon, J. (2005). Shaping the field: The role of academic journal editors in the construction of education as a field of study. British Journal of Sociology of Education, 26(5), 643–655.CrossRefGoogle Scholar
  111. Whittaker, R. J. (2008). Journal review and gender equality: A critical comment on Budden et al. Trends in Ecology & Evolution, 23(9), 478–479.Google Scholar
  112. Wiedermann, C. J. (2016). Ethical publishing in intensive care medicine: A narrative review. World Journal of Critical Care Medicine, 5(3), 171.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Murray State UniversityMurrayUSA

Personalised recommendations