Should Biomedical Publishing Be “Opened Up”? Toward a Values-Based Peer-Review Process

Abstract

Peer review of manuscripts for biomedical journals has become a subject of intense ethical debate. One of the most contentious issues is whether or not peer review should be anonymous. This study aimed to generate a rich, empirically-grounded understanding of the values held by journal editors and peer reviewers with a view to informing journal policy. Qualitative methods were used to carry out an inductive analysis of biomedical reviewers’ and editors’ values. Data was derived from in-depth, open-ended interviews with journal editors and peer reviewers. Data was “read for” themes relevant to reviewer anonymisation and interactions among editors, reviewers, and authors. Editors and peer reviewers provided three arguments that would support a more open and interactive peer-review process. First, a number of participants emphasised the importance of not only ensuring the scientific quality of published research but also nurturing their colleagues and supporting their communities. Second, many spoke about the ongoing moral responsibilities that reviewers and editors felt toward authors. Finally, participants spoke at length about their enjoyment of social interactions and of the value of collective, rather than isolated, reasoning processes. Whether or not journal editors decide to allow anonymous review, the values of editors and reviewers need to be seriously addressed in codes of publication ethics, in the management of biomedical journals, and in the establishment of journal policies.

This is a preview of subscription content, access via your institution.

References

  1. Altman, L. 1996. The Ingelfinger rule, embargoes, and journal peer review—part 1. The Lancet 347(9012): 1382–1386.

    Article  CAS  Google Scholar 

  2. Atkinson, M. 1994. Regulation of science by peer review. Studies in History and Philosophy of Science 25(2): 147–158.

    Article  Google Scholar 

  3. Atkinson, M. 2001. “Peer Review” culture. Science and Engineering Ethics 7(2): 193–204.

    PubMed  Article  CAS  Google Scholar 

  4. Baez, B. 2002. Confidentiality and peer review: The paradox of secrecy in academe. Review of Higher Education 25(2): 163–183.

    Article  Google Scholar 

  5. Bingham, C. 2003. Peer review on the Internet: Are there faster, fairer, more effective methods of peer review? In Peer review in health sciences, ed. F. Godlee and T. Jefferson, 277–296. London: BMJ Books.

    Google Scholar 

  6. Bingham, C., and M. van der Weyden. 1998. Peer review on the Internet: Launching eMJA peer review study 2. The Medical Journal of Australia 169(5): 240–241.

    PubMed  CAS  Google Scholar 

  7. Callaham, M., and J. Tercier. 2007. The relationship of previous training and experience of journal peer reviewers to subsequent review quality. PLoS Medicine 4(1): e40.

    PubMed  Article  Google Scholar 

  8. Charmaz, K. 2006. Constructing grounded theory: A practical guide through qualitative analysis. London: Sage.

    Google Scholar 

  9. Committee on Publication Ethics. No date. http://www.publicationethics.org.uk (accessed June 27, 2010).

  10. Day, F.C., D.L. Schriger, C. Todd, and R.L. Wears. 2002. The use of dedicated methodology and statistical reviewers for peer review: A content analysis of comments to authors made by methodology and regular reviewers. Annals of Emergency Medicine 40(3): 329–333.

    PubMed  Article  Google Scholar 

  11. Dayton, A.I. 2006. Beyond open access: Open discourse, the next great equalizer. Retrovirology 3(55): doi:10.1186/1742-4690-3-55. http://www.retrovirology.com/content/3/1/55.

  12. Dickersin, K., E. Ssemanda, C. Mansell, and D. Rennie. 2007. What do the JAMA editors say when they discuss manuscripts that they are considering for publication? Developing a schema for classifying the content of editorial discussion. BMC Medical Research Methodology 7(44): doi:10.1186/1471-2288-7-44. http://www.biomedcentral.com/1471-2288/7/44.

  13. Fletcher, R.H., and S.W. Fletcher. 2003. The effectiveness of journal peer review. In Peer review in health sciences, ed. F. Godlee and T. Jefferson, 62–75. London: BMJ Books.

    Google Scholar 

  14. Flyvbjerg, B. 2001. Making social science matter: Why social inquiry fails and how it can succeed again. Oxford: Cambridge University Press.

    Google Scholar 

  15. Gannon, F. 2001. The essential role of peer review. EMBO Reports 2(9): 743.

    PubMed  Article  CAS  Google Scholar 

  16. Ginsparg, P. 2003. Alternatives to peer review II: Can peer review be better focused? In Peer review in health sciences, ed. F. Godlee and T. Jefferson, 312–321. London: BMJ Books.

    Google Scholar 

  17. Godlee, F. 2000. The ethics of peer review. In Ethical issues in biomedical publication, ed. A. Hudson Jones and F. McLellan, 59–84. Baltimore: Johns Hopkins University Press.

    Google Scholar 

  18. Godlee, F. 2002. Making reviewers visible: Openness, accountability, and credit. JAMA 287(21): 2762–2765.

    PubMed  Article  Google Scholar 

  19. Godlee, F., J. Smith, and H. Marcovitch. 2011. Wakefield’s article linking MMR vaccine and autism was fraudulent. British Medical Journal 342(7788): c7452.

    PubMed  Article  Google Scholar 

  20. Gosden, H. 2002. Thematic content in peer reviews of scientific papers. Seminários de Linguı́stica 5.

  21. Hojat, M., J. Gonnella, and A. Caelleigh. 2003. Impartial judgment by the “gatekeepers” of science: Fallibility and accountability in the peer review process. Advances in Health Sciences Education. Theory and Practice 8(1): 75–96.

    PubMed  Article  Google Scholar 

  22. International Committee of Medical Journal Editors. 2007. Uniform requirements for manuscripts submitted to biomedical journals: Writing and editing for biomedical publication. http://www.icmje.org/ (accessed June 27, 2010).

  23. Jefferson, T., M. Rudin, S. Brodney Folse, and F. Davidoff. 2007. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database of Systematic Reviews 18(2): MR000016.

    Google Scholar 

  24. Judson, H. 1994. Structural transformations of the sciences and the end of peer review. JAMA 272(2): 92–94.

    PubMed  Article  CAS  Google Scholar 

  25. Kearney, M., and M. Freda. 2005. Nurse editors’ views on the peer review process. Research in Nursing & Health 28(6): 444–452.

    Article  Google Scholar 

  26. Kumashiro, K. 2005. Thinking collaboratively about the peer-review process for journal-article publication. Harvard Educational Review 75(3): 257–285.

    Google Scholar 

  27. Lipworth, W., and I. Kerridge. 2011. Shifting power relations and the ethics of journal peer review. Social Epistemology 25(1): 97–121.

    Article  Google Scholar 

  28. Lipworth, W.L., I.H. Kerridge, S.M. Carter, and M. Little. 2011. Journal peer review in context: A qualitative study of the social and subjective dimensions of manuscript review in biomedical publishing. Social Science and Medicine 72(7): 1056–1063. doi:10.1016/j.socscimed.2011.02.00.

    PubMed  Article  Google Scholar 

  29. McLellan, F., and P. Riis. 2003. Ethical conduct for reviewers of grant applications and manuscripts. In Peer review in health sciences, ed. F. Godlee and T. Jefferson, 136–147. London: BMJ Books.

    Google Scholar 

  30. McNutt, R., A. Evans, R. Fletcher, and S. Fletcher. 1990. The effects of blinding on the quality of peer review. A randomized trial. JAMA 263(10): 1371–1376.

    PubMed  Article  CAS  Google Scholar 

  31. Morse, J.M. 1994. “Emerging from the data”: The cognitive processes of analysis in qualitative inquiry. In Critical issues in qualitative research methods, ed. J.M. Morse, 23–42. Thousand Oaks: Sage.

    Google Scholar 

  32. Mruck, K., and G. Mey. 2002. Between printed past and digital future. Research in Science Education 32(2): 257–268.

    Article  Google Scholar 

  33. Osmond, D. 1983. Malice’s wonderland: Research funding and peer review. Journal of Neurobiology 14(2): 95–112.

    PubMed  Article  CAS  Google Scholar 

  34. Regehr, G., and G. Bordage. 2006. To blind or not to blind? What authors and reviewers prefer. Medical Education 40(9): 832–839.

    PubMed  Article  Google Scholar 

  35. Rennie, D. 1998. Freedom and responsibility in medical publication: Setting the balance right. JAMA 280(3): 300–302.

    PubMed  Article  CAS  Google Scholar 

  36. Rennie, D. 2003a. Editorial peer review: Its development and rationale. In Peer review in health sciences, ed. F. Godlee and T. Jefferson, 1–13. London: BMJ Books.

    Google Scholar 

  37. Rennie, D. 2003b. Misconduct and journal peer review. In Peer review in health sciences, ed. F. Godlee and T. Jefferson, 118–129. London: BMJ Books.

    Google Scholar 

  38. Shiell, A., P. Hawe, and J. Seymour. 1997. Values and preferences are not necessarily the same. Health Economics 6(5): 515–518.

    PubMed  Article  CAS  Google Scholar 

  39. Smith, R. 1997. Peer review: Reform or revolution? British Medical Journal 315(7111): 759–760.

    PubMed  CAS  Google Scholar 

  40. Smith, R. 2003. The future of peer review. In Peer review in health sciences, ed. F. Godlee and T. Jefferson, 329–346. London: BMJ Books.

    Google Scholar 

  41. Smith, R. 2006. The trouble with medical journals. London: Royal Society of Medicine Press.

    Google Scholar 

  42. Snell, L., and J. Spencer. 2005. Reviewers’ perceptions of the peer review process for a medical education journal. Medical Education 39(1): 90–97.

    PubMed  Article  Google Scholar 

  43. Tercier, J., and M. Callaham. 2007a. A normative model of peer review: Qualitative assessment of manuscript reviewers’ attitudes towards peer review. http://escholarship.org/uc/item/4p90p67x (accessed April 2010).

  44. Tercier, J., and M. Callaham. 2007b. Peer review: Consensus and contradiction, a qualitative approach. http://www.escholarship.org/uc/item/2s42m2kw (accessed April 2010).

  45. van Rooyen, S., F. Godlee, S. Evans, N. Black, and R. Smith. 1999. Effect of open peer review on quality of reviews and on reviewers’ recommendations: A randomised trial. British Medical Journal 318(7175): 23–27.

    PubMed  Google Scholar 

  46. van Rooyen, S., F. Godlee, S. Evans, R. Smith, and N. Black. 1998. Effect of blinding and unmasking on the quality of peer review: A randomized trial. JAMA 280(3): 234–237.

    PubMed  Article  Google Scholar 

  47. Wager, E., and A. Herxheimer. 2003. Peer review and the pharmaceutical industry. In Peer review in health sciences, ed. F. Godlee and T. Jefferson, 130–139. London: BMJ Books.

    Google Scholar 

  48. Walsh, E., M. Rooney, L. Appleby, and G. Wilkinson. 2000. Open peer review: A randomised controlled trial. The British Journal of Psychiatry 17(1): 647–651.

    Google Scholar 

  49. World Association of Medical Editors. No date. http://www.wame.org (accessed June 27, 2010).

Download references

Disclosures

The work described here has not been published before, is not under consideration for publication anywhere else, and has been approved by all co-authors. All authors contributed to the conception and design of the study, to data interpretation, and to the writing of the article. The authors declare no competing interests.

This study was funded by a National Health and Medical Research Council (NHMRC) postgraduate scholarship. The NHMRC played no role in the research or the preparation of this article.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Wendy Lipworth.

Additional information

This article was part of a postgraduate research project funded by the National Health and Medical Research Council.

Electronic supplementary materials

Below is the link to the electronic supplementary material.

ESM 1

(DOC 80 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lipworth, W., Kerridge, I.H., Carter, S.M. et al. Should Biomedical Publishing Be “Opened Up”? Toward a Values-Based Peer-Review Process. Bioethical Inquiry 8, 267–280 (2011). https://doi.org/10.1007/s11673-011-9312-4

Download citation

Keywords

  • Peer review
  • Social values
  • Qualitative research
  • Research ethics
  • Bioethics: Medical ethics
  • Research (humans)