Peer review versus editorial review and their role in innovative science

Abstract

Peer review is a widely accepted instrument for raising the quality of science. Peer review limits the enormous unstructured influx of information and the sheer amount of dubious data, which in its absence would plunge science into chaos. In particular, peer review offers the benefit of eliminating papers that suffer from poor craftsmanship or methodological shortcomings, especially in the experimental sciences. However, we believe that peer review is not always appropriate for the evaluation of controversial hypothetical science. We argue that the process of peer review can be prone to bias towards ideas that affirm the prior convictions of reviewers and against innovation and radical new ideas. Innovative hypotheses are thus highly vulnerable to being “filtered out” or made to accord with conventional wisdom by the peer review process. Consequently, having introduced peer review, the Elsevier journal Medical Hypotheses may be unable to continue its tradition as a radical journal allowing discussion of improbable or unconventional ideas. Hence we conclude by asking the publisher to consider re-introducing the system of editorial review to Medical Hypotheses.

This is a preview of subscription content, log in to check access.

Fig. 1

Notes

  1. 1.

    Of course the system of peer review also includes some editorial review. The final responsibility for acceptance or rejection lies with the editor in peer reviewed journals as well. This is why decision letters come from editors, not the peer reviewers.

References

  1. 1.

    Galilei, G. 1632. Dialogo … sopra i due massimi sistemi del mondo. Florence: Gio:Batista Landini.

    Google Scholar 

  2. 2.

    Sobel, D. 2000. Galileo’s daughter: a historical memoir of science, faith, and love. New York: Penguin Books.

    Google Scholar 

  3. 3.

    Charlton, B.G. 2004. Conflicts of interest in medical science: peer usage, peer review and ‘CoI consultancy’. Medical Hypotheses 63: 181–186.

    Article  Google Scholar 

  4. 4.

    Rothwell, P.M., and C.N. Martyn. 2000. Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? Brain 123: 1964–1969.

    Article  Google Scholar 

  5. 5.

    Godlee, F., C.R. Gale, and C.N. Martyn. 1998. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports. Journal of the American Medical Association 280: 237–240.

    Article  Google Scholar 

  6. 6.

    Callaham, M., and C. McCulloch. 2011. Longitudinal trends in the performance of scientific peer reviewers. Annals of Emergency Medicine 57: 141–148.

    Article  Google Scholar 

  7. 7.

    Einstein, A., and N. Rosen. 1937. On gravitational waves. Journal of the Franklin Institute 223: 43.

    Article  Google Scholar 

  8. 8.

    Kennefick, D. 2005. Einstein versus the Physical Review. Physics Today 58: 43.

    Article  Google Scholar 

  9. 9.

    Deming, J.W., and J.A. Baross. 1983. Growth of ‘black smoker’ bacteria at temperatures of at least 250 °C. Nature 303: 423–426.

    Article  Google Scholar 

  10. 10.

    Trent, J.D., R.A. Chastain, and A.A. Yayanos. 1984. Possible artefactual basis for apparent bacterial growth at 250 degrees C. Nature 307: 737–740.

    Article  Google Scholar 

  11. 11.

    Begley, C.G., and L.M. Ellis. 2012. Drug development: raise standards for preclinical cancer research. Nature 483: 531–533.

    Article  Google Scholar 

  12. 12.

    Prinz, F., T. Schlange, and K. Asadullah. 2011. Believe it or not: how much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery 10: 712–713.

    Article  Google Scholar 

  13. 13.

    Couzin-Frankel, J. 2011. Aging genes: the Sirtuin story unravels. Science 334: 1194–1198.

    Article  Google Scholar 

  14. 14.

    Wolfe-Simon, F., J. Switzer Blum, T.R. Kulp, et al. 2011. A bacterium that can grow by using arsenic instead of phosphorus. Science 332: 1163–1166.

    Article  Google Scholar 

  15. 15.

    Alberts, B. 2011. Editor’s note. Science 332: 1149.

    Article  Google Scholar 

  16. 16.

    Benner, S.A. 2011. Comment on “A bacterium that can grow by using arsenic instead of phosphorus”. Science 332: 1149.

    Article  Google Scholar 

  17. 17.

    Borhani, D.W. 2011. Comment on “A bacterium that can grow by using arsenic instead of phosphorus”. Science 332: 1149.

    Article  Google Scholar 

  18. 18.

    Cotner, J.B., and E.K. Hall. 2011. Comment on “A bacterium that can grow by using arsenic instead of phosphorus”. Science 332: 1149.

    Article  Google Scholar 

  19. 19.

    Csabai, I., and E. Szathmary. 2011. Comment on “A bacterium that can grow by using arsenic instead of phosphorus”. Science 332: 1149.

    Article  Google Scholar 

  20. 20.

    Foster, P.L. 2011. Comment on “A bacterium that can grow by using arsenic instead of phosphorus”. Science 332: 1149.

    Article  Google Scholar 

  21. 21.

    Oehler, S. 2011. Comment on “A bacterium that can grow by using arsenic instead of phosphorus”. Science 332: 1149.

    Article  Google Scholar 

  22. 22.

    Redfield, R.J. 2011. Comment on “A bacterium that can grow by using arsenic instead of phosphorus”. Science 332: 1149.

    Article  Google Scholar 

  23. 23.

    Schoepp-Cothenet, B., W. Nitschke, L.M. Barge, A. Ponce, M.J. Russell, and A.I. Tsapin. 2011. Comment on “A bacterium that can grow by using arsenic instead of phosphorus”. Science 332: 1149.

    Article  Google Scholar 

  24. 24.

    Wolfe-Simon, F., J.S. Blum, T.R. Kulp, et al. 2011. Response to comments on “A bacterium that can grow using arsenic instead of phosphorus”. Science 332: 1149.

    Article  Google Scholar 

  25. 25.

    Shatz, D. 2004. Peer review: a critical inquiry. Lanham: Rowman & Littlefield Publishers Inc.

    Google Scholar 

  26. 26.

    The Editors of The New Atlantis. 2006. Rethinking peer review: how the internet is changing science journals. The New Atlantis 13: 106–110.

    Google Scholar 

  27. 27.

    Horrobin, D.F. 1990. The philosophical basis of peer review and the suppression of innovation. Journal of the American Medical Association 263: 1438–1441.

    Article  Google Scholar 

  28. 28.

    Charlton, B.G. 2007. Peer usage versus peer review. British Medical Journal 335: 451.

    Article  Google Scholar 

  29. 29.

    Thurner, S., and R. Hanel. 2011. Peer-review in a world with rational scientists: towards selection of the average. European Physical Journal B 84: 707–711.

    Article  Google Scholar 

  30. 30.

    Fischer, K. 2005. Deformationen von Wissenschaft im universitären System [Deformations of science in the university system]. In Universität und wissenschaftliches Wissen [University and scientific knowledge], ed. E. Eirmbter-Stolbrink, and C. König-Fuchs. Nordhausen: Bautz.

    Google Scholar 

  31. 31.

    Groeben, N. 2006. Zur Kultur des – empirisch-szientifischen – Zeitschriftenaufsatzes [The culture of an empirical-scientific journal article.]. Handlung-Kultur-Interpretation 15: 25–41.

    Google Scholar 

  32. 32.

    Kelly, B.D. 2009. Dear Editor—a note from any imaginary author in response to any referee. Medical Hypotheses 72: 359.

    Article  Google Scholar 

  33. 33.

    Darwin, C. 1859. On the origin of species by means of natural selection, or the preservation of favoured races in the struggle for life, 1st ed. London: John Murray.

    Google Scholar 

  34. 34.

    Quammen, D. 2006. The reluctant Mr. Darwin. New York: Atlas Books.

    Google Scholar 

  35. 35.

    The Editors of Nature Cell Biology. 2006. Appreciating data: warts, wrinkles and all. Nature Cell Biology 8: 203.

    Google Scholar 

  36. 36.

    Charlton, B.G. 2010. The cancer of bureaucracy: how it will destroy science, medicine, education; and eventually everything else. Medical Hypotheses 74: 961–965.

    Article  Google Scholar 

  37. 37.

    Johnson, V. 2008. Statistical analysis of the National Institutes of Health peer review system. Proceedings of the National Academy of Sciences 105: 11076–11080.

    Article  Google Scholar 

  38. 38.

    Gawrylewski, A. 2008. Tackling peer review bias. The Scientist, July 28. http://classic.the-scientist.com/?articles.view/articleNo/26628/. Accessed August 6, 2012.

  39. 39.

    Bains, W. 2009. Leadership and innovation: how consensus management blocks genuine innovation. Bioscience Hypotheses 2: 277–281.

    Article  Google Scholar 

  40. 40.

    Kaplan, D., N. Lacetera, and C. Kaplan. 2008. Sample size and precision in NIH peer review. PLoS ONE 3: e2761.

    Article  Google Scholar 

  41. 41.

    Horrobin, D.F. 1975. Ideas in biomedical science: reasons for the foundation of Medical Hypotheses. Medical Hypotheses 1: 1–2.

    Article  Google Scholar 

  42. 42.

    Charlton, B.G. 2007. Medical Hypotheses 2006 impact factor rises to 1.3—A vindication of the “editorial review” system for revolutionary science. Medical Hypotheses 69: 967–969.

    Article  Google Scholar 

  43. 43.

    Fröhlich, G. 2002. Anonyme Kritik. Peer Review auf dem Prüfstand der empirisch-theoretischen Wissenschaftsforschung [Anonymous criticism Peer review researched]. In Drehscheibe E-Mitteleuropa [Hub e-Middle-Europe], ed. E. Pipp. Wien: Phoibos.

  44. 44.

    Watts, G. 2010. Emasculating hypothetical oddities? British Medical Journal 340: c726.

    Article  Google Scholar 

  45. 45.

    Cressey, D. 2010. Editor says no to peer review for controversial journal. Nature News, March 18. http://www.nature.com/news/2010/100318/full/news.2010.132.html. Accessed August 12, 2012.

  46. 46.

    Enserink, M. 2010. Elsevier to editor: change controversial journal or resign. Science 327: 1316.

    Article  Google Scholar 

  47. 47.

    Duesberg, P.H., D. Mandrioli, A. McCormack, et al. 2011. AIDS since 1984: no evidence for a new, viral epidemic—not even in Africa. Italian Journal of Anatomy and Embryology 116: 73–92.

    Google Scholar 

  48. 48.

    Corbyn, Z. 2012. Paper denying HIV–AIDS link secures publication: work by infamous AIDS contrarian passes peer review. Nature News, January 5, 2012. http://www.nature.com/news/paper-denying-hiv-aids-link-secures-publication-1.9737. Accessed August 12, 2012.

  49. 49.

    Medical Hypotheses guide for authors. 2011. Elsevier. http://www.elsevier.com/wps/find/journaldescription.cws_home/623059/authorinstructions. Accessed July 2011.

  50. 50.

    Mulligan A., P. Campbell, and T. Dorigo. 2010. What’s up with peer review: the future of peer review in policy, research, and public debates. Discussion following the oral presentation at the Euroscience Open Forum, Torino, Italy, July 2–7.

  51. 51.

    Manku, M.S. 2010. Mehar S Manku on assuming the editorship of Medical Hypotheses. Medical Hypotheses 75: 275.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Georg Steinhauser.

Additional information

Author affiliations appear at the end of the article.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Steinhauser, G., Adlassnig, W., Risch, J.A. et al. Peer review versus editorial review and their role in innovative science. Theor Med Bioeth 33, 359–376 (2012). https://doi.org/10.1007/s11017-012-9233-1

Download citation

Keywords

  • Peer review
  • Academic freedom
  • Editorial policy
  • Periodicals as topic
  • Innovation
  • Scientific hypotheses
  • David F. Horrobin