The European Physical Journal B

, Volume 84, Issue 4, pp 707–711 | Cite as

Peer-review in a world with rational scientists: Toward selection of the average

Regular Article Interdisciplinary Physics

Abstract

It is widely believed that one of the virtues of peer review is that it provides a self-regulating selection mechanism for scientific work, papers and projects. Peer review as a selection mechanism is hard to evaluate in terms of its efficiency. Serious efforts to understand its strengths and weaknesses have not yet lead to conclusive answers. In theory peer review works if the involved parties (editors and referees) conform to a set of requirements, such as love for high quality science, objectiveness, and absence of biases, nepotism, friend and clique networks, selfishness, etc. If these requirements are violated, what is the effect on the selection of high quality work? We study this question with a simple agent based model. In particular we are interested in the effects of rationalreferees, who might not have any incentive to see high quality work other than their own published or promoted. We find that a small fraction of incorrect (selfish or rational) referees is sufficient to drastically lower the quality of the published (accepted) scientific standard. We determine the fraction for which peer review will no longer perform better than pure chance. Decline of quality of accepted scientific work is shown as a function of the fraction of rational and unqualified referees. We show how a simple quality-increasing policy of e.g. a journal can lead to additional loss in overall scientific quality, and how mutual support-networks of authors and referees deteriorate the system.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    D. Kennefick, Phys. Today 58, 43 (2009)CrossRefGoogle Scholar
  2. 2.
    Nature’s peer review trial, Nature 05535 (2006), doi: 10.1038/nature0553510.1038/nature05535, www.nature.com/nature/peerreview/debate/nature05535.html
  3. 3.
    Editorial: Peer review and fraud, Nature 444, 971 (2006)Google Scholar
  4. 4.
    Editorial: Working double-blind: should there be author anonymity in peer review? Nature 451, 605 (2008)Google Scholar
  5. 5.
    T. Groves, Nature (Nature Peer Review debate), 04995 (2006), doi: 10.1038/nature04995, www.nature.com/nature/peerreview/debate/nature04995.html
  6. 6.
    T. Jefferson, P. Alderson, E. Wagner, F. Davidoff, J. Am. Med. Assoc. 287, 2784 (2002) CrossRefGoogle Scholar
  7. 7.
    T. Jefferson, Rudin, S. Brodney Folse, F. Davidoff, Cochrane database of systematic reviews 2, MR000016 (2007) Google Scholar
  8. 8.
    P.M. Rothwell, C.N. Martyn, Brain 123, 1964 (2000) CrossRefGoogle Scholar
  9. 9.
    F. Godlee, C.R. Gale, C.N. Martyn, J. Am. Med. Assoc. 280, 237 (1998) CrossRefGoogle Scholar
  10. 10.
    C. Wenneras, A. Wold, Nature 387, 341 (1997) CrossRefADSGoogle Scholar
  11. 11.
    E.S. Brezis, Sci. Public Pol. 34, 691 (2007)CrossRefGoogle Scholar
  12. 12.
    D.F. Horrobin, J. Am. Med. Assoc. 263, 1438 (1990) CrossRefGoogle Scholar
  13. 13.
    M. Mahoney, Cogn. Ther. Res. 1, 161 (1977)CrossRefGoogle Scholar
  14. 14.
    B. Martin, Peer review as scholarly conformity, in Suppression Stories (Fund for Intellectual Dissent, Wollongong, 1997), pp. 69–83.Google Scholar
  15. 15.
    M. Ware, Peer review: benefits, perceptions and alternatives (Publishing Research Consortium Summary Papers, 2008), Vol. 4, pp. 1–20Google Scholar

Copyright information

© EDP Sciences, SIF, Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Section for Science of Complex SystemsMedical University of ViennaViennaAustria
  2. 2.Santa Fe InstituteSanta FeUSA
  3. 3.IIASALaxenburgAustria

Personalised recommendations