Scientometrics

, Volume 84, Issue 2, pp 277–292

Are three heads better than two? How the number of reviewers and editor behavior affect the rejection rate

Article

DOI: 10.1007/s11192-009-0084-0

Cite this article as:
Schultz, D.M. Scientometrics (2010) 84: 277. doi:10.1007/s11192-009-0084-0
  • 253 Downloads

Abstract

Editors of peer-reviewed journals obtain recommendations from peer reviewers as guidance in deciding upon the suitability of a submitted manuscript for publication. To investigate whether the number of reviewers used by an editor affects the rate at which manuscripts are rejected, 500 manuscripts submitted to Monthly Weather Review during 15.5 months in 2007–2008 were examined. Two and three reviewers were used for 306 and 155 manuscripts, respectively (92.2% of all manuscripts). Rejection rates for initial decisions and final decisions were not significantly different whether two or three reviewers were used. Manuscripts with more reviewers did not spend more rounds in review or have different rejection rates at each round. The results varied by editor, however, with some editors rejecting more two-reviewer manuscripts and others rejecting more three-reviewer manuscripts. Editors described using their scientific expertise in the decision-making process, either in determining the number of reviews to be sought or in making decisions once the reviews were received, approaches that differ from that of relying purely upon reviewer agreement as reported previously in the literature. A simple model is constructed for three decision-making strategies for editors: rejection when all reviewers recommend rejection, rejection when any reviewer recommends rejection, and rejection when a majority of reviewers recommend rejection. By plotting the probability of reviewer rejection against the probability of editor rejection, the decision-making process can be graphically illustrated, demonstrating that, for this dataset, editors are likely to reject a manuscript when any reviewer recommends rejection.

Keywords

Journal Editor Monthly Weather Review Peer review Rounds of reviews Reviewer agreement 

PACS

01.20 

Mathematical Subject Classification (2000)

00A99 

JEL Codes

Z00 

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2009

Authors and Affiliations

  1. 1.Division of Atmospheric Sciences and Geophysics, Department of PhysicsUniversity of Helsinki, and Finnish Meteorological InstituteHelsinkiFinland

Personalised recommendations