Abstract
Beyond explicit reviewer commentary, editors may rely on other metrics when evaluating manuscripts under consideration for publication. One potential, indirect measure of merit may be the ease or difficulty in identifying reviewers willing to review a given paper. We sought to determine whether reviewer decisions to agree or decline to review a manuscript are associated with manuscript acceptance. Original Research submissions to “Radiology” from 1/1/2008 to 12/31/2011 were studied. Using Student’s t tests, we studied the association between the ratio number-of-reviewers-declining:number-of-reviewers-agreeing to review manuscripts (“decline:agree ratio”) and editor decision to accept or reject the manuscript. A subgroup analysis of papers in which all four invited reviewers agreed to review the paper (“universal agree-to-review group”) was performed. Pearson’s correlation was used to study decline:agree ratio and accepted manuscript citation rate. Original Research manuscript acceptance rate at Radiology was 14.5% (780/5375). Decline:agree ratio was similar between accepted and rejected manuscripts (0.87 ± 0.84 versus 0.90 ± 0.86 respectively, P = 0.35). “Universal agree-to-review” papers were accepted at similar rates to other papers (15.7% [22/140] versus 14.5% [758/5235] respectively, P = 0.69). Higher decline:agree ratios corresponded to lower manuscript citation rates (r = 0.09, P = 0.048). Our study, based on the lack of correlation between agreement to review rate and acceptance rate to Radiology and the direct correlation between agreement to review rate and manuscript citation rate, suggests that reviewers may have a preference for manuscripts with greater potential scientific relevance, but that reviewer motivation to agree to review does not include the expectation of manuscript acceptance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Birukou, A., Wakeling, J. R., Bartolini, C., Casati, F., Marchese, M., Mirylenka, K., et al. (2011). Alternatives to peer review: Novel approaches for research evaluation. Frontiers in Computational Neuroscience, 5, 56.
Black, N., Van, R. S., Godlee, F., Smith, R., & Evans, S. (1998). What makes a good reviewer and a good review for a general medical journal? JAMA, 280, 231–233.
Bornmann, L., & Daniel, H. D. (2010). The usefulness of peer review for selecting manuscripts for publication: A utility analysis taking as an example a high-impact journal. PLoS ONE, 5(6), e11344.
Bornmann, L., & Werner, M. (2016). The journal impact factor and alternative metrics. Science and Society, 17(8), 1094–1097.
Cobo, E., Cortes, J., Ribera, J. M., Cardellach, F., Selva-O’Callaghan, A., Kostov, B., et al. (2011). Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: Masked randomised trial. BMJ, 343, d6783.
Garfunkel, J. M., Ulshen, M. H., Hamrick, H. J., & Lawson, E. E. (1994). Effect of institutional prestige on reviewers’ recommendations and editorial decisions. JAMA, 272, 137–138.
Gilbert, J. R., Williams, E. S., & Lundberg, G. D. (1994). Is there gender bias in JAMA’s peer review process? JAMA, 272, 139–142.
Johnston, D. (2015). Peer review incentives: A simple idea to encourage fast and effective peer review. European Science Editing, 41(3), 70–71.
Kravitz, R. L., Franks, P., Feldman, M. D., Gerrity, M., Byrne, C., & Tierney, W. M. (2010). Editorial peer reviewers’ recommendations at a general medical journal: Are they reliable and do editors care? PLoS ONE, 5, e10072.
Link, A. M. (1998). US and non-US submissions: An analysis of reviewer bias. JAMA, 280, 246–247.
Peer Review Survey (2009). Sense about Science. http://archive.senseaboutscience.org/pages/peer-review-survey-2009.html. Accessed Jan 25, 2017.
Polak, J. F. (1995). The role of the manuscript reviewer in the peer review process. AJR. American Journal of Roentgenology, 165, 685–688.
Ross, J. S., Gross, C. P., Desai, M. M., Hong, Y., Grant, A. O., et al. (2006). Effect of blinded peer review on abstract acceptance. JAMA, 295, 1675–1680.
Schroter, S., Black, N., Evans, S., Godlee, F., Osorio, L., & Smith, R. (2008). What errors do peer reviewers detect, and does training improve their ability to detect them? Journal of Royal Society of Medicine, 101, 507–514.
Stamm, T., Meyer, U., Wiesmann, H. P., Kleinheinz, J., Cehreli, M., et al. (2007). A retrospective analysis of submissions, acceptance rate, open peer review operations, and prepublication bias of the multidisciplinary open access journal Head & Face Medicine. Head and Face Medicine, 3, 27.
Tite, L., & Schroter, S. (2007). Why do peer reviewers decline to review? Journal of Epidemiology Community Health, 61, 9–12.
Ware, M., & Monkman, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community—An international study. Bristol: Publishing Research Consortium.
Willis, M. (2016). Why do peer reviewers decline to review manuscripts? A study of reviewer invitation responses. Learned Publishing, 29, 5–7.
Zaharie, M. A., & Osoian, C. L. (2016). Peer review motivation frames: A qualitative approach. European Management Journal, 34, 69–79.
Acknowledgements
We thank Debbie Hogan from Radiology for her help in obtaining the data used in this manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kallmes, K.M., Brinjikji, W., Ahmed, A.T. et al. Difficulty in finding manuscript reviewers is not associated with manuscript acceptance rates: a study of the peer-review process at the journal Radiology . Scientometrics 111, 971–978 (2017). https://doi.org/10.1007/s11192-017-2331-0
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-017-2331-0