Characteristics of Peer Review Reports: Editor-Suggested Versus Author-Suggested Reviewers

Shopovski, Jovan; Bolek, Cezary; Bolek, Monika

doi:10.1007/s11948-019-00118-y

Original Research/Scholarship
Published: 17 June 2019

Characteristics of Peer Review Reports: Editor-Suggested Versus Author-Suggested Reviewers

Science and Engineering Ethics volume 26, pages 709–726 (2020)Cite this article

343 Accesses
2 Citations
31 Altmetric
Metrics details

Abstract

Peer review is widely recognized as a mechanism for quality control of academic content. This research article aims at comparing the review reports and decisions of reviewers who are members of the editorial board of the European Scientific Journal (ESJ) with those reviewers suggested by the authors and who are not affiliated with the journal. 457 review reports on 378 papers submitted to the ESJ in the period of October–December 2017 were analysed. Statistical methods including OLS and Wilcoxon rank-sum test were applied based on the score approach toward the reviewers’ assessments of the papers and their characteristics related to the country, gender, and time of revisions. Results show the difference between the decisions these two groups of reviewers made. Even though editor-suggested and author-suggested reviewers need equal time to review a paper, the former are less favourable towards the authors of the papers. It is also concluded that factors such as time and country of the reviewers influence their decisions. In this regard, the editors should avoid relying their decisions solely on review reports received from reviewers suggested by the authors. However, further research with larger sample sizes should be conducted.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

References

Bornmann, L., & Daniel, H.-D. (2010). Do author-suggested reviewers rate submissions more favorably than editor-suggested reviewers? A study on atmospheric chemistry and physics. PLoS ONE,5(10), e13345. https://doi.org/10.1371/journal.pone.0013345.
Article Google Scholar
Earnshaw, J. J., Farndon, J. R., Guillou, P. J., Johnson, C. D., Murie, J. A., & Murray, G. D. (2000). A comparison of reports from referees chosen by authors or journal editors in the peer review process. Annals of the Royal College of Surgeons of England,82(4), 133–135.
Google Scholar
Fox, W. C., Burns, C. S., Anna, D. M., & Mayer, A. J. (2016). Author-suggested reviewers: Gender differences and influences on the peer review process at an ecology journal. Functional Ecology,31(1), 270–280. https://doi.org/10.1111/1365-2435.12665.
Article Google Scholar
Jefferson, T., Alderson, P., Walger, E., & Davidoff, F. (2002). Effects of editorial peer review: A systematic review. JAMA,287(21), 1–4.
Google Scholar
Jelicic, M., & Merckelbach, H. (2002). Peer-review: Let’s imitate the lawyers! Cortex,38(3), 406–407.
Article Google Scholar
Kowalczuk, M. K., Dudbridge, F., Nanda, S., Harriman, S. L., Patel, J., & Moylan, E. C. (2015). Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models. BMJ Open. https://doi.org/10.1136/bmjopen-2015-008707.
Article Google Scholar
Liang, Y. (2018). Should authors suggest reviewers? A comparative study of the performance of author-suggested and editor-selected reviewers at a biological journal. Learned Publishing,31(3), 216–221.
Article Google Scholar
Mahoney, M. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research,1(2), 161–175.
Article Google Scholar
Moore, L. M., Neilson, E. G., & Siegel, V. (2011). Effect of recommendations from reviewers suggested or excluded by authors. Journal of American Society of Nephrology,22(9), 1598–1602. https://doi.org/10.1681/ASN.2011070643.
Article Google Scholar
Murray, D., Siler, K., Lariviere, V., Chan, W. M., Collings, A., Raymond, J., et al. (2018). Gender and international diversity improves equity in peer review. PLoS Biology. https://doi.org/10.1101/400515. (preprint).
Article Google Scholar
Peters, D., & Ceci, S. (1982). Peer-review practices of psychological journals: The fate of submitted articles, submitted again. The Behavioral and Brain Sciences,5(2), 187–195. https://doi.org/10.1017/S0140525X00011183.
Article Google Scholar
Rivara, P. F., Cummings, P., Ringold, S., Bergman, A. B., Joffe, A., & Christakis, D. A. (2007). A comparison of reviewers selected by editors and reviewers suggested by authors. The Journal of Pediatrics,151(2), 202–205. https://doi.org/10.1016/j.jpeds.2007.02.008.
Article Google Scholar
Schroter, S., Tite, L., Hutchings, A., & Black, N. (2006). Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. JAMA,295(3), 314–317. https://doi.org/10.1001/jama.295.3.314.
Article Google Scholar
Smith, R. (2006). Peer review: A flawed process and the heart of science and journals. Journal of the Royal Society of Medicine,99(4), 178–182. https://doi.org/10.1258/jrsm.99.4.178.
Article Google Scholar
Spier, R. (2002). The effects of the peer review process. Trends in Biotechnology,20(8), 357–358.
Article Google Scholar
Teixeira da Silva, J. A., & Al-Khatib, A. (2017). Should authors be requested to suggest peer reviewers? Science and Engineering Ethics,24(1), 275–285. https://doi.org/10.1007/s11948-016-9842-6.
Article Google Scholar
UN Country Classification. (2014). http://www.un.org/en/development/desa/policy/wesp/wesp_current/2014wesp_country_classification.pdf. Accessed October 25, 2018.

Download references

Author information

Affiliations

European Scientific Institute, ESI, Ponta Delgada, Portugal
Jovan Shopovski
Faculty of Management, University of Lodz, Lodz, Poland
Cezary Bolek
Faculty of Economics and Sociology, University of Lodz, Lodz, Poland
Monika Bolek

Authors

Jovan Shopovski
View author publications
You can also search for this author in PubMed Google Scholar
Cezary Bolek
View author publications
You can also search for this author in PubMed Google Scholar
Monika Bolek
View author publications
You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jovan Shopovski.

Ethics declarations

Conflict of interest

Jovan Shopovski is the Managing Editor of the European Scientific Journal (ESJ), and Monika Bolek is a part of the Editorial Board of the same journal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Symbols of Variables

AC:: Countries of origins of authors
PL:: Paper language
DE:: Decisions of editor-suggested reviewers
DA:: Author-suggested reviewers (external)
CE:: Countries of origins of editor-suggested reviewers
CA:: Countries of origins of author-suggested reviewers
GE:: Gender of editor-suggested reviewers
GA:: Gender of author-suggested reviewers
TE:: Time of editor-suggested reviewers
TA:: Time of author-suggested reviewers
TIE:: Editor-suggested reviewer title assessment
AE:: Editor-suggested reviewer abstract assessment
GRE:: Editor-suggested reviewer grammar assessment
ME:: Editor-suggested reviewer methods assessment
BE:: Editor-suggested reviewer body of a paper assessment
COE:: Editor-suggested reviewer conclusions assessment
RE:: Editor-suggested reviewer references assessment
TIA:: Author-suggested reviewer title assessment
AA:: Author-suggested reviewer abstract assessment
GRA:: Author-suggested reviewer grammar assessment
MA:: Author-suggested reviewer methods assessment
BA:: Author-suggested reviewer body of a paper assessment
COA:: Author-suggested reviewer conclusions assessment
RA:: Author-suggested reviewer references assessment
ACTA:: Time of review of author-suggested reviewer in relation to country of author
CATA:: Time of review of author-suggested reviewer in relation to country of reviewer
GATA:: Time of review of author-suggested reviewer in relation to gender of reviewer
ACTE:: Time of review of editor-suggested reviewer in relation to country of author
CETE:: Time of review of editor-suggested reviewer in relation to country of reviewer
GETE:: Time of review of editor-suggested reviewer in relation to gender of reviewer

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Shopovski, J., Bolek, C. & Bolek, M. Characteristics of Peer Review Reports: Editor-Suggested Versus Author-Suggested Reviewers. Sci Eng Ethics 26, 709–726 (2020). https://doi.org/10.1007/s11948-019-00118-y

Download citation

Received: 24 November 2018
Accepted: 07 June 2019
Published: 17 June 2019
Issue Date: April 2020
DOI: https://doi.org/10.1007/s11948-019-00118-y

Keywords

Peer review
Reviewers
Academic publishing
Academia
Review reports

Abstract

Access options

Buy single article

References

Author information

Affiliations

Corresponding author

Ethics declarations

Conflict of interest

Additional information

Publisher's Note

Appendix: Symbols of Variables

Appendix: Symbols of Variables

Rights and permissions

About this article

Cite this article

Share this article

Keywords