Skip to main content

The good, the bad and the rude peer-review

Peer-review is the core of the editorial process and the basis of the publication system. The quality of peer-review depends on the quality of the peer-reviewers. Peer reviewers are supposed to ensure that journals publish high-quality science by evaluating manuscripts and offering suggestions for improvement. Peer-reviewers are typically selected based on their expertise in the areas of research associated with the submitted manuscripts. Although being a peer-reviewer is sometimes frustrating, communication between authors, editors and reviewers in the peer-review process determines the eventual success of the publication; this communication should be formal, constructive, honest and polite [1, 2]. Specific standards of formal and ethical writing are necessary for peer-review. These standards should be maintained throughout the review process of any submitted paper in any particular journal.

Peer-review is considered a biased process with identified defects [3,4,5,6,7,8,9,10,11]; some peer-reviewers are too young with limited experience, not all are equally skilled in the peer-review process, and very few have had a formal training and assessment methods for peer-review [1, 12, 13]. A growing body of quantitative evidence showed violations of objectivity and bias in the peer review process for reasons based on author attributes such as language, institutional affiliation, nationality, and others, authors’ identity such as gender and sexuality, and reviewers’ perceptions of the field such as territoriality within field, personal gripes with authors, scientific dogma, discontent/distrust of methodological advances [14, 15]. Occasionally, reviewers’ comments are rude or unprofessional; in a study, more than half of 1106 anonymous respondents reported receiving at least one “unprofessional” review, and a majority of those said they had received multiple problematic comments including comments tended to personally target a scientist, lack constructive criticism, or unnecessarily harsh or cruel [15]. This editorial note aims to communicate to the readers of the journal the unified set of rules for a good and bad peer-review, and to emphasize on the avoidance and consequences of a rude peer-review.

The good peer-review

A good peer-review should be objective, instructional and informative. It should (1) include critical comments that would be helpful to the editor to make his decision and to the authors to improve their manuscript, (2) indicate the novelty and significance of the work and if/how it adds to the current literature, (3) identify major flaws in the study hypothesis, materials and methods, techniques, approaches and statistics, (4) denote statements requiring clarification, (5) give details for areas to be improved and list changes to be made, (6) be written in formal and clear writing, well-structured and extend at least one half to two pages, (7) number the points and refer to pages, paragraphs of sections or lines in the manuscript, and (8) review all sections of the manuscript including the citations and illustrations [1].

A good peer-reviewer should be responsible and reliable. He/she should (1) accept promptly an invitation, (2) not turn down any invitation for review, (3) return the peer-review on time, (4) provide confidential comments to the editor if he/she detects any misconduct such as plagiarism, unattributed work, unethical procedures, or duplicate publication, and (5) disclose potential conflicts of interest, and decline the invitation if there are any with the authors [1, 16].

A good reviewer should be polite, honest and clear to the authors. He/she should (1) write clearly to be understood by people whose native language is not English and avoid complex or unusual words, (2) review the author’s work as if it was their own and be fair in his/her comments to the editor as if the authors might read them too and as if the reviewer signs the peer-review, (3) respect confidentiality of the work and not use information from the manuscript prior to its publication, or communicate about the manuscript with outsiders or the authors [16].

The bad peer-review

A bad peer-review is non-objective, non-organized and non-instructional or informative. It is a peer-review that (1) fails to identify major flaws, to communicate criticism to authors to improve their manuscript and to inform fraud to the editor, and (2) does not review all manuscript sections including references and illustrations. A bad peer-reviewer in medical writing practice is (1) a snapshot reviewer who only reads and replies briefly if he likes or not the paper, (2) a mixed-signal reviewer who is associated with a discrepancy between the comments and the conclusion/recommendation of the review, (3) a hidden-agenda reviewer who replies only confidential comments to editor without comments to the authors or with different comments to the authors from the confidential comments to the editor, and (4) a reviewer who asks the authors impossible requirements, expresses dishonest remarks and/or discourages the authors [17].

The rude peer-review

A rude peer-review is unfair and biased for reasons unrelated to the quality of the manuscript such as the nationality, gender, English language criticism, opposite theory or conflicts of interest [2, 18]. It is a discouraging, insulting, dishonest and unhelpful review with impossible requests to the authors. It may recommend acceptance of the manuscript with comments that may make the authors give up in despair, or it may reject the paper that otherwise has been quite favorably reviewed. A rude review expresses prejudices, has conflicts of interest, or the paper was not read carefully or thoroughly enough by the reviewer. It includes overly negative wording or personal comments, or overly brief and direct comments that give the review an unfriendly tone. Rude reviews are pervasive and sometimes harmful; rude reviewers offer condescending or outright offensive comments, and/or urge the irrelevant citation of their own work [19].

A study surveyed science, technology, engineering, and mathematics (STEM) scientists that have submitted manuscripts to peer-review journals as first author to understand the impacts of receiving unprofessional peer review comments on the perception of scientific aptitude (confidence as a scientist), productivity (publications per year) and career advancement (ability to advance within the field). Over half of participants reporting that they had received unprofessional comments [15]. This study defined an unprofessional peer-review comment as any statement that is unethical or irrelevant to the nature of the study such as comments that (1) lack constructive criticism, (2) are directed at the author(s) rather than the nature or quality of the study, (3) use personal opinions of the author(s)/study rather than evidence-based criticism, or (4) are “mean-spirited” or cruel [15]. Women and non-binary people were the most likely to report significant delays in productivity after receiving unprofessional reviews. This unprofessional behavior often occurs under the cloak of anonymity and is being perpetuated by the scientific community upon its members. Open reviews may help curtail unprofessional comments, but more research on this topic is necessary [20]. From the authors point of view, receiving a pervasive review with negative comments – stereotype threats [21], no matter how inaccurate they are, can create psychological distress that, in turn, can result in self-doubt, impaired performance, and delayed career advancement [19, 21, 22]. This effect is well documented, particularly on written academic tests; in more than 300 demonstrations, stereotype threat has been shown to reduce test performance among negatively stereotyped groups such as racial/ethnic minority students and women in male-dominated fields [23]. It has also been shown to undermine the quality of doctor-patient interactions leading to worse health outcomes for patients [24], to undermine athletic performance [25], driving performance [26], and workplace success and well-being [27, 28]. Therefore, these threats in peer-review should be reduced to zero.

Providing respectful comments is paramount in peer-review, regardless of the manuscript’s overall quality. Most authors made a substantial effort to conduct their study, prepare and submit their paper, and usually they present their best work for evaluation. Therefore, keeping a collegial and respectful tone, showing appreciation of these efforts is mandatory [16]. Language problems should be brought up only if they make the paper difficult to understand; but this should done politely, and can be communicated with confidential comments to the editor [2, 16, 18]. Native English speakers have an advantage in paper writing and reviewing. It is always a matter of huge respect that non-native English speakers can conduct themselves so well in a ‘foreign’ language both for journals and in meetings. Reviewers should not make assumptions about the quality of a paper based on the authors’ nationality and affiliations as this bias is racism. Instead, they should focus on the research and communicate to the Editor if the English is so poor that the paper cannot be reviewed or he/she cannot provide feedback on the science [29].

Many journals have guidelines and codes for peer-review and reviewers, but there’s also a lot of inconsistency when it comes to how editors address inappropriate reviews. Some journals allow the assigned editors the right not to pass unprofessional or uncourteous comments to the authors, in contrast to other journals that have policies that prevent the editor from redacting anything, and they are obligated to present whatever the reviewer has written to the authors. Journal editors are responsible for policing reviewer comments; researchers should not be discouraged after receiving a rude peer review but they should reply to the editor stating they have received a not accurate and/or rude review of their work, and ask to be assigned to another reviewer to evaluate their paper [18].

Open peer-review may be a solution to poor quality and rude reviews. In a study, a sufficient number (76%) of reviewers agreed to have their name revealed to the authors whose papers they reviewed, making open peer-review feasible. Non-blinded reviews were at least as good as blinded reviews, and maybe of better quality, however, they were more likely to recommend acceptance of the paper [30].

In conclusion, the peer review process is an essential step in protecting the quality and integrity of scientific publications. We are indebted to the reviewers who provide their voluntary, qualified and responsible collaboration to the scholar peer-review process. Being reviewer is sometimes frustrating, however, peer-review should interfere in a positive way with the authors, and rude reviews with pervasive comments are unacceptable.

References

  1. 1.

    Mavrogenis AF, Sun J, Quaile A, Scarlat MM (2019) How to evaluate reviewers – the international orthopedics reviewers score (INOR-RS). Int Orthop 43(8):1773–1777

    Article  Google Scholar 

  2. 2.

    Quaile A, Scarlat MM, Mavrogenis AF, Mauffrey C (2019) International Orthopaedics – instructions for authors, English expression, style and rules. Int Orthop 43(11):2425–2427

    Article  Google Scholar 

  3. 3.

    Peters DP, Ceci SJ (1980) A manuscript masquerade. How well does the review process work? N Y Acad Sci. https://doi.org/10.1002/j.2326-1951.1980.tb01346.x

    Article  Google Scholar 

  4. 4.

    Link AM (1998) US and non-US submissions: an analysis of reviewer bias. JAMA 280(3):246–247

    CAS  Article  Google Scholar 

  5. 5.

    Schroter S, Tite L, Hutchings A, Black N (2006) Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. JAMA 295(3):314–317

    CAS  Article  Google Scholar 

  6. 6.

    Smith R (2006) Peer review: a flawed process at the heart of science and journals. J R Soc Med 99:178–182

    Article  Google Scholar 

  7. 7.

    Wager E, Parkin EC, Tamber PS (2006) Are reviewers suggested by authors as good as those chosen by editors? Results of a rater-blinded, retrospective study. BMC Med 4:13

    Article  Google Scholar 

  8. 8.

    Knobloch-Westerwick S, Glynn CJ (2013) The Matilda effect in science communication: an experiment on gender bias in publication quality perceptions and collaboration interest. Sci Commun 35:603–625

    Article  Google Scholar 

  9. 9.

    Larivière V, Ni C, Gingras Y, Cronin B, Sugimoto CR (2013) Bibliometrics: global gender disparities in science. Nature 504(7479):211–213

    Article  Google Scholar 

  10. 10.

    Kowalczuk MK, Dudbridge F, Nanda S, Harriman SL, Patel J, Moylan EC (2015 Sep 29) Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models. BMJ Open 5(9):e008707

    Article  Google Scholar 

  11. 11.

    Wren JD, Valencia A, Kelso J (2019) Reviewer-coerced citation: case report, update on journal policy and suggestions for future prevention. Bioinformatics. 35(18):3217–3218

    Article  Google Scholar 

  12. 12.

    van Rooyen S, Godlee F, Evans S, Black N, Smith R (1999) Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. BMJ. 318(7175):23–27

    Article  Google Scholar 

  13. 13.

    Messias AMV, Lira RPC, Furtado JMF, Paula JS, Rocha EM (2017) How to evaluate and acknowledge a scientific journal peer reviewer: a proposed index to measure the performance of reviewers. Arq Bras Oftalmol 80(6) V

  14. 14.

    Lee CJ, Sugimoto CR, Zhang G, Cronin B (2013) Bias in peer review. Journal of the American Society for Information Science and Technology 64(1):2–17

    Article  Google Scholar 

  15. 15.

    Silbiger NJ, Stubler AD (2019) Unprofessional peer reviews disproportionately harm underrepresented groups in STEM. PeerJ. 7:e8247

    Article  Google Scholar 

  16. 16.

    Zazgyva A, Kon E, Mauffrey C, Mavrogenis AF, Scarlat MM (2017) Reviews, reviewers and reviewing. Int Orthop 41(1):1–2

    Article  Google Scholar 

  17. 17.

    Cowell HR (2000) Ethical responsibilities of editors, reviewers, and authors. Clin Orthop Relat Res 378:83–89

    Article  Google Scholar 

  18. 18.

    Conroy G Q&A Linda Beaumont: Journals should take action against toxic peer reviews. Available at: https://www.natureindex.com/news-blog/linda-beaumont-research-journals-should-take-action-against-toxic-peer-reviews. Accessed 6 Jan 2020

  19. 19.

    Wilcox C Rude paper reviews are pervasive and sometimes harmful. Available at: https://www.sciencemag.org/news/2019/12/rude-paper-reviews-are-pervasive-and-sometimes-harmful-study-finds. Accessed 6 Jan 2020. https://doi.org/10.1126/science.aba5502

  20. 20.

    Leek JT, Taub MA, Pineda FJ (2011) Cooperation between referees and authors increases peerreview accuracy. PLOS ONE 6:e26895

    CAS  Article  Google Scholar 

  21. 21.

    Lewis NA Jr, Sekaquaptewa D (2016) Beyond test performance: a broader view of stereotype threat. Curr Opin Psychol 11:40–43

    Article  Google Scholar 

  22. 22.

    Spencer SJ, Logel C, Davies PG (2016) Stereotype threat. Annu Rev Psychol 67:415–437

    Article  Google Scholar 

  23. 23.

    Nguyen HHD, Ryan AM (2008) Does stereotype threat affect test performance of minorities and women? A meta-analysis of experimental evidence. J Appl Psychol 93:1314–1334

    Article  Google Scholar 

  24. 24.

    Burgess DJ, Warren J, Phelan S, Dovidio J, Van Ryn M (2010) Stereotype threat and health disparities: what medical educators and future physicians need to know. J Gen Intern Med 25:169–177

    Article  Google Scholar 

  25. 25.

    Krendl A, Gainsburg I, Ambady N (2012) The effects of stereotypes and observer pressure on athletic performance. J Sport Exerc Psychol 34:3–15

    Article  Google Scholar 

  26. 26.

    Lambert AE, Watson JM, Stefanucci JK, Ward N, Bakdash JZ, Strayer DL (2016) Stereotype threat impairs older adult driving. Appl Cogn Psychol 30:22–28

    Article  Google Scholar 

  27. 27.

    Gupta VK, Goktan AB, Gunay G (2014) Gender differences in evaluation of new business opportunity: a stereotype threat perspective. J Bus Venturing 29:273–288

    Article  Google Scholar 

  28. 28.

    Kalokerinos EK, von Hippel C, Zacher H (2014) Is stereotype threat a useful construct for organizational psychology research and practice? Ind Organ Psychol-US 7:381–402

    Article  Google Scholar 

  29. 29.

    Romero-Olivares AL Reviewers, don’t be rude to nonnative English speakers. Available at: https://www.sciencemag.org/careers/2019/10/reviewers-don-t-be-rude-nonnative-english-speakers. Accessed 6 January 2020

  30. 30.

    Walsh E, Rooney M, Appleby L, Wilkinson G (2000) Open peer review: a randomized controlled trial. Br J Psychiatry 176:47–51

    CAS  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Marius M. Scarlat.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mavrogenis, A.F., Quaile, A. & Scarlat, M.M. The good, the bad and the rude peer-review. International Orthopaedics (SICOT) 44, 413–415 (2020). https://doi.org/10.1007/s00264-020-04504-1

Download citation