A content analysis of referees’ comments: how do comments on manuscripts rejected by a high-impact journal and later published in either a low- or high-impact journal differ?
- First Online:
- 420 Downloads
Using the data of a comprehensive evaluation study on the peer review process of Angewandte Chemie International Edition (AC-IE), we examined in this study the way in which referees’ comments differ on manuscripts rejected at AC-IE and later published in either a low-impact journal (Tetrahedron Letters, n = 54) or a high-impact journal (Journal of the American Chemical Society, n = 42). For this purpose, a content analysis was performed of comments which led to the rejection of the manuscripts at AC-IE. For the content analysis, a classification scheme with thematic areas developed by Bornmann et al. (2008) was used. As the results of the analysis demonstrate, a large number of negative comments from referees in the areas “Relevance of contribution” and “Design/Conception” are clear signs that a manuscript rejected at AC-IE will not be published later in a high-impact journal. The number of negative statements in the areas “Writing/Presentation,” “Discussion of results,” “Method/Statistics,” and “Reference to the literature and documentation,” on the other hand, had no statistically significant influence on the probability that a rejected manuscript would later be published in a low- or high-impact journal. The results of this study have various implications for authors, journal editors and referees.
KeywordsJournal peer review Content analysis Thematic areas for manuscript review Fate of rejected manuscripts
- Bornmann, L., & Daniel, H.-D. (2008b). Selecting manuscripts for a high impact journal through peer review: a citation analysis of Communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology, 59(11), 1841–1852.CrossRefGoogle Scholar
- Bornmann, L., Nast, I., & Daniel, H.-D. (2008). Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics, 77(3), 415–432.CrossRefGoogle Scholar
- Daniel, H.-D. (1993/2004), Guardians of science. Fairness and reliability of peer review. Weinheim, Germany: Wiley-VCH. Published online 16 July 2004, Wiley Interscience, doi: 10.1002/3527602208.
- Dickersin, K., Ssemanda, E., Mansell, C., Rennie, D. (2007), What do the JAMA editors say when they discuss manuscripts that they are considering for publication? Developing a schema for classifying the content of editorial discussion. BMC Medical Research Methodology, 7, 44.Google Scholar
- Judge, T., Cable, D., Colbert, A., & Rynes, S. (2007). What causes a management article to be cited - article, author, or journal? The Academy of Management Journal (AMJ), 50(3), 491–506.Google Scholar
- LaFollette, M. C. (1992). Stealing into print: Fraud, plagiarism and misconduct in scientific publishing. Berkeley, CA, USA: University of California Press.Google Scholar
- Shashok, K. (2008). Content and communication: how can peer review provide helpful feedback about the writing? BMC Medical Research Methods, 8(3).Google Scholar
- StataCorp. (2007). Stata statistical software: release 10. College Station, TX, USA: Stata Corporation.Google Scholar
- Weller, A. C. (2002). Editorial peer review: Its strengths and weaknesses. Medford, NJ, USA: Information Today, Inc.Google Scholar
- Ziman, J. (1968). Public knowledge: An essay concerning the social dimension of science. Cambridge, UK: Cambridge University Press.Google Scholar