Numerous studies have sought to uncover violations of objectivity and impartiality in peer review; however the notion of reciprocity has been absent in much of this discussion, particularly as it relates to gendered and ethnicized behaviors of peer review. The current study addresses this gap in research by investigating patterns of reciprocity (i.e., correspondences between patterns of recommendations received by authors and patterns of recommendations given by reviewers in the same social group) by perceived gender and ethnicity of reviewers and authors for submissions to the Journal of the American Society for Information Science and Technology from June 2009 to May 2011. The degree of reciprocity for each social group was examined by employing Monte Carlo resampling to extrapolate more robust patterns from the limited data available. We found that papers with female authors received more negative reviews than reviews for male authors. Reciprocity was suggested by the fact that female reviewers gave lower reviews than male reviewers. Reciprocity was also exhibited by ethnicity, although non-Western reviewers gave disproportionately more recommendations of major revision, while non-Western authors tended to receive more outright rejections. This study provides a novel theoretical and methodological basis for future studies on reciprocity in peer review.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Blank, R. M. (1991). The effects of double-blind versus single-blind reviewing: Experimental evidence from the American Economic Review. American Economic Review, 81(5), 1041–1068.
Blau, P. (1964). Exchange and power in social life. New York: Wiley.
Bornmann, L. (2011). Peer review and bibliometrics: Potentials and problems. In J. C. Shin, R. K. Toutkoushian, & U. Teichler (Eds.), University rankings: Theoretical basis, methodology and impacts on global higher education (pp. 145–164). Berlin: Springer.
Bornmann, L., & Daniel, H.-D. (2010). Do author-suggested reviewers rate submissions more favorably than editor-suggested reviewers? A study on atmospheric chemistry and physics. PLoS One, 5(10), e13345. doi:10.1371/journal.pone.0013345.
Bornmann, L., Mutz, R., & Daniel, H.-D. (2007). Gender differences in grant peer review: A meta-analysis. Journal of Informetrics, 1, 226–238.
Bornmann, L., Mutz, R., & Daniel, H.-D. (2008). How to detect indications of potential sources of bias in peer review: A generalized latent variable modeling approach exemplified by a gender study. Journal of Informetrics, 2, 280–287.
Bornmann, L., Mutz, R., & Daniel, H.-D. (2009). The influence of the applicants’ gender on the modeling of a peer review process by using latent Markov models. Scientometrics, 81(2), 407–411.
Bornmann, L., Mutz, R., & Daniel, H.-D. (2010). A reliabilitygeneralization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants. PLoS One, 5(12), e14331.
Borsuk, R. M., Aarssen, L. W., Budden, A. E., Koricheva, J., Leimu, R., Tregenza, T., et al. (2009). To name or not to name: The effect of changing author gender on peer review. BioScience, 59(11), 985–989.
Budden, A. E., Tregenza, T., Aarssen, L. W., Koricheva, J., & Leimu, R. (2008). Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution, 23, 4–6.
Casement, P. J. (1991). Learning from the patient. New York: Guilford Press.
Ceci, S. J., & Williams, W. M. (2011). Understanding current causes of women’s underrepresentation in science. Proceedings of the National Academy of Sciences of the United States of America, 108(8), 3157–3162.
Cronin, B., & McKenzie, G. (1992). The trajectory of rejection. Journal of Documentation, 48(3), 310–317.
Ernst, E., & Kienbacher, T. (1991). Chauvinism. Nature, 352, 560.
Falk, A., & Fischbacher, U. (2006). A theory of reciprocity. Games and Economic Behavior, 54(2), 293–315.
Family name history. Retrieved from http://genealogy.familyeducation.com/family-names-surnames/meaning-origin
Genealogy data: Frequently occurring surnames from Census 1990 – names files. (1990). Retrieved from http://www.census.gov/genealogy/www/data/1990surnames/names_files.html
Genealogy main. Retrieved from http://www.census.gov/genealogy/www/index.html
Gilbert, J. R., Williams, E. S., & Lundberg, G. D. (1994). Is there gender bias in JAMA’s peer review process? JAMA: The Journal of the American Medical Association, 272(2), 139–142.
Goldstein, W. N. (1991). Clarification of projective identification. American Journal of Psychiatry, 148(2), 153–161.
Gouldner, A. W. (1960). The norm of reciprocity: A preliminary statement. American Sociological Review, 25, 161–178.
Hamilton, N. G. (1990). Splitting and projective identification among healthier individuals. American Journal of Psychotherapy, 44(3), 414–422.
Homans, G. C. (1961). Social behavior: Its elementary forms. New York: Harcourt Brace.
Klein, M. (1946). Notes on some schizoid mechanisms. International Journal of Psycho-Analysis, 27, 99–110.
Lariviere, V., Ni, C., Gingras, Y., Cronin, B., & Sugimoto, C. R. (2013). Global gender disparities in science. Nature, 504(7479), 211–213.
Last name meaning and origins. Retrieved from http://www.ancestry.com/learn/facts
Lee, C., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.
Link, A. M. (1998). US and non-US submissions. Journal of the American Medical Association, 280(3), 246–247.
Manski, C. F. (1993). Identification of endogenous social effects: The reflection problem. The Review of Economic Studies, 60(3), 531–542.
Manzari, L. (2013). Library and information science journal prestige as assessed by library and information science faculty. The Library Quarterly, 83(1), 42–60. doi:10.1086/668574.
Marsh, H. W., Bornmann, L., Mutz, R., Daniel, H.-D., & O’Mara, A. (2009). Gender effects in the peer reviews of grant proposals: A comprehensive meta-analysis comparing traditional and multilevel approaches. Review of Educational Research, 79, 1290–1326.
Marsh, H. W., Jayasinghe, U. W., & Bond, N. W. (2008). Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability. American Psychologist, 63(3), 160–168.
Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations. Chicago: University of Chicago Press.
Moretti, E. (2011). Social learning and peer effects in consumption: Evidence from movie sales. The Review of Economic Studies, 78(1), 356–393.
Neuroscience, Nature. (2006). Women in neuroscience: A numbers game. Nature Neuroscience, 9(7), 853.
Nisonger, T. E., & Davis, C. H. (2005). The perception of library and information science journals by LIS education deans and ARL library directors: A replication of the Kohl-Davis study. College & Research Libraries, 66(4), 341–377.
Ogden, T. H. (1997). Reverie and interpretation: Sensing something human. Northvale: Jason Aronson.
Oswald, A. J. (2008). Can we test for bias in scientific peer-review. IZA discussion paper 3665. Bonn: Institute for the Study of Labor.
Perugini, M., Gallucci, M., Presaghi, F., & Ercolani, A. P. (2002). The personal norm of reciprocity. European Journal of Personality, 17(4), 251–283.
Thibaut, J., & Kelley, H. H. (1959). The social psychology of groups. New York: Wiley.
Valkonen, L., & Brooks, J. (2011). Gender balance in Cortex acceptance rates. Cortex, 47, 763–770.
Weller, A. C. (2002). Editorial peer review: its strengths and weaknesses. Medford: Information Today Inc.
Wennerås, C., & Wold, A. (1997). Nepotism and sexism in peer-review. Nature, 387(6631), 341–343.
Wing, D. A., Benner, R. S., Petersen, R., Newcomb, R., & Scott, J. R. (2010). Differences in editorial board reviewer behavior based on gender. Journal of Women’s Health, 19(10), 1919–1923.
The authors would like to thank JASIST Editor-in-Chief Blaise Cronin and Meghann Knowles (JASIST Editorial Office) for generously providing access to the data used in this study.
About this article
Cite this article
Demarest, B., Freeman, G. & Sugimoto, C.R. The reviewer in the mirror: examining gendered and ethnicized notions of reciprocity in peer review. Scientometrics 101, 717–735 (2014). https://doi.org/10.1007/s11192-014-1354-z
- Peer review
- Scholarly communication
- Monte Carlo resampling
Mathematics subject classification
- C150 (Statistical Simulation Methods: General)
- 62F40 (Bootstrap, jackknife and other resampling methods)