Advertisement

Scientometrics

, Volume 101, Issue 1, pp 717–735 | Cite as

The reviewer in the mirror: examining gendered and ethnicized notions of reciprocity in peer review

  • Bradford Demarest
  • Guo Freeman
  • Cassidy R. Sugimoto
Article

Abstract

Numerous studies have sought to uncover violations of objectivity and impartiality in peer review; however the notion of reciprocity has been absent in much of this discussion, particularly as it relates to gendered and ethnicized behaviors of peer review. The current study addresses this gap in research by investigating patterns of reciprocity (i.e., correspondences between patterns of recommendations received by authors and patterns of recommendations given by reviewers in the same social group) by perceived gender and ethnicity of reviewers and authors for submissions to the Journal of the American Society for Information Science and Technology from June 2009 to May 2011. The degree of reciprocity for each social group was examined by employing Monte Carlo resampling to extrapolate more robust patterns from the limited data available. We found that papers with female authors received more negative reviews than reviews for male authors. Reciprocity was suggested by the fact that female reviewers gave lower reviews than male reviewers. Reciprocity was also exhibited by ethnicity, although non-Western reviewers gave disproportionately more recommendations of major revision, while non-Western authors tended to receive more outright rejections. This study provides a novel theoretical and methodological basis for future studies on reciprocity in peer review.

Keywords

Reciprocity Peer review JASIST Scholarly communication Monte Carlo resampling 

Mathematics subject classification

C150 (Statistical Simulation Methods: General) 

JEL classification

62F40 (Bootstrap, jackknife and other resampling methods) 

Notes

Acknowledgments

The authors would like to thank JASIST Editor-in-Chief Blaise Cronin and Meghann Knowles (JASIST Editorial Office) for generously providing access to the data used in this study.

References

  1. Blank, R. M. (1991). The effects of double-blind versus single-blind reviewing: Experimental evidence from the American Economic Review. American Economic Review, 81(5), 1041–1068.Google Scholar
  2. Blau, P. (1964). Exchange and power in social life. New York: Wiley.Google Scholar
  3. Bornmann, L. (2011). Peer review and bibliometrics: Potentials and problems. In J. C. Shin, R. K. Toutkoushian, & U. Teichler (Eds.), University rankings: Theoretical basis, methodology and impacts on global higher education (pp. 145–164). Berlin: Springer.CrossRefGoogle Scholar
  4. Bornmann, L., & Daniel, H.-D. (2010). Do author-suggested reviewers rate submissions more favorably than editor-suggested reviewers? A study on atmospheric chemistry and physics. PLoS One, 5(10), e13345. doi: 10.1371/journal.pone.0013345.CrossRefGoogle Scholar
  5. Bornmann, L., Mutz, R., & Daniel, H.-D. (2007). Gender differences in grant peer review: A meta-analysis. Journal of Informetrics, 1, 226–238.CrossRefGoogle Scholar
  6. Bornmann, L., Mutz, R., & Daniel, H.-D. (2008). How to detect indications of potential sources of bias in peer review: A generalized latent variable modeling approach exemplified by a gender study. Journal of Informetrics, 2, 280–287.CrossRefGoogle Scholar
  7. Bornmann, L., Mutz, R., & Daniel, H.-D. (2009). The influence of the applicants’ gender on the modeling of a peer review process by using latent Markov models. Scientometrics, 81(2), 407–411.CrossRefGoogle Scholar
  8. Bornmann, L., Mutz, R., & Daniel, H.-D. (2010). A reliabilitygeneralization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants. PLoS One, 5(12), e14331.CrossRefGoogle Scholar
  9. Borsuk, R. M., Aarssen, L. W., Budden, A. E., Koricheva, J., Leimu, R., Tregenza, T., et al. (2009). To name or not to name: The effect of changing author gender on peer review. BioScience, 59(11), 985–989.CrossRefGoogle Scholar
  10. Budden, A. E., Tregenza, T., Aarssen, L. W., Koricheva, J., & Leimu, R. (2008). Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution, 23, 4–6.CrossRefGoogle Scholar
  11. Casement, P. J. (1991). Learning from the patient. New York: Guilford Press.Google Scholar
  12. Ceci, S. J., & Williams, W. M. (2011). Understanding current causes of women’s underrepresentation in science. Proceedings of the National Academy of Sciences of the United States of America, 108(8), 3157–3162.CrossRefGoogle Scholar
  13. Cronin, B., & McKenzie, G. (1992). The trajectory of rejection. Journal of Documentation, 48(3), 310–317.CrossRefGoogle Scholar
  14. Ernst, E., & Kienbacher, T. (1991). Chauvinism. Nature, 352, 560.CrossRefGoogle Scholar
  15. Falk, A., & Fischbacher, U. (2006). A theory of reciprocity. Games and Economic Behavior, 54(2), 293–315.MathSciNetCrossRefzbMATHGoogle Scholar
  16. Genealogy data: Frequently occurring surnames from Census 1990names files. (1990). Retrieved from http://www.census.gov/genealogy/www/data/1990surnames/names_files.html
  17. Genealogy main. Retrieved from http://www.census.gov/genealogy/www/index.html
  18. Gilbert, J. R., Williams, E. S., & Lundberg, G. D. (1994). Is there gender bias in JAMA’s peer review process? JAMA: The Journal of the American Medical Association, 272(2), 139–142.CrossRefGoogle Scholar
  19. Goldstein, W. N. (1991). Clarification of projective identification. American Journal of Psychiatry, 148(2), 153–161.Google Scholar
  20. Gouldner, A. W. (1960). The norm of reciprocity: A preliminary statement. American Sociological Review, 25, 161–178.CrossRefGoogle Scholar
  21. Hamilton, N. G. (1990). Splitting and projective identification among healthier individuals. American Journal of Psychotherapy, 44(3), 414–422.Google Scholar
  22. Homans, G. C. (1961). Social behavior: Its elementary forms. New York: Harcourt Brace.Google Scholar
  23. Klein, M. (1946). Notes on some schizoid mechanisms. International Journal of Psycho-Analysis, 27, 99–110.Google Scholar
  24. Lariviere, V., Ni, C., Gingras, Y., Cronin, B., & Sugimoto, C. R. (2013). Global gender disparities in science. Nature, 504(7479), 211–213.CrossRefGoogle Scholar
  25. Last name meaning and origins. Retrieved from http://www.ancestry.com/learn/facts
  26. Lee, C., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.CrossRefGoogle Scholar
  27. Link, A. M. (1998). US and non-US submissions. Journal of the American Medical Association, 280(3), 246–247.CrossRefGoogle Scholar
  28. Manski, C. F. (1993). Identification of endogenous social effects: The reflection problem. The Review of Economic Studies, 60(3), 531–542.MathSciNetCrossRefzbMATHGoogle Scholar
  29. Manzari, L. (2013). Library and information science journal prestige as assessed by library and information science faculty. The Library Quarterly, 83(1), 42–60. doi: 10.1086/668574.CrossRefGoogle Scholar
  30. Marsh, H. W., Bornmann, L., Mutz, R., Daniel, H.-D., & O’Mara, A. (2009). Gender effects in the peer reviews of grant proposals: A comprehensive meta-analysis comparing traditional and multilevel approaches. Review of Educational Research, 79, 1290–1326.CrossRefGoogle Scholar
  31. Marsh, H. W., Jayasinghe, U. W., & Bond, N. W. (2008). Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability. American Psychologist, 63(3), 160–168.CrossRefGoogle Scholar
  32. Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations. Chicago: University of Chicago Press.Google Scholar
  33. Moretti, E. (2011). Social learning and peer effects in consumption: Evidence from movie sales. The Review of Economic Studies, 78(1), 356–393.CrossRefGoogle Scholar
  34. Neuroscience, Nature. (2006). Women in neuroscience: A numbers game. Nature Neuroscience, 9(7), 853.CrossRefGoogle Scholar
  35. Nisonger, T. E., & Davis, C. H. (2005). The perception of library and information science journals by LIS education deans and ARL library directors: A replication of the Kohl-Davis study. College & Research Libraries, 66(4), 341–377.CrossRefGoogle Scholar
  36. Ogden, T. H. (1997). Reverie and interpretation: Sensing something human. Northvale: Jason Aronson.Google Scholar
  37. Oswald, A. J. (2008). Can we test for bias in scientific peer-review. IZA discussion paper 3665. Bonn: Institute for the Study of Labor.Google Scholar
  38. Perugini, M., Gallucci, M., Presaghi, F., & Ercolani, A. P. (2002). The personal norm of reciprocity. European Journal of Personality, 17(4), 251–283.CrossRefGoogle Scholar
  39. Peters, G. (2013). “Baby Name Guesser”. Retrieved from http://www.gpeters.com/names/baby-names.php Surname database. Retrieved from http://www.surnamedb.com
  40. Thibaut, J., & Kelley, H. H. (1959). The social psychology of groups. New York: Wiley.Google Scholar
  41. Valkonen, L., & Brooks, J. (2011). Gender balance in Cortex acceptance rates. Cortex, 47, 763–770.CrossRefGoogle Scholar
  42. Weller, A. C. (2002). Editorial peer review: its strengths and weaknesses. Medford: Information Today Inc.Google Scholar
  43. Wennerås, C., & Wold, A. (1997). Nepotism and sexism in peer-review. Nature, 387(6631), 341–343.CrossRefGoogle Scholar
  44. Wing, D. A., Benner, R. S., Petersen, R., Newcomb, R., & Scott, J. R. (2010). Differences in editorial board reviewer behavior based on gender. Journal of Women’s Health, 19(10), 1919–1923.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2014

Authors and Affiliations

  • Bradford Demarest
    • 1
  • Guo Freeman
    • 1
  • Cassidy R. Sugimoto
    • 1
  1. 1.School of Informatics and ComputingIndiana University BloomingtonBloomingtonUSA

Personalised recommendations