Advertisement

Scientometrics

, Volume 113, Issue 2, pp 1113–1127 | Cite as

Language and socioeconomics predict geographic variation in peer review outcomes at an ecology journal

  • C. Sean Burns
  • Charles W. Fox
Article

Abstract

Papers submitted by scientists located in western nations generally fare better in the peer review process than do papers submitted by scientists from elsewhere. This paper examines geographic variation in peer review outcomes (whether a manuscript is sent for review, review scores obtained, and final decisions by editors) for 3529 submissions over a 4.5 year period at the journal Functional Ecology. In particular, we test whether geographic variation in language and socioeconomics are adequate to explain most or are all of this variation. There was no relationship between the geographic regions of handling editors and the decisions to send papers for review or invite revision, but there was substantial variation among author geographic locations; generally papers from first authors located in Oceania, the United States, and the United Kingdom fared better, and papers from first authors located in Africa, Asia, and Latin America fared worst. Language and the Human Development Index (HDI) explained the geographic variation in the proportion of papers sent for review, but socioeconomics alone (HDI) was the best predictor of mean review scores obtained by papers and whether authors were invited to submit a revision. Though we cannot exclude a role for editor and reviewer biases against authors based on their geographic location, variation in socioeconomics and language explain much of the variation in manuscript editorial and peer review outcomes among authors from different regions of the world.

Keywords

Peer review Language bias Geographic bias Socioeconomics Human development index 

Mathematics Subject Classification

62P25 

JEL Classification

C12 C13 C14 

Notes

Acknowledgements

We thank the British Ecological Society (BES), owners of the journal Functional Ecology, for permitting us to use their peer review database for this project. Brandi Frisby provided comments on an earlier draft of this paper. This work was reviewed and approved by the Internal Review Board at the University of Kentucky, IRB 14-0570-P4S.

References

  1. Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi: 10.18637/jss.v067.i01.CrossRefGoogle Scholar
  2. Blackburn, J. L., & Hakel, M. D. (2006). An examination of sources of peer-review bias. Psychological Science, 17(5), 378–385.CrossRefGoogle Scholar
  3. Bornmann, L., & Daniel, H.-D. (2007). Gatekeepers of science—Effects of external reviewers’ attributes on the assessments of fellowship applications. Journal of Informetrics, 1, 83–91. doi: 10.1016/j.joi.2006.09.005.CrossRefGoogle Scholar
  4. Burgman, M., Jarrad, F., & Main, E. (2015). Decreasing geographic bias in conservation biology. Conservation Biology, 29(5), 1255–1256. doi: 10.1111/cobi.12589.CrossRefGoogle Scholar
  5. Burnham, K. P., Anderson, D. R., & Huyvaert, K. P. (2011). AIC model selection and multimodel inference in behavioral ecology: Some background, observations, and comparisons. Behavioral Ecology and Sociobiology, 65, 23–35. doi: 10.1007/s00265-010-1029-6.CrossRefGoogle Scholar
  6. Campos-Arceiz, A., Primack, R. B., & Koh, L. P. (2015). Reviewer recommendations and editors’ decisions for a conservation journal: Is it just a crapshoot? And do Chinese authors get a fair shot? Biological Conservation, 186, 22–27. doi: 10.1016/j.biocon.2015.02.025.CrossRefGoogle Scholar
  7. Clavero, M. (2011). Language bias in ecological studies. Frontiers in Ecology and the Environment, 9, 93–94. doi: 10.1890/11.wb.001.MathSciNetCrossRefGoogle Scholar
  8. Cronin, B. (2009). Vernacular and vehicular language. Journal of the American Society for Information Science and Technology, 60(3), 433. doi: 10.1002/asi.21010.CrossRefGoogle Scholar
  9. Daniel, H.-D. (1993). Fairness in manuscript evaluation (W. E. Russey, Trans.). In Guardians of science: fairness and reliability of peer review (pp. 29–46). Weinheim: VCH Verlagsgesellschaft.Google Scholar
  10. Ernst, E., & Kienbacher, T. (1991). Chauvinism. Nature, 352(6336), 560.CrossRefGoogle Scholar
  11. Fayaz-Bakhsh, A., & Mousavi, A. (2015). Science growth and human development index in Iran. Journal of Research in Medical Sciences, 20, 1218.CrossRefGoogle Scholar
  12. Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. Los Angeles: Sage.Google Scholar
  13. Flowerdew, J. (2001). Attitudes of journal editors to nonnative speaker contributions. TESOL Quarterly, 35(1), 121–150.CrossRefGoogle Scholar
  14. Fox, C. W., Albert, A. Y. K., & Vines, T. H. (2017a). Recruitment of reviewers is becoming harder at some journals: A test of the influence of reviewer fatigue at six journals in ecology and evolution. Research Integrity and Peer Review. doi: 10.1186/s41073-017-0027-x.Google Scholar
  15. Fox, C. W., & Burns, C. S. (2015). The relationship between manuscript title structure and success: Editorial decisions and citation performance for an ecological journal. Ecology and Evolution, 5, 1970–1980. doi: 10.1002/ece3.1480.CrossRefGoogle Scholar
  16. Fox, C. W., Burns, C. S., & Meyer, J. A. (2015). Data from: Editor and reviewer gender influence the peer review process but not peer review outcomes at an ecology journal. Dryad Digital Repository. doi: 10.5061/dryad.5090r.Google Scholar
  17. Fox, C. W., Burns, C. S., & Meyer, J. A. (2016a). Editor and reviewer gender influence the peer review process but not peer review outcomes at an ecology journal. Functional Ecology, 30, 140–153. doi: 10.1111/1365-2435.12529.CrossRefGoogle Scholar
  18. Fox, C. W., Burns, C. S., Muncy, A. D., & Meyer, J. A. (2016b). Gender differences in patterns of authorship do not affect peer review outcomes at an ecology journal. Functional Ecology, 30, 126–139. doi: 10.1111/1365-2435.12587.CrossRefGoogle Scholar
  19. Fox, C. W., Burns, C. S., Muncy, A. D., & Meyer, J. A. (2017b). Author-suggested reviewers: Gender differences and influences on the peer review process at an ecology journal. Functional Ecology. doi: 10.1111/1365-2435.12665.Google Scholar
  20. Fox, J., & Weisberg, S. (2011). An R companion to applied regression (2nd ed.). Thousand Oaks: Sage.Google Scholar
  21. Gálvez, A., Maqueda, M., Martínez-Bueno, M., & Valdivia, E. (2000). Scientific publication trends and the developing world: What can the volume of scientific articles tell us about scientific progress in various regions? American Scientist, 88(6), 526–533.CrossRefGoogle Scholar
  22. Goodman, S. N., Berlin, J., Fletcher, S. W., & Fletcher, R. H. (1994). Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Annals of Internal Medicine, 121, 11–21. doi: 10.7326/0003-4819-121-1-199407010-00003.CrossRefGoogle Scholar
  23. Harris, M., Macinko, J., Jimenez, G., Mahfoud, M., & Anderson, C. (2015). Does a research article’s country of origin affect perception of its quality and relevance? A national trial of US public health researchers. British Medical Journal Open. doi: 10.1136/bmjopen-2015-008993.Google Scholar
  24. Hilbe, J. M. (2009). Logistic regression models. Boca Raton: CRC Press.zbMATHGoogle Scholar
  25. Hosmer, D. W., & Lemeshow, S. (1989). Applied logistic regression. New York: Wiley.zbMATHGoogle Scholar
  26. Hsiehchen, D., & Espinoza, M. (2016). Detecting editorial bias in medical publishing. Scientometrics, 106, 453–456. doi: 10.1007/s11192-015-1753-9.CrossRefGoogle Scholar
  27. Harrell, Jr, F. E. (2016). Hmisc: Harrell miscellaneous. R package version 4.0-2. https://CRAN.R-project.org/package=Hmisc.
  28. Justice, A. C., Cho, M. K., Winker, M. A., Berlin, J. A., & Rennie, D. (1998). Does masking author identity improve peer review quality? A randomized controlled trial. JAMA, 280(3), 240–242. doi: 10.1001/jama.280.3.240.CrossRefGoogle Scholar
  29. Keiser, J. (2004). Representation of authors and editors from countries with difference human development indexes in the leading literature on tropical medicine: Survey of current evidence. BMJ. doi: 10.1136/bmj.38069.518137.F6.Google Scholar
  30. King, D. A. (2004). The scientific impact of nations. Nature, 430, 311–316. doi: 10.1038/430311a.CrossRefGoogle Scholar
  31. King, J. E. (2008). Binary logistic regression. In J. W. Osborne (Ed.), Best practices in quantitative methods (pp. 358–384). Chicago: Sage.CrossRefGoogle Scholar
  32. Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the Association for Information Science and Technology, 64(1), 2–17. doi: 10.1002/asi.22784.CrossRefGoogle Scholar
  33. Lesnoff, M., and Lancelot, R. (2012). aod: Analysis of overdispersed data. R package version 1.3. http://cran.r-project.org/package=aod.
  34. Link, A. M. (1998). US and non-US submissions. JAMA, 280, 246–247. doi: 10.1001/jama.280.3.246.CrossRefGoogle Scholar
  35. Man, J. P., Weinkauf, J. G., Tsang, M., & Sin, D. D. (2004). Why do some countries publish more than others? An international comparison of research funding, English proficiency and publication output in highly ranked general medical journals. European Journal of Epidemiology, 19(8), 811–817.CrossRefGoogle Scholar
  36. Meyer, D., Zeileis, A., and Hornick, K. (2016). Vcd: Visualizing categorical data. R package version 1.4-3.Google Scholar
  37. Naik, G. (2017). Peer-review activists push psychology journals toward open data. Nature. doi: 10.1038/nature.2017.21549.Google Scholar
  38. Opthof, T., Coronel, R., & Janse, M. J. (2002). The significance of the peer review process against the background of bias: priority ratings of reviewers and editors and the prediction of citation, the role of geographical bias. Cardiovascular Research, 56(3), 339–346. doi: 10.1016/S0008-6363(02)00712-5.CrossRefGoogle Scholar
  39. Osborne, J. W. (2008). Bringing balance and technical accuracy to reporting odds ratios and the results of logistic regression analysis. In J. W. Osborne (Ed.), Best practices in quantitative methods (pp. 385–389). Chicago: Sage.CrossRefGoogle Scholar
  40. Pedhazur, E. J. (1997). Multiple regression in behavioral research: Explanation and prediction (3rd ed.). South Melbourne: Thomson Learning.zbMATHGoogle Scholar
  41. Primack, R. B., Ellwood, E., Miller-Rushing, A. J., Marrs, R., & Mulligan, A. (2009). Do gender, nationality, or academic age affect review decisions? An analysis of submissions to the journal. Biological Conservation, 142, 2415–2418. doi: 10.1016/j.biocon.2009.06.021.CrossRefGoogle Scholar
  42. Primack, R. B., & Marrs, R. (2008). Bias in the review process. Biological Conservation, 141, 2919–2920. doi: 10.1016/j.biocon.2008.09.016.CrossRefGoogle Scholar
  43. R Core Team. (2016). R: A language and environment for statistical computing. (Version 3.3.1). R foundation for statistical computing. Available at http://www.R-project.org/.
  44. Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., et al. (2011). pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. doi: 10.1186/1471-2105-12-77.Google Scholar
  45. Ross, J. S., Gross, C. P., Desai, M. M., Hong, Y., et al. (2006). Effect of blinded peer review on abstract acceptance. JAMA, 295(14), 1675–1680. doi: 10.1001/jama.295.14.1675.CrossRefGoogle Scholar
  46. Treganza, T. (2002). Gender bias in the refereeing process? Trends in Ecology and Evolution, 17(8), 349–350. doi: 10.1016/S0169-5347(02)02545-4.CrossRefGoogle Scholar
  47. United Nations Development Programme. (2015). International human development indicators. Retrieved from http://hdr.undp.org/en/countries.
  48. Uthman, O. A., Wiysong, C. S., Ota, M. O., Nicol, M., Hussey, G. D., Ndumbe, P. M., et al. (2014). Increasing the value of health research in the WHO African Region beyond 2015—reflecting on the past, celebrating the present and building the future: a bibliometric analysis. British Medical Journal Open, 5, e006340. doi: 10.1136/bmjopen-2014-006340.Google Scholar
  49. Venables, W. N., & Ripley, B. D. (2002). Modern applied statistics with S (4th ed.). New York: Springer.CrossRefzbMATHGoogle Scholar
  50. Waheed, A. (2012). Why developing countries are lesser innovators. International Journal of Social, Behavioral, Educational, Business and Industrial Engineering, 6(7), 1686–1691.Google Scholar
  51. Wickham, H. (2007). Reshaping data with the reshape package. Journal of Statistical Software, 21, 1–20. http://www.jstatsoft.org/v21/i12/.
  52. Wickham, H. (2009). ggplot2: Elegant graphics for data analysis. New York: Springer.CrossRefzbMATHGoogle Scholar
  53. Wickham, H. (2011). The split-apply-combine strategy for data analysis. Journal of Statistical Software, 40, 1–29. http://www.jstatsoft.org/v40/i01/.
  54. Wickham, H., and Francois, R. (2016). dplyr: A grammar of data manipulation. R package version 0.5.0. https://CRAN.R-project.org/package=dplyr.
  55. Witze, A. (2016). Research gets increasingly international. Nature. doi: 10.1038/nature.2016.19198.Google Scholar
  56. Zambrano, E. (2014). An axiomatization of the human development index. Social Choice and Welfare, 42(4), 853–872. doi: 10.1007/s00355-013-0756-9.MathSciNetCrossRefzbMATHGoogle Scholar
  57. Zhang, X. (2012). Effect of reviewer’s origin on peer review: China vs. non-China. Learned Publishing, 25(4), 265–270. doi: 10.1087/20120405.MathSciNetCrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2017

Authors and Affiliations

  1. 1.School of Information ScienceUniversity of KentuckyLexingtonUSA
  2. 2.Department of EntomologyUniversity of KentuckyLexingtonUSA

Personalised recommendations