Advertisement

Order matters: Alphabetizing in-text citations biases citation rates

  • Jeffrey R. Stevens
  • Juan F. Duque
Brief Report

Abstract

Though citations are critical for communicating science and evaluating scholarly success, properties unrelated to the quality of the work—such as cognitive biases—can influence citation decisions. The primacy effect, in particular, is relevant to lists, which for in-text citations could result in citations earlier in the list receiving more attention than those later in the list. Therefore, how citations are ordered could influence which citations receive the most attention. Using a sample of 150,000 articles, we tested whether alphabetizing in-text citations biases readers into citing more often articles with first authors whose surnames begin with letters early in the alphabet. We found that surnames earlier in the alphabet were cited more often than those later in the alphabet when journals ordered citations alphabetically compared with chronologically or numerically. This effect seemed to be stronger in psychology journals (which have a culture of alphabetizing citations) compared with biology or geoscience journals (which primarily order chronologically or numerically) and was strongest among moderately and highly cited articles. Therefore, alphabetizing in-text citations biases citation decisions toward authors with surnames occurring early in the alphabet. These citation decisions result from an interaction between cognitive biases (more attention devoted to items earlier in a list) and the structure of the citation environment (the style in which citations are ordered). We suggest that journals using alphabetically ordered citations switch to chronological ordering to minimize this arbitrary alphabetical citation bias.

Keywords

Alphabetical order Bounded rationality Chronological order Citation decisions Citation style Primacy effect 

Supplementary material

13423_2018_1532_MOESM1_ESM.pdf (1.2 mb)
ESM 1 (PDF 1211 kb)
13423_2018_1532_MOESM2_ESM.csv (9.5 mb)
ESM 2 (CSV 9693 kb)
13423_2018_1532_MOESM3_ESM.r (48 kb)
ESM 3 (R 48 kb)

References

  1. American Psychological Association. (2010). Publication manual of the American Psychological Association (4th). Washington, DC: American Psychological Association.Google Scholar
  2. Barabási, A.-L., Song, C., & Wang, D. (2012). Handful of papers dominates citation. Nature, 491(7422), 40.CrossRefGoogle Scholar
  3. Berger, J. (2016). Does presentation order impact choice after delay? Topics in Cognitive Science, 8(3), 670–684.CrossRefGoogle Scholar
  4. Bigham, J. (1894). Studies from the Harvard Psychological Laboratory (II): B. Memory. Psychological Review, 1(5), 453–461.Google Scholar
  5. Clauss, M., Müller, D. W. H., & Codron, D. (2013). Source references and the scientist’s mind-map: Harvard vs. Vancouver style. Journal of Scholarly Publishing, 44(3), 274–282.CrossRefGoogle Scholar
  6. Dubin, D. (2004). The most influential paper Gerard Salton never wrote. Library Trends, 52(4), 748–764.Google Scholar
  7. Einav, L., & Yariv, L. (2006). What’s in a surname? The effects of surname initials on academic success. Journal of Economic Perspectives, 20(1), 175–187.CrossRefGoogle Scholar
  8. Ellison, G. (2010). How does the market use citation data? The Hirsch Index in economics (Working Paper No. 16419). Cambridge: National Bureau of Economic Research.CrossRefGoogle Scholar
  9. Feenberg, D., Ganguli, I., Gaulé, P., & Gruber, J. (2017). It’s good to be first: Order bias in reading and citing NBER working papers. The Review of Economics and Statistics, 99(1), 32–39.CrossRefGoogle Scholar
  10. Huang, W. (2015). Do ABCs get more citations than XYZs? Economic Inquiry, 53(1), 773–789.CrossRefGoogle Scholar
  11. Lehmann, S., Jackson, A. D., & Lautrup, B. E. (2006). Measures for measures. Nature, 444(7122), 1003–1004.CrossRefGoogle Scholar
  12. Murdock, B. B. J. (1962). The serial position effect of free recall. Journal of Experimental Psychology, 64(5), 482–488.CrossRefGoogle Scholar
  13. Price, D. D. S. (1976). A general theory of bibliometric and other cumulative advantage processes. Journal of the American Society for Information Science, 27(5), 292–306.CrossRefGoogle Scholar
  14. R Core Team. (2018). R: A language and environment for statistical computing [Computer software]. Vienna: R Foundation for Statistical Computing.Google Scholar
  15. Rouder, J. N., & Morey, R. D. (2012). Default Bayes factors for model selection in regression. Multivariate Behavioral Research, 47(6), 877–903.CrossRefGoogle Scholar
  16. Simon, H. A. (1955). A behavioral model of rational choice. Quarterly Journal of Economics, 69(1), 99–118.CrossRefGoogle Scholar
  17. Simon, H. A. (1990). Invariants of human behavior. Annual Review of Psychology, 41, 1–20.CrossRefGoogle Scholar
  18. Thelwall, M., & Wilson, P. (2014). Regression for citation data: An evaluation of different methods. Journal of Informetrics, 8(4), 963–971.CrossRefGoogle Scholar
  19. van Doorn, J., Ly, A., Marsman, M., & Wagenmakers, E.-J. (2018). Bayesian inference for Kendall’s rank correlation coefficient. The American Statistician.  https://doi.org/10.1080/00031305.2016.1264998
  20. Van Praag, C. M., & Van Praag, B. M. S. (2008). The benefits of being economics Professor A (rather than Z). Economica, 75(300), 782–796.CrossRefGoogle Scholar
  21. Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14(5), 779–804.CrossRefGoogle Scholar
  22. Wagenmakers, E.-J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., … Morey, R. D. (2018). Bayesian inference for psychology. Part II: Example applications with JASP. Psychonomic Bulletin & Review, 25(1), 58–76.CrossRefGoogle Scholar
  23. Weber, M. (2018) The effects of listing authors in alphabetical order: A review of the empirical evidence. Research Evaluation, 27(3), 238–245.  https://doi.org/10.1093/reseval/rvy008.CrossRefGoogle Scholar
  24. Williams, R. B. (2011). Citation systems in the biosciences: A history, classification and descriptive terminology. Journal of Documentation, 67(6), 995–1014.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  1. 1.Department of Psychology, Center for Brain, Biology & BehaviorUniversity of Nebraska–LincolnLincolnUSA
  2. 2.Department of PsychologyArcadia UniversityGlensideUSA

Personalised recommendations