, Volume 100, Issue 1, pp 245–260 | Cite as

Extending citer analysis to journal impact evaluation

  • Kun Lu
  • Isola Ajiferuke
  • Dietmar Wolfram


The concept of citer analysis investigated earlier by Ajiferuke and Wolfram (In: B. Larsen, J. Leta (eds.) Proceedings of the 12th international conference of the international society for scientometrics and informetrics (ISSI) pp. 798–808, 2009, Scientometrics 83:623–638, 2010) is extended to journals where different citing units (citers, citing articles, citing journals) are compared with the journal impact factor and each other to determine if differences in ranking arise from different measures. The citer measures for the 31 high impact journals studied from information science and library science are significantly correlated, even more so than the earlier citer analysis findings, indicating that there is a close relationship among the different units of measure. Still, notable differences in rankings for the journals examined were evident for the different measures used, especially from either 5-year impact factor or number of citing articles per publication to the number of citing journals per publication. The journals that are adversely affected seem to be those whose citations are concentrated in a few journals. This informed the need to develop a journal citation concentration index, which can serve as a complementary measure to the existing journal impact indices.


Journal citer analysis Citation analysis Journal impact factor Journal citation concentration index 

JEL Classification

C4 Econometric and statistical methods and methodology Special topics 



This study represents an expanded version of a paper presented at the 14th International Society for Scientometrics and Informetrics Conference held in Vienna in July 2013.


  1. Ajiferuke, I., Lu, K., & Wolfram, D. (2010). A comparison of citer and citation-based measure outcomes for multiple disciplines. Journal of the American Society for Information Science and Technology, 61(10), 2086–2096.CrossRefGoogle Scholar
  2. Ajiferuke, I., Lu, K., & Wolfram, D. (2011). Who are the disciples of an author? Examining recitation and oeuvre citation exahaustivity. Journal of Informetrics, 5(2), 292–302.CrossRefGoogle Scholar
  3. Ajiferuke, I., & Wolfram, D. (2009). Citer analysis as a measure of research impact: Library and information science as a case study. In B. Larsen & J. Leta (Eds.), Proceedings of the 12th international conference of the international society for scientometrics and informetrics (ISSI) (pp. 798–808). Rio de Janeiro.Google Scholar
  4. Ajiferuke, I., & Wolfram, D. (2010). Citer analysis as a measure of research impact: library and information science as a case study. Scientometrics, 83(3), 623–638.CrossRefGoogle Scholar
  5. Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2008). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, 60(1), 27–34.CrossRefGoogle Scholar
  6. Archambault, E., & Larivière, V. (2009). History of the journal impact factor: contingencies and consequences. Scientometrics, 79, 635–649.CrossRefGoogle Scholar
  7. Bergstrom, C. T., West, J. D., & Wiseman, M. A. (2008). The Eigenfactor metrics. The Journal of Neuroscience, 28(45), 11433–11434.CrossRefGoogle Scholar
  8. Bollen, J., Van de Sompel, H., Smith, J. A., & Luce, R. (2005). Toward alternative metrics of journal impact: a comparison of download and citation data. Information Processing and Management, 41(6), 1419–1440.CrossRefGoogle Scholar
  9. Dieks, D., & Chang, H. (1976). Differences in impact of scientific publications: some indices derived from a citation analysis. Social Studies of Science, 6, 247–267.CrossRefGoogle Scholar
  10. Egghe, L. (2012). A rationale for the relation between the citer h-index and the classical h-index of a researcher. Scientometrics, 1-4.Google Scholar
  11. Franceschini, F., Maisano, D., Perotti, A., & Proto, A. (2010). Analysis of the ch-index: an indicator to evaluate the diffusion of scientific research output by citers. Scientometrics, 85(1), 203–217.CrossRefGoogle Scholar
  12. Garfield, E., & Sher, I. H. (1963). New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14(3), 195–201.CrossRefGoogle Scholar
  13. Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.CrossRefGoogle Scholar
  14. Gonzalez-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals’ scientific prestige: the SJR indicator. Journal of Informetrics, 4(3), 379–391.CrossRefGoogle Scholar
  15. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102, 16569–16572.CrossRefGoogle Scholar
  16. Ioannidis, J.P. (2006). Concentration of the most-cited papers in the scientific literature: Analysis of journal ecosystems. PLoS One, 1(1), e5.
  17. MacRoberts, M. H., & MacRoberts, B. R. (1989). Problems of citation analysis: a critical review. Journal of the American Society for Information Science, 40(5), 342–349.CrossRefGoogle Scholar
  18. Moed, H. F., Colledge, L., Reedijk, J., Moya-Anegon, F., Guerrero-Bote, V., Plume, A., et al. (2012). Citation-based metrics are appropriate tools in journal assessment provided that they are accurate and used in an informed way. Scientometrics, 92(2), 367–376.CrossRefGoogle Scholar
  19. Pendlebury, D. A. (2009). The use and misuse of journal metrics and other citation indicators. Archivum Immunologiae et Therapiae Experimentalis, 57(1), 1–11.CrossRefGoogle Scholar
  20. Pratt, A. D. (1977). A measure of class concentration in bibliometrics. Journal of the American Society for Information Science, 28(5), 285–292.CrossRefGoogle Scholar
  21. Rousseau, R. (2002). Journal evaluation: technical and practical issues. Library Trends, 50(3), 418–439.Google Scholar
  22. Rousseau, R. (2012). Updating the journal impact factor or total overhaul? Scientometrics, 92, 413–417.CrossRefGoogle Scholar
  23. Smalheiser, N. R., & Torvik, V. I. (2009). Author name disambiguation. Annual Review of Information Science and Technology, 43(1), 1–43.CrossRefGoogle Scholar
  24. Strotmann, A., Zhao, D., & Bubela, T. (2009). Author name disambiguation for collaboration network analysis and visualization. Proceedings of the American Society for Information Science and Technology, 46(1), 1–20.Google Scholar
  25. Vanclay, J. K. (2012). Impact factor: outdated artefact or stepping-stone to journal certification? Scientometrics, 92, 211–238.CrossRefGoogle Scholar
  26. Zitt, M., & Bassecoulard, E. (1998). Internationalization of scientific journals: a measurement based on publication and citation scope. Scientometrics, 41(1), 255–271.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2014

Authors and Affiliations

  1. 1.School of Library and Information StudiesUniversity of OklahomaNormanU.S.A
  2. 2.Faculty of Information and Media StudiesUniversity of Western OntarioLondonCanada
  3. 3.School of Information StudiesUniversity of Wisconsin-MilwaukeeMilwaukeeU.S.A

Personalised recommendations