Skip to main content

Extending citer analysis to journal impact evaluation

Scientometrics volume 100, pages 245–260 (2014)Cite this article

Abstract

The concept of citer analysis investigated earlier by Ajiferuke and Wolfram (In: B. Larsen, J. Leta (eds.) Proceedings of the 12th international conference of the international society for scientometrics and informetrics (ISSI) pp. 798–808, 2009, Scientometrics 83:623–638, 2010) is extended to journals where different citing units (citers, citing articles, citing journals) are compared with the journal impact factor and each other to determine if differences in ranking arise from different measures. The citer measures for the 31 high impact journals studied from information science and library science are significantly correlated, even more so than the earlier citer analysis findings, indicating that there is a close relationship among the different units of measure. Still, notable differences in rankings for the journals examined were evident for the different measures used, especially from either 5-year impact factor or number of citing articles per publication to the number of citing journals per publication. The journals that are adversely affected seem to be those whose citations are concentrated in a few journals. This informed the need to develop a journal citation concentration index, which can serve as a complementary measure to the existing journal impact indices.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

  • Ajiferuke, I., Lu, K., & Wolfram, D. (2010). A comparison of citer and citation-based measure outcomes for multiple disciplines. Journal of the American Society for Information Science and Technology, 61(10), 2086–2096.

    Article  Google Scholar 

  • Ajiferuke, I., Lu, K., & Wolfram, D. (2011). Who are the disciples of an author? Examining recitation and oeuvre citation exahaustivity. Journal of Informetrics, 5(2), 292–302.

    Article  Google Scholar 

  • Ajiferuke, I., & Wolfram, D. (2009). Citer analysis as a measure of research impact: Library and information science as a case study. In B. Larsen & J. Leta (Eds.), Proceedings of the 12th international conference of the international society for scientometrics and informetrics (ISSI) (pp. 798–808). Rio de Janeiro.

  • Ajiferuke, I., & Wolfram, D. (2010). Citer analysis as a measure of research impact: library and information science as a case study. Scientometrics, 83(3), 623–638.

    Article  Google Scholar 

  • Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2008). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, 60(1), 27–34.

    Article  Google Scholar 

  • Archambault, E., & Larivière, V. (2009). History of the journal impact factor: contingencies and consequences. Scientometrics, 79, 635–649.

    Article  Google Scholar 

  • Bergstrom, C. T., West, J. D., & Wiseman, M. A. (2008). The Eigenfactor metrics. The Journal of Neuroscience, 28(45), 11433–11434.

    Article  Google Scholar 

  • Bollen, J., Van de Sompel, H., Smith, J. A., & Luce, R. (2005). Toward alternative metrics of journal impact: a comparison of download and citation data. Information Processing and Management, 41(6), 1419–1440.

    Article  Google Scholar 

  • Dieks, D., & Chang, H. (1976). Differences in impact of scientific publications: some indices derived from a citation analysis. Social Studies of Science, 6, 247–267.

    Article  Google Scholar 

  • Egghe, L. (2012). A rationale for the relation between the citer h-index and the classical h-index of a researcher. Scientometrics, 1-4.

  • Franceschini, F., Maisano, D., Perotti, A., & Proto, A. (2010). Analysis of the ch-index: an indicator to evaluate the diffusion of scientific research output by citers. Scientometrics, 85(1), 203–217.

    Article  Google Scholar 

  • Garfield, E., & Sher, I. H. (1963). New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14(3), 195–201.

    Article  Google Scholar 

  • Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.

    Article  Google Scholar 

  • Gonzalez-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals’ scientific prestige: the SJR indicator. Journal of Informetrics, 4(3), 379–391.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102, 16569–16572.

    Article  Google Scholar 

  • Ioannidis, J.P. (2006). Concentration of the most-cited papers in the scientific literature: Analysis of journal ecosystems. PLoS One, 1(1), e5. http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0000005.

  • MacRoberts, M. H., & MacRoberts, B. R. (1989). Problems of citation analysis: a critical review. Journal of the American Society for Information Science, 40(5), 342–349.

    Article  Google Scholar 

  • Moed, H. F., Colledge, L., Reedijk, J., Moya-Anegon, F., Guerrero-Bote, V., Plume, A., et al. (2012). Citation-based metrics are appropriate tools in journal assessment provided that they are accurate and used in an informed way. Scientometrics, 92(2), 367–376.

    Article  Google Scholar 

  • Pendlebury, D. A. (2009). The use and misuse of journal metrics and other citation indicators. Archivum Immunologiae et Therapiae Experimentalis, 57(1), 1–11.

    Article  Google Scholar 

  • Pratt, A. D. (1977). A measure of class concentration in bibliometrics. Journal of the American Society for Information Science, 28(5), 285–292.

    Article  Google Scholar 

  • Rousseau, R. (2002). Journal evaluation: technical and practical issues. Library Trends, 50(3), 418–439.

    Google Scholar 

  • Rousseau, R. (2012). Updating the journal impact factor or total overhaul? Scientometrics, 92, 413–417.

    Article  Google Scholar 

  • Smalheiser, N. R., & Torvik, V. I. (2009). Author name disambiguation. Annual Review of Information Science and Technology, 43(1), 1–43.

    Article  Google Scholar 

  • Strotmann, A., Zhao, D., & Bubela, T. (2009). Author name disambiguation for collaboration network analysis and visualization. Proceedings of the American Society for Information Science and Technology, 46(1), 1–20.

    Google Scholar 

  • Vanclay, J. K. (2012). Impact factor: outdated artefact or stepping-stone to journal certification? Scientometrics, 92, 211–238.

    Article  Google Scholar 

  • Zitt, M., & Bassecoulard, E. (1998). Internationalization of scientific journals: a measurement based on publication and citation scope. Scientometrics, 41(1), 255–271.

    Article  Google Scholar 

Download references

Acknowledgments

This study represents an expanded version of a paper presented at the 14th International Society for Scientometrics and Informetrics Conference held in Vienna in July 2013.

Author information

Authors and Affiliations

  1. School of Library and Information Studies, University of Oklahoma, Bizzell Library, Room 120, 401 West Brooks, Norman, OK, 73019-6032, U.S.A

    Kun Lu

  2. Faculty of Information and Media Studies, University of Western Ontario, London, ON, N6A 5B7, Canada

    Isola Ajiferuke

  3. School of Information Studies, University of Wisconsin-Milwaukee, P.O. Box 413, Milwaukee, WI, 53201, U.S.A

    Dietmar Wolfram

Authors
  1. Kun Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isola Ajiferuke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dietmar Wolfram
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Isola Ajiferuke.

Additional information

Kun Lu: This research was conducted while the author was a member of the faculty at the School of Information Management, Wuhan University, Wuhan, Hubei, China.

Appendix

Appendix

See Tables 9, 10, and 11.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lu, K., Ajiferuke, I. & Wolfram, D. Extending citer analysis to journal impact evaluation. Scientometrics 100, 245–260 (2014). https://doi.org/10.1007/s11192-014-1274-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-014-1274-y

Keywords

  • Journal citer analysis
  • Citation analysis
  • Journal impact factor
  • Journal citation concentration index

JEL Classification

  • C4 Econometric and statistical methods and methodology
  • Special topics