Correlation between the Journal Impact Factor and three other journal citation indices
- 546 Downloads
To determine the degree of correlation among journal citation indices that reflect the average number of citations per article, the most recent journal ratings were downloaded from the websites publishing four journal citation indices: the Institute of Scientific Information’s journal impact factor index, Eigenfactor’s article influence index, SCImago’s journal rank index and Scopus’ trend line index. Correlations were determined for each pair of indices, using ratings from all journals that could be identified as having been rated on both indices. Correlations between the six possible pairings of the four indices were tested with Spearman’s rho. Within each of the six possible pairings, the prevalence of identifiable errors was examined in a random selection of 10 journals and among the 10 most discordantly ranked journals on the two indices. The number of journals that could be matched within each pair of indices ranged from 1,857 to 6,508. Paired ratings for all journals showed strong to very strong correlations, with Spearman’s rho values ranging from 0.61 to 0.89, all p < 0.001. Identifiable errors were more common among scores for journals that had very discordant ranks on a pair of indices. These four journal citation indices were significantly correlated, providing evidence of convergent validity (i.e. they reflect the same underlying construct of average citability per article in a journal). Discordance in the ranking of a journal on two indices was in some cases due to an error in one index.
KeywordsBibliometric analysis Citation analysis Impact factor
We thank Jessica Patton and Narelle Cox for their assistance with retrieval of the STL scores and checking for errors.
- Adam, D. (2002). The counting house. Nature, 415(6873), 726–729.Google Scholar
- Banks, M. A., & Dellavalle, R. (2008). Emerging alternatives to the impact factor. OCLC Systems & Services: International Digital Library Perspectives, 24(3), 167–173.Google Scholar
- Cheung, C. K. (2008). Audience matters: A study of how authors select educational journals. The Asia-Pacific Education Researcher, 17(2), 191–201.Google Scholar
- Cozby, P. C. (2008). Methods in Behavioral Research (10 ed.). New York: McGraw Hill.Google Scholar
- Franceschet, M. (2010). Journal influence factors. Journal of Informetrics. doi: 10.1016/j.joi.2009.12.002.
- Hernán, M. A. (2008). Epidemiologists (of all people) should question journal impact factors. Epidemiology, 19, 366–368.Google Scholar
- Kurmis, A. P. (2003). Understanding the limitations of the journal impact factor. The Journal of Bone and Joint Surgery, 85, 2449–2454.Google Scholar
- Midorikawa, N., et al. (1984). The relationships among the citation measures and the factors influence on them. Information Services and Use, 4(6), 417–424.Google Scholar
- Monastersky, R. (2005). The number that’s devouring science. The Chronicle of Higher Education, 52(8), A12–A22.Google Scholar
- PLoS Medicine Editors. (2006). The impact factor game. PLoS Medicine, 3(6), p. e291, 1–2.Google Scholar
- Reedijk, J. (1998). Sense and nonsense of science citation analyses: Comments on the monopoly position of ISI and citation inaccuracies. Risks of possible misuse and biased citation and impact data. New Journal of Chemistry, 22, 767–770.Google Scholar
- Rousseau, R., & STIMULATE 8 Group. (2009). On the relation between the WoS impact factor, the eigenfactor, the SCImago journal rank, the article influence score and the journal h-index. http://eprints.rclis.org/16448/. Accessed 4 July 2009.
- Yue, W., Wilson, C. S., & Rousseau, R. (2004). The immediacy index and the journal impact factor: Two highly correlated derived measures. The Canadian Journal of Infomation and Library Science, 28(1), 33–48.Google Scholar