Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor

Abstract

The journal Impact Factor (IF) is not comparable among fields of science and social science because of systematic differences in publication and citation behaviour across disciplines. In this work, a decomposing of the field aggregate impact factor into five normally distributed variables is presented. Considering these factors, a principal component analysis is employed to find the sources of the variance in the Journal Citation Reports (JCR) subject categories of science and social science. Although publication and citation behaviour differs largely across disciplines, principal components explain more than 78 % of the total variance and the average number of references per paper is not the primary factor explaining the variance in impact factors across categories. The categories normalized impact factor based on the JCR subject category list is proposed and compared with the IF. This normalization is achieved by considering all the indexing categories of each journal. An empirical application, with one hundred journals in two or more subject categories of economics and business, shows that the gap between rankings is reduced around 32 % in the journals analyzed. This gap is obtained as the maximum distance among the ranking percentiles from all categories where each journal is included.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2009). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, 60(1), 27–34.

    Article  Google Scholar 

  2. Bensman, S. J. (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, 41(1), 93–155.

    Article  Google Scholar 

  3. Bergstrom, C. (2007). Eigenfactor: Measuring the value and prestige of scholarly journals. College and Research Libraries News, 68(5), 314.

    Google Scholar 

  4. Bornmann, L., & Daniel, H. D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.

    Article  Google Scholar 

  5. Chen, J., & Konstan, J. A. (2010). Conference paper selectivity and impact. Communications of the ACM, 53(6), 79–83.

    Article  Google Scholar 

  6. Dorta-González, P., & Dorta-González, M. I. (2010). Indicador bibliométrico basado en el índice h. Revista Española de Documentación Científica, 33(2), 225–245.

    Article  Google Scholar 

  7. Dorta-González, P., & Dorta-González, M. I. (2011a). Aplicación empírica de un indicador bibliométrico basado en el índice h. Cultura y Educación, 23(2), 297–313.

    Article  Google Scholar 

  8. Dorta-González, P., & Dorta-González, M. I. (2011b). Central indexes to the citation distribution: A complement to the h-index. Scientometrics, 88(3), 729–745.

    Article  Google Scholar 

  9. Egghe, L., & Rousseau, R. (2002). A general framework for relative impact indicators. Canadian Journal of Information and Library Science, 27(1), 29–48.

    Google Scholar 

  10. Freyne, J., Coyle, L., Smyth, B., & Cunningham, P. (2010). Relative status of journal and conference publications in computer science. Communications of the ACM, 53(11), 124–132.

    Article  Google Scholar 

  11. Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178(4060), 471–479.

    Article  Google Scholar 

  12. Garfield, E. (1979a). Citation indexing: Its theory and application in science, technology, and humanities. New York: Wiley.

    Google Scholar 

  13. Garfield, E. (1979b). Is citation analysis a legitimate evaluation tool? Scientometrics, 1(4), 359–375.

    Article  Google Scholar 

  14. González-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2011). A new approach to the metric of journals’ scientific prestige: The SJR indicator. Journal of Informetrics, 4(3), 379–391.

    Article  Google Scholar 

  15. Leydesdorff, L. (2006). Can scientific journals be classified in terms of aggregated journal–journal citation relations using the Journal Citation Reports? Journal of the American Society for Information Science and Technology, 57(5), 601–613.

    Article  Google Scholar 

  16. Leydesdorff, L., & Bornmann, (2011). How fractional counting of citations affects the Impact Factor: Normalization in terms of differences in citation potentials among fields of science. Journal of the American Society for Information Science and Technology, 62(2), 217–229.

    Article  Google Scholar 

  17. Leydesdorff, L., & Opthof, T. (2010a). Normalization at the field level: Fractional counting of citations. Journal of Informetrics, 4(4), 644–646.

    Article  Google Scholar 

  18. Leydesdorff, L., & Opthof, T. (2010b). Scopus’s source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology, 61(11), 2365–2369.

    Article  Google Scholar 

  19. Leydesdorff, L., & Rafols, I. (2011). Indicators of the interdisciplinarity of journals: Diversity, centrality, and citations. Journal of Informetrics, 5(1), 87–100.

    Article  Google Scholar 

  20. Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 265–277.

    Article  Google Scholar 

  21. Opthof, T., & Leydesdorff, L. (2010). Caveats for the journal and field normalizations in the CWTS (“Leiden”) evaluations of research performance. Journal of Informetrics, 4(3), 423–430.

    Article  Google Scholar 

  22. Pudovkin, A. I., & Garfield, E. (2002). Algorithmic procedure for finding semantically related journals. Journal of the American Society for Information Science and Technology, 53(13), 1113–1119.

    Article  Google Scholar 

  23. Rafols, I., & Leydesdorff, L. (2009). Content-based and algorithmic classifications of journals: Perspectives on the dynamics of scientific communication and indexer effects. Journal of the American Society for Information Science and Technology, 60(9), 1823–1835.

    Article  Google Scholar 

  24. Ramírez, A. M., García, E. O., & Del Río, J. A. (2000). Renormalized impact factor. Scientometrics, 47(1), 3–9.

    Article  Google Scholar 

  25. Rosvall, M., & Bergstrom, C. T. (2008). Maps of random walks on complex networks reveal community structure. Proceedings of the National Academy of Sciences, 105(4), 1118–1123.

    Article  Google Scholar 

  26. Rosvall, M., & Bergstrom, C. T. (2010). Mapping change in large networks. PLoS ONE, 5(1), e8694.

    Article  Google Scholar 

  27. Sombatsompop, N., & Markpin, T. (2005). Making an equality of ISI impact factors for different subject fields. Journal of the American Society for Information Science and Technology, 56(7), 676–683.

    Article  Google Scholar 

  28. Van Raan, A. F. J., Van Leeuwen, T. N., Visser, M. S., Van Eck, N. J., & Waltman, L. (2010). Rivals for the crown: Reply to Opthof and Leydesdorff. Journal of Informetrics, 4(3), 431–435.

    Article  Google Scholar 

  29. Wagner, C., Roessner, J. D., Bobb, K., Klein, J., Boyack, K., Keyton, J., et al. (2011). Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature. Journal of Informetrics, 5(1), 14–26.

    Article  Google Scholar 

  30. Waltman, L., & Van Eck, N. J. (2010). The relation between Eigenfactor, audience factor, and influence weight. Journal of the American Society for Information Science and Technology, 61(7), 1476–1486.

    Article  Google Scholar 

  31. Waltman, L., Yan, E., & Van Eck, N. J. (2011). A recursive field-normalized bibliometric performance indicator: An application to the field of library and information science. Scientometrics, 89(1), 301–314.

    Article  Google Scholar 

  32. Zitt, M., & Small, H. (2008). Modifying the journal impact factor by fractional citation weighting: The audience factor. Journal of the American Society for Information Science and Technology, 59(11), 1856–1860.

    Article  Google Scholar 

Download references

Acknowledgments

This research has been supported by the Ministry of Science and Technology of Spain under the research project ECO2008-05589.

Author information

Affiliations

Authors

Corresponding author

Correspondence to P. Dorta-González.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Tables (DOC 441 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Dorta-González, P., Dorta-González, M.I. Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor. Scientometrics 95, 645–672 (2013). https://doi.org/10.1007/s11192-012-0929-9

Download citation

Keywords

  • Citation
  • Impact factor
  • Journal evaluation
  • Source normalized indicator
  • JCR subject categories