Journal Impact Factors for evaluating scientific performance: use of h-like indicators
This article introduces the Impact Factor squared or IF2-index, an h-like indicator of research performance. This indicator reflects the degree to which large entities such as countries and/or their states participate in top-level research in a field or subfield. The IF2-index uses the Journal Impact Factor (JIF) of research publications instead of the number of citations. This concept is applied to other h-type indexes and their results compared to the IF2-index. These JIF-based indexes are then used to assess the overall performance of cancer research in Australia and its states over 8 years from 1999 to 2006. The IF2-index has three advantages when evaluating larger research units: firstly, it provides a stable value that does not change over time, reflecting the degree to which a research unit participated in top-level research in a given year; secondly, it can be calculated closely approximating the publication date of yearly datasets; and finally, it provides an additional dimension when a full article-based citation analysis is not feasible. As the index reflects the degree of participation in top-level research it may favor larger units when units of different sizes are compared.
KeywordsJournal Impact Factor IF2-index Research evaluation Australia Cancer research h-Index h-Type indexes h-Like indexes Scientometrics
The authors acknowledge with thanks the support of the John Metcalfe Memorial Fund and helpful comments from the anonymous reviewers.
- Bornmann, L., Mutz, R., & Daniel, H.-D. (2008a). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.CrossRefGoogle Scholar
- Braun, T., Glänzel, W., & Schubert, A. (2005). A Hirsch-type index for journals. The Scientist, 19(22), 8.Google Scholar
- Burrell, Q. L. (2007a). Should the h-index be discounted? ISSI Newsletter, 3(S), 65–68.Google Scholar
- Burrell, Q. L. (2009). On Hirsch’s h, Egghe’s g and Kosmulski’s h(2). Scientometrics. doi: 10.1007/s11192-009-0405-3 [Online First].
- Egghe, L. (2010). The Hirsch index and related impact measures. Annual Review of Information Science and Technology, 44, 65–114.Google Scholar
- Jin, B. (2006). h-index: An evaluation indicator proposed by scientist. Science Focus, 1(1), 8–9.Google Scholar
- Kaltenborn, K.-F., & Kuhn, K. (2004). The journal impact factor as a parameter for the evaluation of researchers and research. Revista Española de Enfermedades Digestivas, 96(7), 460–476. http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1130-01082004000700004&lng=en&nrm=iso [Online].
- Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.Google Scholar
- Rousseau, R. (2006). New developments related to the Hirsch index. Accessed October 14, 2008, from http://eprints.rclis.org/archive/00006376/01/Hirsch_new_developments.pdf [Online].
- Rousseau, R. (2008). Reflections on recent developments of the h-index and h-type indices. In H. Kretschmer & F. Havemann (Eds.), Proceedings of WIS 2008. Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting. Berlin: HumboldtUniversität zu Berlin, Institute for Library and Information Science. http://www.collnet.de/Berlin-2008/RousseauWIS2008rrd.pdf [Online].
- Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91(1), 42–46.Google Scholar
- Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.Google Scholar
- Welberry, H., Edwards, C., Weston, A., Harvey, C., Wilson, C. S., Boell, S. K., Lo, M., & Bishop, J. K. (2008). Cancer research in New South Wales 2001–2006. Sydney: Cancer Institute NSW. http://www.cancerinstitute.org.au/cancer_inst/publications/pdfs/rm-2008-1_cancer-research-in-nsw-2001-2006.pdf [Online].
- Wilson, C. S. (2005). General analyses of cancer research publications in Australian states using the science and social science citations indexes. In P. Ingwersen & B. Larsen (Eds.), Proceedings of ISSI 2005 (pp. 168–176). Stockholm: Karolinska University Press.Google Scholar
- Wilson, C. S., Boell, S. K., & Lo, M. (2007). Systematic review of the quantity and quality of cancer research publications in New South Wales: 1999 to 2006. Sydney: University of New South Wales, School of Information Systems, Technology and Management.Google Scholar
- Wilson, C. S., & Pittman, S. (2000). Assessments of outputs: Quantity and quality of cancer research publications in New South Wales from 1994 to 1998. Sydney: The University of New South Wales, School of Information Systems, Technology and Management.Google Scholar
- Yue, W., Wilson, C. S., & Boller, F. (2007). Peer assessment of journal quality in clinical neurology. Journal of the Medical Library Association, 95(1), 70–76.Google Scholar