Citation analysis and peer ranking of Australian social science journals
- 513 Downloads
Citation analyses were performed for Australian social science journals to determine the differences between data drawn from Web of Science and Scopus. These data were compared with the tier rankings assigned by disciplinary groups to the journals for the purposes of a new research assessment model, Excellence in Research for Australia (ERA), due to be implemented in 2010. In addition, citation-based indicators including an extended journal impact factor, the h-index, and a modified journal diffusion factor, were calculated to assess whether subsequent analyses influence the ranking of journals. The findings suggest that the Scopus database provides higher number of citations for more of the journals. However, there appears to be very little association between the assigned tier ranking of journals and their rank derived from citations data. The implications for Australian social science researchers are discussed in relation to the use of citation analysis in the ERA.
KeywordsCitation analysis Social science journals Research Assessment Citation sources Australia Journal ranking
- Australian Bureau of Statistics. (2008). Australian and New Zealand Standard Research Classification. Retrieved September 16, 2009, from http://www.abs.gov.au/Ausstats/abs@.nsf/Latestproducts/6BB427AB9696C225CA2574180004463E?opendocument.
- Australian Research Council. (2008). ERA Indicator Principles. Retrieved December 21, 2008, from http://www.arc.gov.au/era/indicators.htm.
- Australian Research Council. (2009a). Draft ERA submission guidelines: Physical, Chemical and Earth Sciences (PCE) and Humanities and Creative Arts (HCA) clusters. Retrieved January 29, 2009, from http://www.arc.gov.au/pdf/Draft_ERA_Sub_Guide.pdf.
- Australian Research Council. (2009b). ERA Indicators Consultation Paper. Retrieved October 4, 2009, from http://www.arc.gov.au/pdf/ERA_indicators_consult.pdf.
- Bakkalbasi, N., Bauer, K., Glover, J., & Wang, L. (2006). Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries, 3(7).Google Scholar
- Bauer, K., & Bakkalbasi, N. (2005). An examination of citation counts in a new scholarly communication environment. D-Lib Magazine, 11(9), [np].Google Scholar
- Bosman, J., Van Mourik, I., Rasch, M., Sieverts, E., & Verhoeff, H. (2006). Scopus Reviewed and Compared: The Coverage and Functionality of the Citation Database Scopus, Including Comparisons with Web of Science and Google Scholar. Utrecht: Utrecht University Library, http://igitur-archive.library.uu.nl/DARLIN/2006-1220-200432/UUindex.html.
- Bourke, P. (1994, 19–25 May). Quantitative research indicators: Citations treatment ‘deficient’. Campus Review, p. 9.Google Scholar
- Butler, L. (2006). RQF Pilot Study Project: History and Political Science: Methodology for Citation Analysis: REPP. Retrieved September 8, 2009, from http://www.chass.org.au/papers/pdf/PAP20061102LB.pdf.
- De Solla Price, D. J. (1970). Citation measures of hard science, soft science, technology, and nonscience. In C. E. Nelson & D. K. Pollock (Eds.), Communication Among Scientists and Engineers (pp. 3–22). Lexington, MA: Heath Lexington Books.Google Scholar
- Genoni, P., & Haddow, G. (2009). ERA and the ranking of Australian humanities journals. Australian Humanities Review, 46((May)), 7–26.Google Scholar
- Haddow, G. (2008). Quality Australian journals in the humanities and social sciences. AARL: Australian Academic and Research Libraries, 39(2), 79–91.Google Scholar
- Haddow, G., & Genoni, P. (2009). Australian education journals: Quantitative and qualitative indicators. AARL: Australian Academic & Research Libraries, 40(2), 88–104.Google Scholar
- Hicks, D. (2004). The four literatures of social science. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research: The use of publication and patent statistics in studies of S&T systems (pp. 473–496). Dordrecht: Kluwer Academic Publishers.Google Scholar
- Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102, 16569–16572. Retrieved June 16, 2007, from http://www.pnas.org/content/102/46/16569.full.
- HM Treasury. (2006). Government Meeting Science Goals. Retrieved December 28, 2008, from http://www.hm-treasury.gov.uk/press_53_06.htm.
- Holmes, A., & Oppenheim, C. (2001). Use of citation analysis to predict the outcome of the 2001 Research Assessment Exercise for Unit of Assessment (UoA) 61: Library and Information Management. Information Research, 6(2).Google Scholar
- Jacso, P. (2005). As we may search—comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Science, 89(9), 1537–1547.Google Scholar
- Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.Google Scholar
- Oppenheim, C., & Summers, M. A. C. (2008). Citation counts and the Research Assessment Exercise, part IV: Unit of assessment 67 (music). Information Research, 13(2), paper 342.Google Scholar
- Research Evaluation & Policy Project. (2005). Quantitative Indicators for Research Assessment: A Literature Review Canberra: Research Evaluation and Policy Project, Australian National University. Retrieved March 17, 2007.Google Scholar
- Royle, P. (1994). A citation analysis of Australian science and social science journals. Australian Academic and Research Libraries, 25(3), 162–171.Google Scholar
- Royle, P., & OVER, R. (1994). The use of bibliometric indicators to measure the research productivity of Australian academics. AARL: Australian Academic & Research Libraries, 25(2), 77–88.Google Scholar
- Smith, K., & Middleton, M. (2009). Australian library and information studies (LIS) researchers ranking of LIS journals. AARL: Australian Academic & Research Libraries, 40(4), 1–20.Google Scholar
- Smyth, R. (1999). A citation analysis of Australian economic journals. AARL: Australian Academic & Research Libraries, 30(2), 119–133.Google Scholar