The Australian Educational Researcher

, Volume 39, Issue 2, pp 221–235 | Cite as

Benchmarking citation measures among the Australian education professoriate

  • Peter R. Albion


Individual researchers and the organisations for which they work are interested in comparative measures of research performance for a variety of purposes. Such comparisons are facilitated by quantifiable measures that are easily obtained and offer convenience and a sense of objectivity. One popular measure is the journal impact factor, based on citation rates, but it is a measure intended for journals rather than individuals. Moreover, educational research publications are not well represented in the databases most widely used for calculation of citation measures, leading to doubts about the usefulness of such measures in education. Newer measures and data sources offer alternatives that provide wider representation of education research. However, research has shown that citation rates vary according to discipline and that valid comparisons depend upon the availability of discipline-specific benchmarks. This study sought to provide such benchmarks for Australian educational researchers based on analysis of citation measures obtained for the Australian education professoriate.


Citation measures Bibliometrics Research impact Educational research Publication 


  1. Abramo, G., D’Angelo, C. A., & Viel, F. (2010). A robust benchmark for the h- and g-indexes. Journal of the American Society for Information Science and Technology, 61(6), 1275–1280.Google Scholar
  2. Australian Research Council. (2009). The Excellence in Research for Australia (ERA) Initiative. Accessed 24 March 2010.
  3. Bar-Ilan, J. (2008). Which h-index?—A comparison of WoS. Scopus and Google Scholar. Scientometrics, 74(2), 257–271.Google Scholar
  4. Bates, R. (2003). Phelan’s bibliometric analysis of the impact of Australian educational research. Australian Educational Researcher, 30(2), 57–64.CrossRefGoogle Scholar
  5. Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h-index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385.CrossRefGoogle Scholar
  6. Bornmann, L., & Daniel, H.-D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.CrossRefGoogle Scholar
  7. Bornmann, L., & Daniel, H.-D. (2009). The state of h-index research. EMBO Reports, 10(1), 2–6.CrossRefGoogle Scholar
  8. Bornmann, L., Mutz, R., Neuhaus, C., & Daniel, H. D. (2008). Citation counts for research evaluation: Standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics, 8(1), 93–102.CrossRefGoogle Scholar
  9. Campbell, P. (2008). Escape from the impact factor. Ethics in Science and Environmental Politics, 8, 5–7.CrossRefGoogle Scholar
  10. Clarke, R. (2009). A citation analysis of Australian information systems researchers: Towards a new ERA? Australasian Journal of Information Systems, 15(2), 23–44.Google Scholar
  11. Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.CrossRefGoogle Scholar
  12. Goodyear, R. K., Brewer, D. J., Gallagher, K. S., Tracey, T. J. G., Claiborn, C. D., Lichtenberg, J. W., et al. (2009). The intellectual foundations of education: Core journals and their impacts on scholarship and practice. Educational Researcher, 38(9), 700–706.CrossRefGoogle Scholar
  13. Harzing, A. W. (2009). Publish or Perish (Version 2.8.3644).Google Scholar
  14. Harzing, A. W. K., & van der Wal, R. (2008). Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, 8(1), 61–73.CrossRefGoogle Scholar
  15. Henzinger, M., Suñol, J., & Weber, I. (2010). The stability of the h-index. Scientometrics, 84(2), 465–479.CrossRefGoogle Scholar
  16. Herther, N. K. (2009). Research evaluation and citation analysis: Key issues and implications. The Electronic Library, 27(3), 361–375.CrossRefGoogle Scholar
  17. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National academy of Sciences of the United States of America, 102(46), 16569–16572.CrossRefGoogle Scholar
  18. Hodder, A. P. W., & Hodder, C. (2010). Research culture and New Zealand’s performance-based research fund: Some insights from bibliographic compilations of research outputs. Scientometrics, 84(3), 887–901.CrossRefGoogle Scholar
  19. Jarwal, S. D., Brion, A. M., & King, M. L. (2009). Measuring research quality using the journal impact factor, citations and ‘Ranked Journals’: Blunt instruments or inspired metrics? Journal of Higher Education Policy and Management, 31(4), 289–300.CrossRefGoogle Scholar
  20. Levine-Clark, M., & Gil, E. (2009). A comparative analysis of social sciences citation tools. Online Information Review, 33(5), 986–996.CrossRefGoogle Scholar
  21. Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105–2125.CrossRefGoogle Scholar
  22. Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.Google Scholar
  23. Moed, H. F. (2008). UK research assessment exercises: Informed judgments on research quality or quantity? Scientometrics, 74(1), 153–161.CrossRefGoogle Scholar
  24. Norris, M., & Oppenheim, C. (2010). Peer review and the h-index: Two studies. Journal of Informetrics, 4(3), 221–232.CrossRefGoogle Scholar
  25. Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.CrossRefGoogle Scholar
  26. Phelan, T. J., Anderson, D. S., & Bourke, P. (2000). Educational research in Australia: A bibliometric analysis. In DETYA (Ed.), The Impact of Educational Research (pp. 575–671). Canberra: Higher Education Division Department of Education, Training and Youth Affairs.Google Scholar
  27. Saad, G. (2010). Applying the h-index in exploring bibliometric properties of elite marketing scholars. Scientometrics, 83(2), 423–433.CrossRefGoogle Scholar
  28. Smith, A. (2008). Benchmarking google scholar with the New Zealand PBRF research assessment exercise. Scientometrics, 74(2), 309–316.CrossRefGoogle Scholar
  29. Vaughan, L., & Shaw, D. (2008). A new look at evidence of scholarly citation in citation indexes and from web sources. Scientometrics, 74(2), 317–330.CrossRefGoogle Scholar

Copyright information

© The Australian Association for Research in Education, Inc. 2012

Authors and Affiliations

  1. 1.University of Southern QueenslandToowoombaAustralia

Personalised recommendations