Scientometrics

, Volume 102, Issue 1, pp 727–749 | Cite as

Proof over promise: towards a more inclusive ranking of Dutch academics in Economics & Business

Article

Abstract

The Dutch Economics top-40, based on publications in ISI listed journals, is—to the best of our knowledge—the oldest ranking of individual academics in Economics and is well accepted in the Dutch academic community. However, this ranking is based on publication volume, rather than on the actual impact of the publications in question. This paper therefore uses two relatively new metrics, the citations per author per year (CAY) metric and the individual annual h-index (hIa) to provide two alternative, citation-based, rankings of Dutch academics in Economics & Business. As a data source, we use Google Scholar instead of ISI to provide a more comprehensive measure of impact, including citations to and from publications in non-ISI listed journals, books, working and conference papers. The resulting rankings are shown to be substantially different from the original ranking based on publications. Just like other research metrics, the CAY or hIa-index should never be used as the sole criterion to evaluate academics. However, we do argue that the hIa-index and the related CAY metric provide an important additional perspective over and above a ranking based on publications in high impact journals alone. Citation-based rankings are also shown to inject a higher level of diversity in terms of age, gender, discipline and academic affiliation and thus appear to be more inclusive of a wider range of scholarship.

Keywords

Rankings Citations ISI Google Scholar Economics 

References

  1. Abbring, J. H., Bronnenberg, B. J., Gautier, P. A., & van Ours, J. C. (2014). Dutch Economists top 40. De Economist, 162, 107–114.Google Scholar
  2. Adler, N., & Harzing, A. W. (2009). When Knowledge Wins: Transcending the sense and nonsense of academic rankings. The Academy of Management Learning & Education, 8(1), 72–95.CrossRefGoogle Scholar
  3. Börner, K., Dall’Asta, L., Ke, W., & Vespignani, A. (2005). Studying the emerging global brain: Analyzing and visualizing the impact of co-authorship teams. Complexity, 10(4), 57–67.CrossRefGoogle Scholar
  4. Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. D. (2011). A multilevel meta-analysis of studies reporting correlations between the h-index and 37 different h-index variants. Journal of Informetrics, 5(3), 346–359.CrossRefGoogle Scholar
  5. Franses, P. H. (2014). Trends in three decades of rankings of Dutch Economists. Scientometrics, 98(2), 1257–1268.CrossRefGoogle Scholar
  6. García, J. A., Rodriguez-Sánchez, R., & Fdez-Valdivia, J. (2012). A comparison of top economics departments in the US and EU on the basis of the multidimensional prestige of influential articles in 2010. Scientometrics, 93(3), 681–698.CrossRefGoogle Scholar
  7. Glänzel, W., & Thijs, B. (2004). Does co-authorship inflate the share of self-citations? Scientometrics, 61(3), 395–404.CrossRefGoogle Scholar
  8. Harzing, A. W. (2005). Australian research output in Economics & Business: High volume, low impact? Australian Journal of Management, 30(2), 183–200.CrossRefGoogle Scholar
  9. Harzing, A. W. (2007) Publish or Perish, Retrieved February 3, 2014, from http://www.harzing.com/pop.htm
  10. Harzing, A. W. (2013). A preliminary test of Google Scholar as a source for citation data: A longitudinal study of Nobel Prize winners. Scientometrics, 93(3), 1057–1075.CrossRefGoogle Scholar
  11. Harzing, A. W., Alakangas, S., & Adams, D. (2014). hIa: An individual annual h-index to accommodate disciplinary and career length differences. Scientometrics, 99(3), 811–821.CrossRefGoogle Scholar
  12. Harzing, A. W., & van der Wal, R. (2009). A Google Scholar h-index for journals: An alternative metric to measure journal impact in Economics & Business? Journal of the American Society for Information Science and Technology, 60(1), 41–46.CrossRefGoogle Scholar
  13. Jacso, P. (2010). Metadata mega mess in Google Scholar. Online Information Review, 34(1), 175–191.CrossRefGoogle Scholar
  14. Jin, J. C., & Choi, E. K. (2014). Citations of Most Often Cited Economists: Do Scholarly Books Matter More than Quality Journals? Pacific Economic Review, 19(1), 8–24.CrossRefMathSciNetGoogle Scholar
  15. Judge, T. A., Cable, D. M., Colbert, A. E., & Rynes, S. L. (2007). What causes a management article to be cited—Article, author, or journal? Academy of Management Journal, 50(3), 491–506.CrossRefGoogle Scholar
  16. Kalaitzidakis, P., Mamuneas, T. P., & Stengos, T. (2003). Rankings of academic journals and institutions in economics. Journal of the European Economic Association, 1(6), 1346–1366.CrossRefGoogle Scholar
  17. Katz, J. S., & Martin, B. R. (1997). What is research collaboration? Research policy, 26(1), 1–18.CrossRefGoogle Scholar
  18. Kodrzycki, Y. K., & Yu, P. (2006). New approaches to ranking economics journals. Contributions in Economic Analysis & Policy, 5(1), 1–40.Google Scholar
  19. London School of Economics and Political Science. (2011). Impact of the social sciences: Maximizing the impact of academic research. Retrieved February 3, 2014, from http://blogs.lse.ac.uk/impactofsocialsciences/.
  20. Nederhof, A. J. (2008). Policy impact of bibliometric rankings of research performance of departments and individuals in economics. Scientometrics, 74(1), 163–174.CrossRefGoogle Scholar
  21. Prathap, G. (2010). The 100 most prolific economists using the p-index. Scientometrics, 84(1), 167–172.CrossRefGoogle Scholar
  22. Scott, L. C., & Mitias, P. M. (1996). Trends in rankings of economics departments in the US: An update. Economic inquiry, 34(2), 378–400.CrossRefGoogle Scholar
  23. Seglen, P. O. (1992). The skewness of science. Journal of the American Society for Information Science, 43(9), 628–638.CrossRefGoogle Scholar
  24. Singh, G., Haddad, K. M., & Chow, C. W. (2007). Are articles in “top” management journals necessarily of higher quality? Journal of Management Inquiry, 16(4), 319–331.CrossRefGoogle Scholar
  25. Starbuck, W. H. (2005). How much better are the most-prestigious journals? The statistics of academic publication. Organization Science, 16(2), 180–200.CrossRefGoogle Scholar
  26. Tol, R. S. (2009). The h-index and its alternatives: An application to the 100 most prolific economists. Scientometrics, 80(2), 317–324.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2014

Authors and Affiliations

  1. 1.ESCP Europe, 527London UK
  2. 2.Erasmus University RotterdamRotterdamThe Netherlands

Personalised recommendations