Economic Theory

, Volume 52, Issue 3, pp 863–884 | Cite as

The intellectual influence of economic journals: quality versus quantity

Research Article

Abstract

The evaluation of scientific output has a key role in the allocation of research funds and academic positions. Decisions are often based on quality indicators for academic journals, and over the years, a handful of scoring methods have been proposed for this purpose. Discussing the most prominent methods (de facto standards) we show that they do not distinguish quality from quantity at article level. The systematic bias we find is analytically tractable and implies that the methods are manipulable. We introduce modified methods that correct for this bias, and use them to provide rankings of economic journals. Our methodology is transparent; our results are replicable.

Keywords

Modified invariant method Invariance to article-splitting Influence of economic journals Impact factor LP method Invariant method 

JEL Classification

A1 C8 D72 Y1 

References

  1. Altman, A., Tennenholtz, M.: Ranking systems: the PageRank axioms. In: Proceedings of the 6th ACM Conference on Electronic Commerce (EC-05), pp. 1–8. ACM Press, New York (2005)Google Scholar
  2. Bergstrom, C.T.: Eigenfactor: measuring the value and prestige of scholarly journals. Coll. Res. Libr. News 68(5), 314–316 (2007)Google Scholar
  3. Bergstrom, C.T., West, J.D., Wiseman, M.A.: The Eigenfactor metrics. J. Neurosci. 28(45), 11433–11434 (2008)CrossRefGoogle Scholar
  4. Braun, T., Glänzel, W., Schubert, A.: A Hirsch-type index for journals. Scientometrics 69(1), 169–173 (2006)CrossRefGoogle Scholar
  5. Brin, S., Page, L.: The anatomy of a large-scale hypertextual web search engine. Comput. Netw. ISDN Syst. 30(1–2), 107–117 (1998)CrossRefGoogle Scholar
  6. Cambell, D.E., Kelly, J.S.: Gains from manipulating social choice rules. Econ. Theory 40(3), 349–371 (2009)CrossRefGoogle Scholar
  7. Campbell, D.E., Kelly, J.S.: Losses due to manipulation of social choice rules. Econ. Theory 45(3), 453–467 (2010)CrossRefGoogle Scholar
  8. Combes, P.P., Linnemer, L.: Where are the economists who publish? Publication concentration and rankings in Europe based on cumulative publications. J. Eur. Econ. Assoc. 1(6), 1250–1308 (2003)CrossRefGoogle Scholar
  9. Coupé, T.: Revealed performances: worldwide rankings of economists and economics departments, 1990–2000. J. Eur. Econ. Assoc. 1(6), 1309–1345 (2003)CrossRefGoogle Scholar
  10. De Groot, M.H.: Reaching a consensus. J. Am. Stat. Assoc. 69(345), 118–121 (1974)CrossRefGoogle Scholar
  11. Garfield, E.: Citation indexes to science: a new dimension in documentation through association of ideas. Science 122(3159), 108–111 (1955)CrossRefGoogle Scholar
  12. Golub, B., Jackson, M.O.: Naive learning in social networks and the wisdom of crowds. Am. Econ. J. Microecon. 2(1), 112–149 (2010)CrossRefGoogle Scholar
  13. González-Pereira, B., Guerrero-Bote, V.P., Moya-Anegón, F.: The SJR indicator: a new indicator of journals’ scientific prestige. Techical Report, vol. 4141 (2009). ArXiv.orgGoogle Scholar
  14. Hirsch, J.E.: An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. 102(46), 16569–16572 (2004)CrossRefGoogle Scholar
  15. Jemec, G.B.E.: Impact factor to assess academic output. Lancet 358(9290), 1373 (2001)CrossRefGoogle Scholar
  16. Kalaitzidakis, P., Mamuneas, T.P., Stengos, T.: Rankings of academic journals and institutions in economics. J. Eur. Econ. Assoc. 1(6), 1346–1366 (2003)CrossRefGoogle Scholar
  17. Kóczy, L.Á., Strobel, M.: The invariant method can be manipulated. Scientometrics 81(1), 291–293 (2008)CrossRefGoogle Scholar
  18. Kóczy, L.Á., Strobel, M.: The world cup of economics journals: a ranking by a tournament method. Iehas Discussion Papers, Institute of Economics, Hungarian Academy of Sciences (2010)Google Scholar
  19. Laband, D.N., Piette, M.J.: The relative impacts of economics journals: 1970–1990. J. Econ. Lit. 32, 640–666 (1994)Google Scholar
  20. Langville, A.N., Meyer, C.D.: Google’s PageRank and Beyond: The Science of Search Engine Rankings. Princeton University Press, Princeton (2006)Google Scholar
  21. Liebowitz, S.J., Palmer, J.C.: Assessing the relative impacts of economics journals. J. Econ. Lit. 22(1), 77–88 (1984)Google Scholar
  22. Marcus, M., Minc, M.: On two theorems of frobenius. Pac. J. Math. 60(2), 149–151 (1975)CrossRefGoogle Scholar
  23. Mirrlees, J.A., Neary, P.J., Tirole, J.: Evaluating economics research in europe: an introduction. J. Eur. Econ. Assoc. 1(6), 1239–1249 (2003)CrossRefGoogle Scholar
  24. Opátrný, T.: Playing the system to give low-impact journal more clout. Nature 455(7210), 167–167 (2008)CrossRefGoogle Scholar
  25. Palacio-Huerta, I., Volij, O.: The measurement of intellectual influence. Econometrica 72(3), 963–977 (2004)Google Scholar
  26. Pinski, G., Narin, F.: Citation influence for journal aggregates of scientific publications: theory, with application to the literature of physics. Inf. Process. Manag. 12(5), 297–312 (1976)CrossRefGoogle Scholar
  27. Roy, S., Saberi, A., Wan, Y. (2008) Majorization for the dominant eigenvector of a nonnegative matrix. In: American Control Conference, pp 1965–1966. Westin Seattle Hotel, SeattleGoogle Scholar
  28. Shoham, Y., Leyton-Brown, K. (eds.): Multiagent Systems. Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, Cambridge (2009)Google Scholar
  29. Smith, R.: Journal accused of manipulating impact factor. Br. Med. J. 314(7079), 461 (1997)CrossRefGoogle Scholar
  30. Sobel, J.: Economists’ models of learning. J. Econ. Theory 94(1), 241–261 (2000)CrossRefGoogle Scholar
  31. Zimmermann, C.: Academic Rankings with RePEc. Department of Economics Working Paper Series 2007–36, University of Connecticut, Storrs (2007)Google Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  1. 1.Centre for Economic and Regional Sciences, Hungarian Academy of Sciences Óbuda UniversityBudapestHungary
  2. 2.School of Economics and FinanceUniversity of St. Andrews Castlecliffe The ScoresUK

Personalised recommendations