, Volume 92, Issue 3, pp 767–780 | Cite as

Field normalized citation rates, field normalized journal impact and Norwegian weights for allocation of university research funds

  • Per AhlgrenEmail author
  • Cristian Colliander
  • Olle Persson


We compared three different bibliometric evaluation approaches: two citation-based approaches and one based on manual classification of publishing channels into quality levels. Publication data for two universities was used, and we worked with two levels of analysis: article and department. For the article level, we investigated the predictive power of field normalized citation rates and field normalized journal impact with respect to journal level. The results for the article level show that evaluation of journals based on citation impact correlate rather well with manual classification of journals into quality levels. However, the prediction from field normalized citation rates to journal level was only marginally better than random guessing. At the department level, we studied three different indicators in the context of research fund allocation within universities and the extent to which the three indicators produce different distributions of research funds. It turned out that the three distributions of relative indicator values were very similar, which in turn yields that the corresponding distributions of hypothetical research funds would be very similar.


Field normalized citation rates Journal impact Norwegian model Research fund allocation 


  1. Auranen, O., & Nieminen, M. (2010). University research funding and publication performance-An international comparison. Research Policy, 39(6), 822–834.CrossRefGoogle Scholar
  2. Braun, T., & Glänzel, W. (1990). United Germany–the new scientific superpower. Scientometrics, 19(5–6), 513–521.CrossRefGoogle Scholar
  3. Butler, L. (2004). What happens when funding is linked to publication counts? In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 389–405). Dordrecht: Kluwer.Google Scholar
  4. Debackere, K., & Glänzel, W. (2004). Using a bibliometric approach to support research policy making: The case of the Flemish BOF-key. Scientometrics, 59(2), 253–276.CrossRefGoogle Scholar
  5. Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874.MathSciNetCrossRefGoogle Scholar
  6. Geuna, A. (2001). The changing rationale for European university research funding: Are there negative unintended consequences? Journal of Economic Issues, 35(3), 607–632.Google Scholar
  7. Geuna, A., & Martin, B. R. (2003). University research evaluation and funding: An international comparison. Minerva, 41(4), 277–304.CrossRefGoogle Scholar
  8. Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). New York: Wiley.zbMATHCrossRefGoogle Scholar
  9. Lundberg, J. (2007). Lifting the crown-citation z-score. Journal of Informetrics, 1(2), 145–154.CrossRefGoogle Scholar
  10. Moed, H. F., De Bruin, R. E., & van Leeuwen, T. N. (1995). New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications. Scientometrics, 33(3), 381–422.CrossRefGoogle Scholar
  11. Opthof, T., & Leydesdorff, L. (2010). Caveats for the journal and field normalizations in the CWTS (“Leiden”) evaluations of research performance. Journal of Informetrics, 4(3), 423–430.CrossRefGoogle Scholar
  12. Sandström, U., & Sandström, E. (2009). The field factor: towards a metric for academic institutions. Research Evaluation, 18(3), 243–250.CrossRefGoogle Scholar
  13. Schneider, J. W. (2009). An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway. European Political Science, 8(3), 364–378.CrossRefGoogle Scholar
  14. Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45(1), 1–11.CrossRefGoogle Scholar
  15. Sivertsen, G. (2010). A performance indicator based on complete data for the scientific publication output at research institutions. ISSI Newsletter, 6(1), 22–28.Google Scholar
  16. van Raan, A. F. J. (1996). Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics, 36(3), 397–420.CrossRefGoogle Scholar
  17. Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62(1), 117–131.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2012

Authors and Affiliations

  • Per Ahlgren
    • 1
    • 2
    Email author
  • Cristian Colliander
    • 2
  • Olle Persson
    • 2
  1. 1.Department of e-ResourcesUniversity Library, Stockholm UniversityStockholmSweden
  2. 2.Department of SociologyInforsk, Umeå UniversityUmeåSweden

Personalised recommendations