, Volume 101, Issue 2, pp 1325–1344 | Cite as

Influence of a performance indicator on Danish research production and citation impact 2000–12

  • Peter IngwersenEmail author
  • Birger Larsen


This paper analyses the patterns of Danish research productivity, citation impact and (inter)national collaboration across document types 2000–2012, prior to and after the introduction of the Norwegian publication point-based performance indicator in 2008. Document types analysed are: research articles; conference proceedings papers excluding meeting abstracts; and review articles. The Danish Research & Innovation Agency’s basic statistics combined with Web of Science (WoS) are used for data collection and analyses. Findings demonstrate that the research article productivity increases steeply (37 %) after the start of the performance indicator and the citation impact progresses linearly over the entire period, regardless the introduction of the performance indicator. Academic staff progression is only 24 % during the same time period. The collaboration ratio between purely Danish and internationally cooperated research articles remains stable during the period, the number of collaborative countries increases while the ratio declines significantly for proceedings papers. The citation impact of internationally cooperated research articles increases since 2009 but drops for proceedings papers; also their productivity declines slightly from 2009 according to Research Agency statistics. Since 2006 the WoS indexing of proceedings papers is fast declining; as a consequence the ratio between Danish proceedings papers and research articles declines in WoS. According to Research Agency statistics a decline likewise takes place, starting from 2009. The positive growth in research articles mainly derives from the Science and Technology fields published in prestigious Level 2 journals; the development of articles published in less prestigious Level 1 journals derives from all fields. Three of the eight Danish universities have significantly altered their research publication profiles since 2009. The publication performance model is regarded as the significant accelerator of these processes in recent years.


Publication performance indicator Citation analyses Publication patterns Denmark Research articles Review articles Proceedings papers 



We wish to thank Karen Skytte Larsen from the Danish Agency of Research & Innovation for access to the Agency data.


  1. Aagaard, K., Bloch, C., Schneider, J., Henriksen, D., Kjeldager Ryan, T., & Lauridsen, P. S. (2014). Evaluering af den norske publiceringsindikator (Eng.: Evaluation of the Norwegian publication indicator). Aarhus: Dansk center for Forskningsanalyse, Aarhus Universitet.Google Scholar
  2. Aagaard, K., & Schneider, J. W. (2012). Den danske bibliometriske model; En dårlig kopi af den norske. Forskningspolitikk, 1, 28–29.Google Scholar
  3. Butler, L. (2003). Explaining Australia’s increased share of ISI publications—the effects of a funding formula based on publication counts. Research Policy, 32(1), 143–155.CrossRefGoogle Scholar
  4. Butler, L. (2004). What happens when funding is linked to publication counts? In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of Quantitative Science and Technology (pp. 340–389). London: Kluwer Academic Publishers.Google Scholar
  5. Butler, L., & Visser, M. S. (2006). Extending citation analysis to non-source items. Scientometrics, 66(10), 327–343.CrossRefGoogle Scholar
  6. Debackere, K., & Glänzel, W. (2004). Using a bibliometric approach to support research policy making: The case of the Flemish BOF-key. Scientometrics, 59, 253–276.CrossRefGoogle Scholar
  7. Engels, T. C. E., Ossenblok, T. L. B., & Spruyt, E. H. J. (2012). Changing publication patterns in the Social Sciences and Humanities, 2000–2009. Scientometrics, 93, 373–390.CrossRefGoogle Scholar
  8. Forskningsstyrelsen. (2013). The Danish Research & Innovation Agency. (in Danish) available at:
  9. Glänzel, W. (2002). Co-authorship patterns and trends in the sciences (1980–1998): A bibliometric study with implications for database indexing and search strategies. Library Trends, 50(3), 461–473.Google Scholar
  10. He, Y., & Guan, J. C. (2008). Contribution of Chinese publications in computer science: A case study on LNCS. Scientometrics, 75(28), 519–534.CrossRefGoogle Scholar
  11. Ingwersen, P., & Larsen, B. (2013). Influence of university mergers and the Norwegian performance indicator on overall Danish citation impact 2000-12. Proceedings of the International Society of Scientometrics and Informetrics, Wien, August 2013.Vol 1, 16 p. Available in:
  12. Ingwersen, P., Larsen, B., Garcia-Zorita, J.C., Serrano-López, E.A., & Sanz-Casado, E. (2014). Influence of Proceedings Papers on Citation Impact in Seven Sub-fields of Sustainable Energy Research 2005-2011. Scientometrics (under review).Google Scholar
  13. Luo, J. Y., Flynn, J. M., Solnick, R. E., Ecklund, E. H., & Matthews, K. R. W. (2011). International stem cell collaboration: How disparate policies between the United States and the United Kingdom impact research. Plos One, 6(3), 1–7.CrossRefGoogle Scholar
  14. Martins, W. S., Goncalves, M. A., Laender, A. H. F., & Ziviani, N. (2010). Assessing the quality of scientific conferences based on bibliographic citations. Scientometrics, 83(19), 133–155.CrossRefGoogle Scholar
  15. Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.Google Scholar
  16. Narin, F., Stevens, K., & Whitlow, E. S. (1991). Scientific cooperation in Europe and the citation of multinationally authored papers. Scientometrics, 21(3), 313–323.CrossRefGoogle Scholar
  17. Ossenblok, T. L. B., Engels, T. C. E., & Sivertsen, G. (2012). The representation of the social sciences and humanities in the Web of Science. A comparison of publication patterns and incentive structures in Flanders and Norway (2005–2009). Research Evaluation, 21(4), 280–290.CrossRefGoogle Scholar
  18. Schneider, J. W. (2009). An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway. European Political Science, 8(3), 364–378.CrossRefGoogle Scholar
  19. Sivertsen, G. (2010). A performance indicator based on complete data for the scientific publication output at research institutions. ISSI Newsletter, 6(19), 22–28.Google Scholar
  20. Sivertsen, G., & Schneider J. W. (2012). Evaluering av den bibliometriske forskningsindikator. Oslo: NIFU. 75 p. (Rapport 17/2012, in Norwegian).Google Scholar
  21. van Raan, A. F. J. (1999). Advanced bibliometric methods for evaluation of universities. Scientometrics, 45(3), 417–423.CrossRefGoogle Scholar
  22. van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.CrossRefGoogle Scholar
  23. Visited 22 Apr 2013.
  24. Wainer, J., de Oliveira, H. P., & Anido, R. (2011). Patterns of bibliographic references in the ACM published papers. Information Processing & Management, 47(13), 135–142.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2014

Authors and Affiliations

  1. 1.Royal School of Information and Library ScienceUniversity of CopenhagenCopenhagen SDenmark
  2. 2.Department of Communication and PsychologyAalborg University CopenhagenCopenhagen SVDenmark

Personalised recommendations