Scientometrics

, Volume 109, Issue 3, pp 2217–2239 | Cite as

Assessing the effects of the German Excellence Initiative with bibliometric methods

  • Torger Möller
  • Marion Schmidt
  • Stefan Hornbostel
Article

Abstract

The German Excellence Initiative started in 2006 as a public funding program of crucial importance for German universities. Since this time, several studies on different aspects of the program have been conducted, but there have been no analyses using bibliometric methods to measure the direct effects of funding—which is apparently due to the fact that publications resulting from the funding program are not publicly and comprehensively documented. This paper uses the concept of highly cited publications to measure excellent research, and explores two methodological approaches in order to attribute highly cited publications to the Excellence Initiative. To this end, the paper focuses on publications produced by the clusters of excellence (CoEs). The clusters of excellence constitute only one of three funding lines, but receive 60 % of the total funding of the excellence program and form the core research units of the Excellence Initiative. The highly cited publications of the CoEs are identified via self-selected lists of publications in the CoE renewal proposals and via a funding acknowledgement analysis. The validity of both data sources is analyzed comparatively. Based on the objectives of the Excellence Initiative, its effects on the level of funded clusters, universities and the overall German research system are explored. The bibliometric analysis gives evidence that the funding program has succeeded in concentrating excellent research and fostering collaborations between universities and the non-university research sector, but has not caused massive changes to the overall German research system.

Keywords

Bibliometric evaluation Excellence Initiative Germany Research system Funding acknowledgement Highly cited 

References

  1. Aagaard, K., Bloch, C., & Schneider, J. W. (2015). Impacts of performance-based research funding systems: The case of the Norwegian Publication Indicator. Research Evaluation, 24(2), 106–117. doi:10.1093/reseval/rvv003.CrossRefGoogle Scholar
  2. Aksnes, D. W., & Rip, A. (2009). Researchers’ perceptions of citations. Research Policy, 38(6), 895–905.CrossRefGoogle Scholar
  3. Aksnes, D. W., Schneider, J. W., & Gunnarsson, M. (2012). Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods. Journal of Informetrics, 6(1), 36–43. doi:10.1016/j.joi.2011.08.002.CrossRefGoogle Scholar
  4. Beck, U. (1986). Risikogesellschaft Auf dem Weg in ein andere Moderne. Frankfurt a. M.: Suhrkamp.Google Scholar
  5. Belter, C. W. (2013). A bibliometric analysis of NOAA’s Office of Ocean Exploration and Research. Scientometrics, 95(2), 629–644. doi:10.1007/s11192-012-0836-0.CrossRefGoogle Scholar
  6. Bloch, R., Keller, A., Lottmann, A., & Würmann, C. (2008). Making Excellence: Grundlagen, Praxis und Konsequenzen der Exzellenzinitiative. Bielefeld: Bertelsmann.Google Scholar
  7. Bornmann, L. (2014). How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature. Research Evaluation, 23(2), 166–173. doi:10.1093/reseval/rvu002.CrossRefGoogle Scholar
  8. Bornmann, L. (2016). Is the promotion of research reflected in bibliometric data? A network analysis of highly cited papers on the Clusters of Excellence supported under the Excellence Initiative in Germany. Scientometrics, 107(3), 1041–1061. doi:10.1007/s11192-016-1925-2.CrossRefGoogle Scholar
  9. Bornmann, L., & Leydesdorff, L. (2013). The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000. Journal of Informetrics, 7(2), 286–291. doi:10.1016/j.joi.2012.12.003.CrossRefGoogle Scholar
  10. Bukow, S., & Möller, T. (2013). Die Rekrutierung wissenschaftlichen Spitzenpersonals in der Exzellenzinitiative. Berlin: iFQ.Google Scholar
  11. Destatis (2014). Finanzen und Steuern 2012 (Fachserie 14 Reihe 3.6). Wiesbaden: Statistisches Bundesamt.Google Scholar
  12. Deutsche Forschungsgemeinschaft (2015). Förderatlas 2015Kennzahlen zur öffentlich finanzierten Forschung in Deutschland. Bonn: Deutsche Forschungsgemeinschaft.Google Scholar
  13. Deutsche Forschungsgemeinschaft DFG (2011). General information on preparing a proposal and template for initial/renewal proposals for a Cluster of Excellence Second Programme Phase 20122017, ExIn304e. Retrieved from http://www.dfg.de/formulare/exin304e/exin304e_rtf.rtf (2014-2-24).
  14. Díaz-Faes, A. A., & Bordons, M. (2014). Acknowledgments in scientific publications: Presence in Spanish science and text patterns across disciplines: Acknowledgments in Scientific Publications. Journal of the Association for Information Science and Technology, 65(9), 1834–1849. doi:10.1002/asi.23081.CrossRefGoogle Scholar
  15. Engels, A., Ruschenberg, T., & Zuber, S. (2012). Chancengleichheit in der Spitzenforschung: Institutionelle Erneuerung der Forschung in der Exzellenzinitiative des Bundes und der Länder. In T. Heinze & G. Krücken (Eds.), Institutionelle Erneuerungsfähigkeit der Forschung (pp. 187–217). Wiesbaden: VS-Verlag.Google Scholar
  16. ExV, Exzellenzvereinbarung. (2005). Bund-Länder-Vereinbarung gemäß Artikel 91 b des Grundgesetzes (Forschungsförderung) über die Exzellenzinitiative des Bundes und der Länder zur Förderung von Wissenschaft und Forschung an deutschen Hochschulen. Retrieved from http://www.gwk-bonn.de/fileadmin/Papers/exzellenzvereinbarung.pdf.
  17. ExV II. (2009). Exzellenzvereinbarung II: Verwaltungsvereinbarung zwischen Bund und Ländern gemäß Artikel 91b Abs. 1 Nr. 2 des Grundgesetztes über die Fortsetzung der Exzellenzinititaive des Bundes und der Länder zur Förderung von Wissenschaft und Forschung an deutschen Hochschulen. Retrieved from www.gwk-bonn.de/fileadmin/Papers/Exzellenzvereinbarung-II-2009.pdf.
  18. Gemeinsame Wissenschaftskonferenz (2005). Pakt für Forschung und Innovation I. Retrieved from http://www.gwk-bonn.de/fileadmin/Papers/pakt_fuer_forschung_und_innovation.pdf.
  19. Gemeinsame Wissenschaftskonferenz. (2014). Pakt für Forschung und Innovation III. Retrieved from www.gwk-bonn.de/fileadmin/Papers/PFI-III_2016-2020.pdf.
  20. Hartmann, M. (2006). Die Exzellenzinitiative—ein Paradigmenwechsel in der deutschen Hochschulpolitik. Leviathan, 34(4), 447–465.CrossRefGoogle Scholar
  21. Helmholtz-Gemeinschaft. (2015). Website der Helmholtz-Gemeinschaft, Helmholtz-Zentren und Netzwerke. Retrieved from http://www.helmholtz.de/helmholtz_zentren_netzwerke.
  22. Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. doi:10.1038/520429a.CrossRefGoogle Scholar
  23. Hornbostel, S., & Möller, T. (2015). Die Exzellenzinitiative und das deutsche Wissenschaftssystem: eine bibliometrische Wirkungsanalyse. Berlin: Berlin-Brandenburgische Akademie der Wissenschaften.Google Scholar
  24. Hornbostel, S., Simon, D., & Heise, S. (2008). Exzellente Wissenschaft. Das Problem, der Diskurs, das Programm und die Folgen (Vol. 4). Bonn: iFQ.Google Scholar
  25. Internationale Kommission. (1999). Forschungsförderung in Deutschland. Bericht der internationalen Kommission zur Systemevaluation der Deutschen Forschungsgemeinschaft und der Max-Planck-Gesellschaft. Hannover: Volkswagen-Stiftung.Google Scholar
  26. Langfeldt, L., Benner, M., Sivertsen, G., Kristiansen, E. H., Aksnes, D. W., Borlaug, S. B., et al. (2015). Excellence and growth dynamics: A comparative study of the Matthew effect. Science and Public Policy, 42(5), 661–675. doi:10.1093/scipol/scu083.CrossRefGoogle Scholar
  27. Leibfried, S. (2010). Die Exzellenzinitiative Zwischenbilanz und Perspektiven. Frankfurt a. M.: Campus.Google Scholar
  28. Leydesdorff, L., Bornmann, L., Mutz, R., & Opthof, T. (2011). Turning the tables on citation analysis one more time: Principles for comparing sets of documents. Journal of the American Society for Information Science and Technology, 62(7), 1370–1381. doi:10.1002/asi.21534.CrossRefGoogle Scholar
  29. Markova, H. (2013). Exzellenz durch Wettbewerb und Autonomie? Deutungsmuster hochschulpolitischer Eliten am Beispiel der Exzellenzinitiative. Konstanz UVK Verlagsgesellschaft.Google Scholar
  30. Mittermaier, B. (2011). Publizieren Spitzen-Unis mehr? duz Magazin 09. Retrieved from http://www.duz.de/duz-magazin/2011/09/publizieren-spitzen-unis-mehr/27.
  31. Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.Google Scholar
  32. Moed, H. F. (2008). UK Research Assessment Exercises: Informed judgments on research quality or quantity? Scientometrics, 74(1), 153–161. doi:10.1007/s11192-008-0108-1.CrossRefGoogle Scholar
  33. Möller, T. (2016). Messung möglicher Auswirkungen der Exzellenzinitiative sowie des Pakts für Forschung und Innovation auf die geförderten Hochschulen und außeruniversitären Forschungseinrichtungen. Berlin: Expertenkommission Forschung und Innovation (EFI). Retrieved from http://www.e-fi.de/fileadmin/Innovationsstudien_2016/StuDIS_09_2016.pdf.
  34. Möller, T., Antony, P., Hinze, S., & Hornbostel, S. (2012). Exzellenz begutachtet: Die Befragung der Gutachter in der Exzellenzinitiative (Vol. 11). Berlin: iFQ.Google Scholar
  35. Morillo, F., Costas, R., & Bordons, M. (2015). How is credit given to networking centres in their publications? A case study of the Spanish CIBER research structures. Scientometrics, 103(3), 923–938. doi:10.1007/s11192-015-1564-z.CrossRefGoogle Scholar
  36. Münch, R. (2006). Wissenschaft im Schatten von Kartell, Monopol und Oligarchie. Die latenten Effekte der Exzellenzinitiative. Leviathan, 34(4), 466–486.Google Scholar
  37. Münch, R. (2007). Die akdemische Elite Zur sozialen Konstruktion wissenschaftlicher Exzellenz. Frankfurt a. M.: Suhrkamp.Google Scholar
  38. Pasternack, P. (2009). Jenseits der Exzellenzinitiative. Alternative Optionen für die ostdeutsche Hochschulentwicklung. Die Hochschule, 1, 142–154.Google Scholar
  39. Rigby, J. (2013). Looking for the impact of peer review: Does count of funding acknowledgements really predict research impact? Scientometrics, 94(1), 57–73. doi:10.1007/s11192-012-0779-5.CrossRefGoogle Scholar
  40. Rigby, J., & Julian, K. (2014). On the horns of a dilemma: does more funding for research lead to more research or a waste of resources that calls for optimization of researcher portfolios? An analysis using funding acknowledgement data. Scientometrics, 101(2), 1067–1075. doi:10.1007/s11192-014-1259-x.CrossRefGoogle Scholar
  41. Scherb, J. (2012). Lissabon-Strategie (Lissabon-Prozess). Handlexikon der Europäischen Union. Baden-Baden: Nomos.Google Scholar
  42. Simon, D. (1991, Dezember 9). Die Universität ist verrottet. Der Spiegel, S. 52–53.Google Scholar
  43. Sirtes, D. (2013). Funding acknowledgements for the German Research Foundaten (DFG). The dirty data of the Web of Science database and how to clean it up. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the International Conference on Scientometrics and Informetrics. Vienna: 14th International Society of Scientometrics and Informetrics Conference (ISSI).Google Scholar
  44. Sirtes, D., Riechert, M., Donner, P., Aman, V., & Möller, T. (2015). Funding Acknowledgements in der Web of Science-Datenbank (Studien zum deutschen Innovationssystem Nr. XX-2014). Berlin: Expertenkommission Forschung und Innovation (EFI).Google Scholar
  45. Sondermann, M., Simon, D., Scholz, A., & Hornbostel, S. (2008). Die Exzellenzinitiative. Beobachtungen aus der Implementierungsphase (Bd. 5). Bonn: iFQ.Google Scholar
  46. Teddlie, C., & Yu, F. (2008). Mixed methods sampling: A typology with examples. In V. L. Plano Clark & J. W. Creswell (Eds.), The Mixed Methods Reader. Los Angeles: Sage.Google Scholar
  47. Testa, J. (2009). The globalization of Web of Science 2005–2010. Retrieved from http://wokinfo.com/media/pdf/globalwosessay.pdf.
  48. Thomson Reuters. (2016). Funding Acknowledgements. Retrieved from http://wokinfo.com/products_tools/multidisciplinary/webofscience/fundingsearch/.
  49. Tijssen, R. J. W., Visser, M. S., & van Leeuwen, T. N. (2002). Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference? Scientometrics, 54(3), 381–397.CrossRefGoogle Scholar
  50. Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E. C. M., Tijssen, R. J. W., van Eck, N. J., et al. (2012). The Leiden ranking 2011/2012: Data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology, 63(12), 2419–2432. doi:10.1002/asi.22708.CrossRefGoogle Scholar
  51. Waltman, L., & Schreiber, M. (2013). On the calculation of percentile-based bibliometric indicators. Journal of the American Society for Information Science and Technology, 64(2), 372–379. doi:10.1002/asi.22775.CrossRefGoogle Scholar
  52. Waltman, L., & van Eck, N. J. (2015). Field-normalized citation impact indicators and the choice of an appropriate counting method. Journal of Informetrics, 9(4), 872–894. doi:10.1016/j.joi.2015.08.001.CrossRefGoogle Scholar
  53. Wang, J., & Shapira, P. (2015). Is there a relationship between research sponsorship and publication impact? An analysis of funding acknowledgments in nanotechnology papers. PLoS One, 10(2), e0117727. doi:10.1371/journal.pone.0117727.CrossRefGoogle Scholar
  54. Winterhager, M., Schwechheimer, H., & Rimmert, C. (2014). Institutionenkodierung als Grundlage für bibliometrische Indikatoren. Bibliometrie—Praxis und Forschung, 3(14), 1–22.Google Scholar
  55. Yegros-Yegros, A., & Costas, R. (2013). Analysis of the web of science funding acknowledgement infomation for the design of indicators on external funding attraction. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the International Conference on Scientometrics and Informetrics (S. 84–95). Vienna: 14th International Society of Scientometrics and Informetrics Conference (ISSI).Google Scholar
  56. Zuber, S., & Hüther, O. (2013). Interdisziplinarität in der Exzellenzinitiative—auch eine Frage des Geschlechts? Beiträge zur Hochschulforschung, 4, 54–81.Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2016

Authors and Affiliations

  • Torger Möller
    • 1
  • Marion Schmidt
    • 1
  • Stefan Hornbostel
    • 1
  1. 1.DZHW - German Centre for Higher Education Research and Science StudiesBerlinGermany

Personalised recommendations