Skip to main content
SpringerLink
Log in

How to measure the performance of a Collaborative Research Center

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

New Public Management helps universities and research institutions to perform in a highly competitive research environment. Evaluating publicly financed research improves transparency, helps in reflection and self-assessment, and provides information for strategic decision making. In this paper we provide empirical evidence using data from a Collaborative Research Center (CRC) on financial inputs and research output from 2005 to 2016. After selecting performance indicators suitable for a CRC, we describe main properties of the data using visualization techniques. To study the relationship between the dimensions of research performance, we use a time fixed effects panel data model and fixed effects Poisson model. With the help of year dummy variables, we show how the pattern of research productivity changes over time after controlling for staff and travel costs. The joint depiction of the time fixed effects and the research project’s life cycle allows a better understanding of the development of the number of discussion papers over time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Abramo, G., D’Angelo, C. A., & Di Costa, F. (2009). Research collaboration and productivity: Is there correlation? Higher Education, 57(2), 155–171.

    Article  Google Scholar 

  • Abramo, G., & D’Angelo, C. A. (2011). National-scale research performance assessment at the individual level. Scientometrics, 86, 347–364.

    Article  Google Scholar 

  • Barra, C., & Zotti, R. (2016). Measuring efficiency in higher education: An empirical study using a bootstrapped data envelopment analysis. International Advances in Economic Research, 22, 11–33.

    Article  Google Scholar 

  • Bolli, T., & Somogyi, F. (2011). Do competitively acquired funds induce universities to increase productivity? Research Policy, 40(1), 136–147.

    Article  Google Scholar 

  • Bornmann, L. (2013). How to analyze percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes, and top-cited papers. Journal of the American Society for Information Science and Technology, 64(3), 587–595.

    Article  Google Scholar 

  • Cameron, A. C., & Miller, D. L. (2015). A practitioner’s guide to cluster–robust inference. Journal of Human Resources, 50(2), 317–373.

    Article  Google Scholar 

  • Carayol, N., & Matt, M. (2004). Does research organization influence academic production? Laboratory level evidence from a large European university. Research Policy, 33(8), 1081–1102.

    Article  Google Scholar 

  • CRC 649. (2016). http://sfb649.wiwi.hu-berlin.de/about/index.php. Accessed 11 January 2018.

  • CRC Project on GitHub. http://www.github.com/QuantLet/CRC. Accessed 11 January 2018.

  • Deutsche Forschungsgemeinschaft (DFG, eng. - German Research Foundation). (2017). Chancengleichheits-Monitoring 2017. Antragstellung und -erfolg von Wissenschaftlerinnen bei der DFG. http://www.dfg.de/download/pdf/foerderung/grundlagen_dfg_foerderung/chancengleichheit/chancengleichheits_monitoring_2017.pdf. Accessed 01 August 2018.

  • Deutsche Forschungsgemeinschaft (DFG, eng. - German Research Foundation). (2018). http://www.dfg.de/en//research_funding/programmes/coordinated_programmes/collaborative_research_centres/index.html. Accessed 11 January 2018.

  • Diem, A., & Wolter, S. C. (2013). The use of bibliometrics to measure research performance in educational sciences. Research in Higher Education, 54, 86–114.

    Article  Google Scholar 

  • Donner, P., & Aman, V. (2015). Quantilbasierte Indikatoren für impact und Publikationsstartegie. Ergebnisse für Deutschland in allen Fachdisziplinen in den Jahren 2000 bis 2011, (p. 8). IFQ: Studien zum deutschen Innovationssystem.

  • Grözinger, G. & Leusing, B. (2006). Wissenschaftsindikatoren an Hochschulen. Europa-Universität Flensburg, International Institute of Management, Discussion Papers. 012, https://EconPapers.repec.org/RePEc:fln:wpaper:012.

  • Hausman, J., Hall, B. H., & Griliches, Z. (1984). Econometric models for count data with an application to the patents-R & D relationship. Econometrica, 52(4), 909–938.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  Google Scholar 

  • Hood, C. (1991). A public management for all seasons? Public Administration, 69(1), 3–19.

    Article  Google Scholar 

  • Hornbostel, S. (1991). Drittmitteleinverbung. Ein Indikator für universitäre Forschungsleistungen. Beiträge zu Hochschulforschung, 1, 57–84.

    Google Scholar 

  • Jansen, D., Wald, A., Franke, K. et al. (2007). Drittmittel als Performanzindikator der Wissenschaftlichen Forschung. Koelner Z. Soziol. u. Soz. Psychol., 59(1), 125–149.

    Article  Google Scholar 

  • JEL (Journal of Economic Literature) Classification System. (2018). https://www.aeaweb.org/econlit/jelCodes.php?view=jel. Accessed 11 January 2018.

  • Lee, G. J. (2010). Assessing publication performance of research units: Extensions through operational research and economic techniques. Scientometrics, 84(3), 717–734.

    Article  Google Scholar 

  • Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 265–277.

    Article  Google Scholar 

  • Moed, H. F., de Moya-Anegón, F., López-Illescas, C., & Visser, M. (2011). Is concentration of university research associated with better research performance? Journal of Informetrics, 5, 649–658.

    Article  Google Scholar 

  • Pastor, J. M., & Serrano, L. (2016). The determinants of the research output of universities: Specialization, quality and inefficiencies. Scientometrics, 1029(2), 1255–1281.

    Article  Google Scholar 

  • Pastor, J. M., Serrano, L., & Zaera, I. (2015). The research output of European higher education institutions. Scientometrics, 102(3), 1867–1893.

    Article  Google Scholar 

  • Schmoch, U., & Schubert, T. (2009). Sustainability of incentives for excellent research—The German case. Scientometrics, 81(1), 195–218.

    Article  Google Scholar 

  • Schröder, S., Welter, F., Leisten, I., Richert, A., & Jeschke, S. (2014). Research performance and evaluation? Empirical results from collaborative research centers and clusters of excellence in Germany. Research Evaluation, 23(3), 221–232.

    Article  Google Scholar 

  • Stock, J. H., & Watson, M. W. (2003). Introduction to econometrics (1st ed.). London: Pearson.

    Google Scholar 

  • Van den Berghe, H., Houben, J. A., de Bruin, R. E., Moed, H. F., Kint, A., Luwel, M., et al. (1998). Bibliometric indicators of university research performance in Flanders. Journal of the American Society for Information Science, 49(1), 59–67.

    Article  Google Scholar 

  • Wissenschaftsrat. (2004). Empfehlungen zu Rankings im Wissenschaftssystem Teil 1: Forschung (pp. 6285–04). Drs: Hamburg.

  • Wissenschaftsrat. (2011). Empfehlungen zur Bewertung und Steuerung von Forschungsleistung, Drs 1656–11.

  • Wissenschaftsrat. (2012). Bericht der Steuerungsgruppe zur Pilotstudie zur Weiterentwicklung des Forschungsratings (pp. 2815–12). Drs: Köln.

  • Wooldridge, J. M. (1999). Distribution-free estimation of some nonlinear panel data models. Journal of Econometrics, 90, 77–97.

    Article  MathSciNet  Google Scholar 

  • Wooldridge, J. M. (2002). Econometric analysis of cross section and panel data (1st ed.). Cambridge: MIT Press Books.

    MATH  Google Scholar 

  • Wooldridge, J. M. (2016). Introductory econometrics: A modern approach (6th ed.). Boston: Cengage Learning.

    Google Scholar 

  • Zharova, A., Härdle, W.K. & Lessmann, S. (2017). Is scientific performance a function of funds? SFB 649 Discussion Paper, 2017(28).

  • Zheng, J., & Liu, N. (2015). Mapping of important international academic awards. Scientometrics, 104(3), 763–791.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Business and Economics, Humboldt-Universität zu Berlin, Berlin, Germany

    Alona Zharova & Wolfgang Karl Härdle

  2. domino e.V., Birkenwerder, Germany

    Janine Tellinger-Rice

  3. Singapore Management University, Singapore, Republic of Singapore

    Wolfgang Karl Härdle

Authors
  1. Alona Zharova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janine Tellinger-Rice
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wolfgang Karl Härdle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alona Zharova.

Additional information

Financial support from the German Research Foundation (DFG) via Collaborative Research Center 649 “Economic Risk” and International Research Training Group 1792 “High Dimensional Nonstationary Time Series”, Humboldt-Universität zu Berlin, is gratefully acknowledged. We are thankful for the assistance provided by Nicole Hermann und Dominik Prugger.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zharova, A., Tellinger-Rice, J. & Härdle, W.K. How to measure the performance of a Collaborative Research Center. Scientometrics 117, 1023–1040 (2018). https://doi.org/10.1007/s11192-018-2910-8

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-018-2910-8

Keywords

Mathematics Subject Classification

JEL

Search

Navigation