Who gets Horizon 2020 research grants? Propensity to apply and probability to succeed in a two-step analysis

Abstract

This paper presents a timely analysis of participation in the 8th European Framework Programme for Research and Innovation (EU FP) Horizon 2020. Our dataset comprises the entire population of research organizations in Norway, enabling us to distinguish between non-applicants, non-successful applicants, and successful participants. We find it important to distinguish two stages of the participation process: the self-selection stage in which organizations decide whether they wish to apply for EU funding, and the second stage in which the European Commission selects the best applications for funding. Our econometric results indicate that the propensity to apply is enhanced by prior participation in EU FPs and the existence of complementary national funding schemes; further, that the probability of succeeding is strengthened by prior participation as well as the scientific reputation of the applicant organization.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. 1.

    Hospitals without university function are formally a part of the PRO sector, whereas those with university instruction functions fall within the HEI sector (Research Council of Norway 2015).

  2. 2.

    The data retrieved were entered into ECORDA in July 2015, but it covers project applications only until February 2015. The EU Commission has decided that there must be a time-lag of approx. 5 months from the application deadline until data are published.

  3. 3.

    In additional exercises not reported in this paper, we used two corresponding variables indicating whether an organization has served as coordinator of an FP6 or FP7 project, in order to take into account the role of network centrality in previous participation. These results confirm the positive role of network centrality for participation in EU research. We have also sought to use “coordinated projects” (rather than mere participation) as the dependent variable in the regressions. However, our sample includes only a limited number of coordinated projects, so these additional exercises should be interpreted with caution.

  4. 4.

    For this variable, we have positive citation data for only 89 institutions, because of the set threshold of minimum 500 publications for inclusion in this database. We set “reputation” for missing organizations at 0, as done by Lepori et al. (2015).

  5. 5.

    Lepori et al. (2015) used external funding as an indicator for third-party funds, including external funding from the EU, Research Councils and private companies. We have chosen to treat these variables separately and divide them by total funding to get a measure of the amount of funding by total revenue.

  6. 6.

    Some of our explanatory variables have skewed distribution, and in particular those measuring R&D capabilities, financial conditions, and size. We log transformed these variables before entering the regression model.

  7. 7.

    In previous research, due the absence of data enabling to distinguish between non-applicants and unsuccessful applicants, both of these groups were combined together and typically given a value of 0 in the regressions, whereas successful applicants were given a positive integer value (1 if the dependent variable was defined as a dummy, and values equal or greater than 1 if the variable was measured as a count). However, combining together non-applicants and unsuccessful applicants in the same regression tends to underestimate the estimated slopes of interest (the larger the mean of a given explanatory variable for the group of non-applicants, the larger the bias). We overcome this problem by estimating Eq. 2 only on the sub-sample of applicants. Table 3 reports the median of our explanatory variables for four groups of observations: (1) non-applicants, (2) unsuccessful applicants, (3) applicants that have been awarded one project, (4) applicants that have been awarded more than one project. The table shows that, for the group of non-applicants, three variables (publication points, national funding, and size) have positive median. Hence, our econometric approach yields more precise estimates for these three variables in particular. Table 4 reports a comparison of means of successful applicants (at least one project funded) and unsuccessful applicants (no projects funded) for all explanatory variables, along with t-tests of mean differences. The t-tests illustrate clearly that successful applicants have on average higher values in relation to the variables measuring previous participation, reputation and national funding. Hence, justifying the sample and the hypotheses.

  8. 8.

    We have also estimated our model by following different econometric approaches. First, we have reproduced the approach used by Geuna (1998) and Lepori et al. (2015) on our dataset, i.e. estimating Eq. 2 by means of a logit (for a dummy dependent variable: funded vs. not-funded), and by means of a truncated regression (for a count-dependent variable that excludes organizations with 0 funded projects). The problem with these approaches is that, due to the relatively small size of our sample, the variability is limited and it is difficult to obtain precise results. Second, we have estimated Eqs. 1 and 2 jointly by means of a Heckman sample selection model (Heckman 1979). However, this approach requires the dependent variable in the second step to be dichotomous. Given the fairly small sample in our study, using a dummy-dependent variable greatly diminishes the data variability. Sample selection models that account for count outcomes are still under development and are not yet incorporated in standard econometric software. We tried using the sample selection model wrapper (SSM) based on generalized linear latent and mixed models (GLLAMM), developed by Miranda and Rabe-Hesketh (2006), but the procedure failed to produce the required output.

  9. 9.

    A possible reason why the size variable is not significant in the estimations in step 2 may be due to multicollinearity. The correlation coefficients in Table 2 indicate SIZE to be positively correlated with the FP6 and FP7 participation variables (although the VIF statistics for these variables are below critical threshold levels for multicollinearity). If we exclude the latter from the regressions, the size indicator becomes statistically significant. To test further the effect of size on H2020 participation, we have also carried out another exercise based on a matching approach. The results of matching results, reported at the end of this section, show indeed that size has a significant correlation to H2020 participation.

  10. 10.

    A cross-classification table between participation to FP7 and H2020 shows that persistence is important in our sample, as expected, although it does not completely predict participation patterns. In fact, 24 % of organizations that got at least one project in FP7 also got funding from H2020, whereas 25 % were not able to succeed in H2020. On the other hand, 48 % of the organizations that did not have any funded project in FP7 did not have any H2020 project either; and only 3 % of organizations managed to get a H2020 project although they did not have any FP7 project.

References

  1. Annerberg, R., Begg, I., Acherson, H., Borrás, S., Hallen, A., Maimets, T., et al. (2010). Interim evaluation of the seventh framework programme: Report of the expert group. Brussels: European Commission.

  2. Breschi, S., Cassi, L., Malerba, F., & Vonortas, N. S. (2009). Networked research: European policy intervention in ICTs. Technology Analysis & Strategic Management, 21(7), 833–857.

    Article  Google Scholar 

  3. Breschi, S., & Cusmano, L. (2004). Unveiling the texture of a European Research Area: Emergence of oligarchic networks under EU Framework Programmes. International Journal of Technology Management, 27(8), 747–772.

    Article  Google Scholar 

  4. Caliendo, M., & Kopeinig, S. (2008). Some practical guidance for the implementation of propensity score matching. Journal of economic surveys, 22(1), 31–72. doi:10.1111/j.1467-6419.2007.00527.x.

    Article  Google Scholar 

  5. Cerulli, G. (2010). Modelling and measuring the effect of public subsidies on business R&D: A critical review of the econometric literature. Economic Record, 86(274), 421–449.

    Article  Google Scholar 

  6. Defazio, D., Lockett, A., & Wright, M. (2009). Funding incentives, collaborative dynamics and scientific productivity: Evidence from the EU framework program. Research Policy, 38(2), 293–305.

    Article  Google Scholar 

  7. Di Cagno, D., Fabrizi, A., & Meliciani, V. (2014). The impact of participation in European joint research projects on knowledge creation and economic growth. The Journal of Technology Transfer, 39(6), 836–858.

    Article  Google Scholar 

  8. Dinges, M., & Lepori, B. (2006). Public project funding of research activities: National differences and implications for the creation of a European Research Council. Paper presented at the Platform fteval Conference—New Frontiers in Evaluation, 24–25 April.

  9. Eurostat. (2008). Statistical regions for the EFTA countries and candidate countries. Brussels: European Commission.

  10. Geuna, A. (1996). The participation of higher education institutions in European Union Framework Programmes. Science and Public Policy, 23(5), 287–296.

    Google Scholar 

  11. Geuna, A. (1998). Determinants of university participation in EU-funded R&D cooperative projects. Research Policy, 26(6), 677–687.

    Article  Google Scholar 

  12. Godø, H., Langfeldt, L., & Kaloudis, A. (2009). In need of a better framework for success: An evaluation of the Norwegian participation in the EU 6th Framework Programme (2003–2006) and the first part of the EU 7th Framework Programme (2007–2008). Oslo: NIFU STEP.

    Book  Google Scholar 

  13. Gornitzka, Å., & Langfeldt, L. (2008). The internationalisation of national knowledge policies. In A. Gornitzka & L. Langfeldt (Eds.), Borderless knowledge. Understanding the “New” Internationalisation of Research and Higher Education in Norway (Vol. 22, pp. 141–169). Amsterdam: Springer.

    Chapter  Google Scholar 

  14. Greene, W. H. (1994). Accounting for excess zeros and sample selection in Poisson and negative binomial regression models. New York University, Stern School of Business, Working paper 94-10, 3.

  15. Hakala, J., Kutinlahti, P., & Kaukonen, E. (2002). Becoming international, becoming European: EU research collaboration at Finnish universities. Innovation: The European Journal of Social Science Research, 15(4), 357–379.

    Google Scholar 

  16. Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica: Journal of the Econometric Society, 47(1), 153–161.

    MathSciNet  MATH  Article  Google Scholar 

  17. Henriques, L., Schoen, A., & Pontikakis, D. (2009). Europe’s top research universities in FP6: Scope and drivers of participation. JRC Technical Notes (Vol. 53681): European Commission.

  18. Hoekman, J., Scherngell, T., Frenken, K., & Tijssen, R. (2012). Acquisition of European research funds and its effect on international scientific collaboration. Journal of Economic Geography, 1–30. doi:10.1093/jeg/lbs011.

  19. Langfeldt, L., Godø, H., Gornitzka, Å., & Kaloudis, A. (2012). Integration modes in EU research: Centrifugality versus coordination of national research policies. Science and Public Policy, 39(1), 88–98.

    Article  Google Scholar 

  20. Laredo, P. (1998). The networks promoted by the framework programme and the questions they raise about its formulation and implementation. Research Policy, 27(6), 589–598.

    Article  Google Scholar 

  21. Lepori, B., Veglio, V., Heller-Schuh, B., Scherngell, T., & Barber, M. (2015). Participations to European Framework Programs of higher education institutions and their association with organizational characteristics. Scientometrics, 105(3), 2149–2178. doi:10.1007/s11192-015-1768-2.

    Article  Google Scholar 

  22. Luukkonen, T. (1998). The difficulties in assessing the impact of EU framework programmes. Research Policy, 27(6), 599–610.

    Article  Google Scholar 

  23. Luukkonen, T. (2000). Additionality of EU framework programmes. Research Policy, 29(6), 711–724.

    Article  Google Scholar 

  24. Luukkonen, T., & Nedeva, M. (2010). Towards understanding integration in research and research policy. Research Policy, 39(5), 674–686.

    Article  Google Scholar 

  25. Makkonen, T., & Mitze, T. (2016). Scientific collaboration between ‘old’ and ‘new’ member states: Did joining the European Union make a difference? Scientometrics, 106(3), 1–23.

    Article  Google Scholar 

  26. Matt, M., Robin, S., & Wolff, S. (2012). The influence of public programs on inter-firm R&D collaboration strategies: Project-level evidence from EU FP5 and FP6. The Journal of Technology Transfer, 37(6), 885–916.

    Article  Google Scholar 

  27. Miranda, A., & Rabe-Hesketh, S. (2006). Maximum likelihood estimation of endogenous switching and sample selection models for binary, ordinal, and count variables. Stata Journal, 6(3), 285–308.

    Article  Google Scholar 

  28. Nedeva, M. (2013). Between the global and the national: Organising European science. Research Policy, 42(1), 220–230.

    Article  Google Scholar 

  29. Nokkala, T., Heller-Schuh, B., & Paier, M. (2011). Ranking lists and European Framework Programmes: Does university status matter for performance in Framework Programmes? In P. T. D. Dill (Ed.), Public vices, private virtues? Assessing the effects of marketization in higher education (pp. 111–139). Rotterdam: Sense Publishers.

    Chapter  Google Scholar 

  30. Norwegian Ministry of Education and Research. (2014). Strategy for research and innovation cooperation with the EUHorizon 2020 and ERA. Oslo: Norwegian Ministry of Education and Research.

  31. Norwegian Ministry of Education and Research. (2015). Forskningsbarometeret 2015. Oslo: Norwegian Ministry of Education and Research.

  32. Okubo, Y., & Zitt, M. (2004). Searching for research integration across Europe: A closer look at international and inter-regional collaboration in France. Science and Public Policy, 31(3), 213–226.

    Article  Google Scholar 

  33. Ortega, J. L., & Aguillo, I. F. (2010). Describing national science and technology systems through a multivariate approach: Country participation in the 6th Framework Programmes. Scientometrics, 84(2), 321–330.

    Article  Google Scholar 

  34. Paier, M., & Scherngell, T. (2011). Determinants of collaboration in European R&D networks: Empirical evidence from a discrete choice model. Industry and Innovation, 18(1), 89–104.

    Article  Google Scholar 

  35. Perumean-Chaney, S. E., Morgan, C., McDowall, D., & Aban, I. (2013). Zero-inflated and overdispersed: What’s one to do? Journal of Statistical Computation and Simulation, 83(9), 1671–1683. doi:10.1080/00949655.2012.668550.

    MathSciNet  MATH  Article  Google Scholar 

  36. Polt, W., & Streicher, G. (2005). Trying to capture additionality in Framework Programme 5—Main findings. Science and Public Policy, 32(5), 367–373.

    Article  Google Scholar 

  37. Primeri, E., & Reale, E. (2012). How Europe shapes academic research: Insights from participation in European union framework programmes. European Journal of Education, 47(1), 104–121.

    Article  Google Scholar 

  38. Protogerou, A., Caloghirou, Y., & Siokas, E. (2010). Policy-driven collaborative research networks in Europe. Economics of Innovation and New Technology, 19(4), 349–372.

    Article  Google Scholar 

  39. Research Council of Norway. (2015). Det norske forsknings- og innovasjonssystemet - statistikk og indikatorer. Oslo: Research Council of Norway.

    Google Scholar 

  40. Roediger-Schluga, T., & Barber, M. J. (2008). R&D collaboration networks in the European Framework Programmes: Data processing, network construction and selected results. International Journal of Foresight and Innovation Policy, 4(3–4), 321–347.

    Article  Google Scholar 

  41. Tijssen, R. J. (2008). Are we moving towards an integrated European Research Area? Some macro-level bibliometric perspectives. Collnet Journal of Scientometrics and Information Management, 2(1), 19–25.

    Article  Google Scholar 

  42. Ukrainski, K., Masso, J., & Kanep, H. (2014). Cooperation patterns in science within Europe: The standpoint of small countries. Scientometrics, 99(3), 845–863.

    Article  Google Scholar 

  43. Viner, N., Powell, P., & Green, R. (2004). Institutionalized biases in the award of research grants: A preliminary analysis revisiting the principle of accumulative advantage. Research Policy, 33(3), 443–454.

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank two anonymous referees of this journal for valuable comments and suggestions. The authors are solely responsible for any remaining error and omission. Work on this paper was supported by the Norwegian Research Council’s Public Sector Ph.D. Program (Grant Number 246964/H20).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Simen G. Enger.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Enger, S.G., Castellacci, F. Who gets Horizon 2020 research grants? Propensity to apply and probability to succeed in a two-step analysis. Scientometrics 109, 1611–1638 (2016). https://doi.org/10.1007/s11192-016-2145-5

Download citation

Keywords

  • Horizon 2020
  • EU Framework Programs
  • Research funding
  • Research policy
  • Higher education institutions
  • Public research organizations