Skip to main content

Prioritization and R&D Support Mechanisms: Turkish Case

Abstract

This study aims to detect differences in the features of the proposed and supported projects for different priority technology areas (PTAs) of the TUBITAK Prioritized Areas R&D Grant Program together with the analysis on the output, input, and behavioral additionality of the supported projects. Non-existence of a previous study on the evaluation of the Prioritized Grant Program of TUBITAK contributes to the originality of this study in addition to being one of the limited examples about the studies on the efficient allocation of R&D incentives in the literature. During this study, firstly, descriptive statistics of program indicators is analyzed. Then, the relationship between output amount and the characteristics of the supported projects and their calls is estimated for different PTAs by the Ordinary Least Square (OLS) method. Moreover, interviews involving questions to measure output, behavioral and input additionalities are also conducted with a sample of supported project coordinators. Using both quantitative and qualitative methods together, not seen in the literature frequently, increases reliability and originality of the study. Consequently, it is detected that the amount of proposed and supported projects and average requested and given fund per project differ with PTAs and project characteristics. Additionally, the effects of these characteristics on output amount differentiate with PTAs. Moreover, supported projects and their outputs are inadequate to meet the specific targets of the grant program despite their significant project and input additionalities. Some policies, which could be helpful to increase the effectiveness and efficiency of the Program are recommended as a conclusion of this study.

Introduction

Governments use policy tools related to the R&D and innovation systems with the aim of contributing to scientific and technological development and thus, economic growth. The Scientific and Technological Research Council of Turkey (TUBITAK) is one of the governmental agents of Turkey responsible for such policies with its duties of promoting, developing, organizing, conducting, and coordinating R&D in line with national targets and priorities of Turkey. In this direction, TUBITAK encourages R&D, innovation, and entrepreneurship activities of public and private institutions with the development of human resources required for S&T via a number of programs. Besides, the Supreme Council for Science and Technology (SCST) of Turkey appointed TUBITAK to specify new science and technology (S&T) policy of Turkey for the period until 2023, which is 100th anniversary of the Turkish Republic, in December 2000. (TUBITAK 2017)

Priority Areas R&D Grant Program of TUBITAK is one of the S&T policy tools. It was started in 2012 with the aim of supporting and coordinating the domestic R&D projects which are result-oriented, having observable targets and looking after dynamics of related science and technology fields. Grants are given for ten different priority technology areas (PTAs) specified in the meetings of the SCST and Science Board (SB) of TUBITAK considering development plans, results of Technology Foresight Project (sub-project of Vision 2023), and STI policies and strategies. These areas are İnformation and Communication Technology (ICT), Automotive and Machine-Production, in which R&D capacity of Turkey is high; additional to Energy, Water, Agriculture, Health, and Aerospace, in which acceleration is essential. Moreover, Chemistry and Social Sciences and Humanity (SSH) are the ones which are chosen as PTA by the SB of TUBITAK.

The researchers working as full-time in universities, research institutions/centers, and public and private institutions can submit project proposal to the Program. Projects are divided into three scales according to their budget, small (up to 500.000 TRY), medium (up to 1.000.000 TRY), and large (up to 2.500.000 TRY).Footnote 1 Medium and large scale projects can include sub-projects up to three. Small scale projects could last for at most 24 months and other projects could last for at most 36 months. However, for some calls, there are some restrictions about the total funding budget, sub-project amount, and scale of the projects proposed in call texts.

Project proposals for the Grant Program are received with the specific calls launched on PTAs by TUBITAK. Two-stage application and evaluation procedure, described in Fig. 1, is used as application and evaluation procedure. As a result of the second-stage evaluation, the projects getting the point above the pre-identified passing score for each evaluation criteria and for total point are funded.

Fig. 1
figure1

Flow chart of projects proposed to prioritized R&D grant program

Until the time that data was retrieved, 166 calls had been launched under the Program. Distribution of these calls according to their PTAs is given in Fig. 2. It is observed that ICT, Energy, and Health are areas dominating the Program in terms of call amount while Aerospace, Chemistry, Machine-Production, and SSH are the ones having the lowest call amount since they are relatively new prioritized areas. Relatively fewer call for Automotive, on the other hand, might be due to its intensive requirements for the technological infrastructure and machinery-equipment investment, which could not be provided with the funding limits of the Program or being not preferred globally to study on.

Fig. 2
figure2

Number of calls launched for each PTA

PTAs are totally different from each other in terms of not only expected impacts and outputs from funded projects but also their level of development in Turkey. However, social, economic, technological, and scientific impacts of PTAs and supported projects are not monitored to revise (sub-) PTAs and reallocating grant budget among them. Moreover, although scientific knowledge in the literature, technical and human infrastructure, and technological progress are different for each PTA, evaluation and supporting criteria are the same for all calls. All of these facts lead the program to lose its effectiveness and efficiency in contributing to the level of development and growth.

The main target of this study is making one of the R&D policy tools implemented by TUBITAK more effective and efficient. It also intends to maximize the impacts of the Program and to increase output, input, and behavioral additionality of it with minimum amount of grant. By answering the question of “Does the program achieve the desired qualitative and quantitative impacts specific to each PTA and how can these impacts be improved?” as the research question of the study, the more contribution could be provided more effectively to the mission of the Program, which is reaching the level of developed countries and competing with them by decreasing economic vulnerability, budget deficit, and foreign source dependency while increasing economic growth and social welfare. In other words, outputs and impacts of the funded projects and the Grant Program could really and always serve to the strategic plan and targets of Turkey by implementing the policies, which will be suggested as a result of this study. Moreover, the suggested policies could be guideline for the countries which will be applied to public prioritization policies for R&D incentives.

This is the first study on the evaluation of the Prioritized Grant Program of TUBITAK additional to being one of the limited examples about the studies on the allocation of R&D incentives efficiently in the literature. As stated in the following section, there exist many studies in the literature on the efficient allocation of R&D budgets to projects. However, most of these studies are at the project selection level and the efficiency of the project is measured individually to support those with low risk and budget, as well as the high potential to produce value-added output. The studies considering R&D support program’s overall efficiency, on the other hand are so few. In addition, the number of studies on the efficiency and effectiveness of R&D projects is also limited for Turkish cases and most of the existing studies are impact analysis studies consisting of only output analysis of R&D projects in a selected field. All of these facts contribute to the originality of this study.

This study begins with the review of the literature. In this section, prioritization policy on R&D of Turkey is compared with other country examples and then, previous studies on evaluation of funding mechanisms and resource allocation in the literature are critically reviewed. It is followed by the methodology of the study including the description of the data and applied methods. Then, the results are analyzed, compared, and discussed to create more efficient and effective grant portfolio. Finally, the study is concluded with policy recommendations to make the Prioritized Areas Grant Program and R&D funding of Turkey more efficient and effective.

Review of the Prioritization Policies on R&D in Retrospect: an Empirical Discussion

Competition in international markets has extensively increased and become more complicated. However, the available technological, financial, and human resources of a country used to conduct research activities are scarce. Thus, there is a need for setting priorities for research activities considering not only international trends but also national needs, socio-economic structures, research infrastructure, and competences. There are two approaches in the literature for priority setting used by both developed and emerging economies: top-down approach with thematic priorities and bottom-up one with functional priorities. Priorities are dictated by governmental bodies for the former while foresight studies, surveys, and group discussions are conducted with the participation of all stakeholders to reach a consensus on priorities for the latter. The STI priorities of different countries are summarized on Table 1 with the targets lying under them and the methods used during the decision process, as benchmarking.

Table 1 Summary of prioritization policies applied in different countries

For Turkish case, these approaches are used in an integrated manner. Thematic priorities, i.e., PTAs, are announced by the SCST and the SB of TUBITAK with top-down approach. However, foresight studies are conducted with bottom-up approach to determine sub-technology areas of PTAs and the priority calls of the Program with their title, scope, aims, and special issues.

When PTAs and sub-technology areas of Turkey are compared with thematic priorities of other countries, it is seen that most of them are similar with the international trends. While ICT, Health, Agriculture, SSH, Water, and Energy are the areas prioritized by nearly all of the developed and emerging countries, Production Technologies is chosen as prioritized area only by China and EU, which are relatively less-developed countries. Moreover, (Aero)space is prioritized by two of the developed countries: Japan and Canada. On the contrary, one of the PTAs of Turkey, Automotive, does not exist within the areas prioritized by the countries expressed in benchmarking. Besides, there are also areas prioritized by many other countries, but not by Turkey directly, which are Transportation, National Defense, Public Security, Waste, and Environment.

In order to detect success of national STI policies in terms of economic growth and social welfare, direct and indirect short-run and long-run effects of R&D activities and innovation should be measured. To achieve this, three types of impact analysis are used integrated and sequentially: ex-ante to set targets, interim to monitor processes; and ex-post to evaluate the success of activities. There are various quantitative and qualitative methods in the literature for all types of impact analysis conducted for evaluation of STI systems.

Econometric analyses including Tobit models, Data envelopment analysis (DEA), stochastic frontier analysis (SFA), Tobit model estimation, and Maximum Likelihood Estimation (MLE) methods are quantitative methods. Feldman and Kelley (2006) identify projects having the greatest prospective impact on economic and social issues using multivariate LOGIT regression model with maximum likelihood estimation. As another example, Conte et al. (2009) compare the relative impact of R&D activities financed by different EU states and their innovation performance. DEA is preferred over the SFA to measure and compare efficiency scores, quantitatively. In addition, a complementary survey is also conducted to find out the contribution of their policy instruments to the efficiency of R&D and innovation policies at national level. Variables related directly to R&D and indirectly to human resource infrastructure, industrial dynamics, and policy instruments provide consideration of not only direct but also indirect effects from both economic and social perspectives. In addition, using both quantitative and qualitative approaches together to gather data contributes to the effectiveness of the study.

Peer-reviews and group analysis, on the other hand, can be given as examples to qualitative impact analysis methods. With the help of survey evidence obtained from Austrian firms, which another qualitative method, Falk (2007) measures effects of innovation policy in Austria. Although the applied methods and the obtained results are reasonable, relying only on the results of the survey which may be subjective and biased since questions may be answered with concerns of further subsidy applications is a weakness. In addition, long-term impacts could not be seen since there is not a time interval after the support given to the firms in the sample.

There are also methods which can be both quantitative and qualitative according to characteristics of data used, like Propensity Score Matching (PSM) and Difference-in Difference (DiD). Fedderke and Goldschmidt (2015) apply PSM to evaluate South African R&D funding mechanism. In the study of Czarnitzki and Hussinger (2004), the same method is used as a quantitative method to investigate the impact of subsidies on R&D and innovation output. The method and the sample used here are appropriate to measure long-term impacts of R&D subsidies given to the private sector. However, narrowing down performance measure only to patenting behavior and using only quantitative methods to detect the impact of R&D subsidy make unable to measure the economic and social impacts of R&D subsidies only with them. In the study of Tandoğan (2011), which is the most comprehensive example of studies on the impact of R&D funds given in Turkey, on the other hand, such methods are used as the qualitative ones. Existence of input additionality as a result of R&D subsidies given to private sector firms is investigated by using PSM and DiD, additional to quantitative Tobit model. As a result, not only effectiveness of Industrial R&D Support Program of TUBİTAK but also relationship between private R&D intensity and receiving a subsidy are found out. This study is an example one of the few studies conducted as the impact assessment of R&D subsidies given in Turkey. Moreover, using quantitative methods with both quantitative and qualitative data increases the reliability, effectiveness, and objectivity of the study.

To increase the benefits obtained from R&D activities, scarce resources should be allocated to prioritized fields in an efficient and effective way. In the literature, there exist both quantitative methods applied for this aim, some of which are similar with the ones used for impact analysis.

Garrison et al. (2011) propose a quantitative model based on measuring the unit mortality and morbidity cost impacts of innovation activities in order to choose the most cost-effective alternative. Linton et al. (2002), on the other hand, use DEA and Value Creation Model (VCM) to measure R&D performance or potential and choose an optimal project portfolio. Eilat et al. (2008) also use DEA integrated to Balanced Scorecard (BSC) to evaluate R&D projects in different stages of their life cycle to distribute the scarce resource for them optimally.

The interactive system developed by Wonglimpiyarat (2008) is an impact analysis process allocating resources efficiently and effectively among projects from different technology fields.

European Commission (2011) allocates funds to policy options more effectively and efficiently considering descriptive statistics of science-technology-innovation indicators and results of econometric estimation exercises on the data obtained from expert panels and online surveys. Although the budgets allocated to priorities of H2020 are not reallocated to technology fields related to them, which is a missing part of this study, the methods used for allocation of the budget to the priorities of H2020 are reasonable. It could also be more objective by adding some indicators and variables to the quantitative analyses and a few questions to qualitative surveys. Similarly, Volinskiy et al. (2011) work on allocation of Canadian public research funds to agricultural biotechnology studies. Firstly, Bayesian decision-making framework and a probabilistic target criterion are used to determine quantitative individual utility values. Then, via the choice experiment study, the best alternative is chosen from five different research funding allocations. Although the applied method is appropriate for this case, the assumptions may be problematic. In particular, the assumption that panel members and decision makers have no information about the R&D returns will be invalid since the theory fully depends on their preferences and expectations.

Despite the importance and benefits of impact analysis, examples of such studies are limited due to its high costs, its requirement of long time, and difficulties in obtaining data. Even if there is no study about the impact evaluation of TUBITAK’s Prioritized Areas R&D Grant Program. In addition, although an effective impact analysis should also consider all effects, studies classified as impact analyses in the literature only consider outputs of R&D studies. Moreover, it should be noted that most of the studies in the literature related to resource allocation are at project selection level. There is not a comprehensive study on distribution of financial resources to the different technology fields and different R&D policy tools. Furthermore, for most of the studies in the literature, only qualitative or quantitative data and/or method is used, which limits the reliability and objectivity of the studies. In conclusion, it can be inferred from all of these facts that the analyses done in the remaining sections of this study and policies recommended as a result of them will significantly contribute to the literature.

Methodology

The Data

There are two different datasets used during this study. These are retrieved from TUBITAK database on April 21, 2017.

The first dataset includes the information of projects proposed for the first and second stage of the calls launched from 2012 to retrieval date of data. These are type (first/second stages and main/sub-project), final situation (Proposed, Returned, Rejected, Supported, In-Process, Finalized, etc.), calls, durations, and requested and funded budgets of the projects with information about team members of them (institution, institution type, city, etc.) and information to relate second-stage projects with the ones passing first stage.

The other dataset is about the outputs belonging to supported projects. Type of the output (scientific paper, presentation, thesis, dissemination, patent application, book chapter, new project, prize, registration) and the information about the project to which it belongs is included in this dataset.

Methods

In this study, TUBITAK’s Grant Program is evaluated considering differences between technology areas to recommend new policies promoting the efficiency and effectiveness of the program by reallocating the available resources for these areas.

For this evaluation study, the program indicators are identified and analyzed additional to the econometric analysis applied to estimate the relation of these indicators with outputs obtained from the supported projects. Besides, interviews are conducted with a sample of supported project coordinators. Econometric analysis and interviews are conducted only for three of PTAs: ICT, Energy, and Health. The most important reason of this situation is the amount of launched calls and project proposals are higher for these PTAs with the higher finalization rate of their projects. Additionally, the fact that these areas represent the ones having different characteristic is also the reason of choosing them to implement such an analysis. ICT represents fields in which R&D and innovation capacity of Turkey is strong, while Energy is the area in which acceleration is essential according to the STI strategy papers of Turkey. Health is also the one requiring acceleration, but it is added to this strategy in January 2013.

As the first stage, the program indicators are identified as historical baseline by using datasets of projects and outputs. This study is conducted with the aim of due diligence and ex-post impact assessment. To achieve this, firstly, total and average numbers of proposed, supported, and finalized projects are examined. Their distributions with respect to scale, type of institution in which projects are conducted and sub-project amount are also examined on the basis of PTAs to investigate the differences in the success of the projects having different features for each PTAs. It is followed by analyses on requested and given funds. It should be noted that average amounts are calculated as dividing the total amount by the number of calls launched for the respective PTA. This part is finalized with the examination of outputs obtained from supported projects, which is also an example of impact evaluation methods applied on funding programs in the literature. All these analyses are conducted to find out the current situation of the Grant Program with its inefficient and ineffective points and whether these inefficiencies change with PTAs and/or project characteristics. Moreover, these indicators provide information for the selection of independent variables to econometric analyses.

As the second method, econometric analyses are conducted to find out the output additionality property of project characteristics and call features. The characteristics of projects and calls detected as the root of inefficiency and ineffectiveness from the analysis of program indicators are chosen as independent variables additional to the ones used in the similar studies discussed in the “Literature Review.” Two different model types are estimated by using the Ordinary Least Square (OLS) method.

In the first model, the total output amount of supported projects is regressed on the characteristics of projects. In the second model, on the other hand, the mean output amount of projects supported for each call is regressed on both the features of calls and mean value of the variables used in the previous model if it is applicable. The dependent and independent variables used in these models are explained in Tables 2 and 3, respectively.

Table 2 Dependent and independent variables of project-based estimation model
Table 3 Independent variables of call-based estimation model

Instead of the original value of output amount and the one weighted with respect to output type for each PTA is regressed. Weight for each output type is determined by considering not only the distribution of outputs according to their types for different PTAs but also responses received from the coordinators participated to the interviews. Weighted total output amount (woutput) of each project is calculated as follow:

$$ {\sum}_i Weigh{t}_{ij}\ast {\left( Output\ Amount\right)}_i/\mathrm{Total}\ \mathrm{Output}\ \mathrm{Amount} $$

Where “i” represents output types and “j” represents PTA to which related project/call belongs.

As an example, the calculation of the weighted output amount for a sample of projects from different PTAs is given in Table 4.

Table 4 Examples of weighted output amount calculation for different PTAs

As the last evaluation method, interviews are organized with the sample of coordinators. The sample includes coordinators of 16 supported projects (both finalized -7- and being in-process -9-) having and not having outputs. Coordinators in the sample are distributed to each PTA proportional with the results of descriptive statistics of the program indicators. Considered statistics are about gender, application year, institution type, and location of institutions. Most of the interview questions are prepared by adapting the survey and interview questions used in the similar studies in the literature. Some additional questions related to the nature of this case, especially questions in the part of about the policy of the overall program, are also used. The questions asked during the interviews are given in Appendix A. The main target of this exercise is to detect not only output but also input, project, and behavioral additionality of the Grant Program from the perspective of stakeholders for different technology fields. The questions about the cooperation of the team members before and after the project are used to detect behavioral additionality with those about the opportunities and opportunity costs faced by the team members and the coordinator institution as a result of the project. The projects proposed by the coordinators to the funding programs of TUBITAK before and after the funded project are also questioned to find out the project additionality. Besides, there are also questions to reveal the output additionality in terms of both long-run impacts of the project, and their scientific contribution to the literature. The questions about the ability of the project to train new qualified researchers also enable the detection of the input additionality. Lastly, there are also questions on how to sort out the output types considering their contributions to the related research areas and the Program. These rankings are used to weight outputs according to their types for each PTA during the econometric analyses stage of this study.

Methods applied in this thesis with their interactions are summarized in Fig. 3.

Fig. 3
figure3

Graphical expression of analyses conducted in this study

Results and Discussions

The analyses described in the previous section are conducted integrated and significant results are emerged about the effectiveness and efficiency of the Program.

Descriptive statistics of program indicators give information about the distribution of proposed and supported project with requested and given funds in terms of project characteristics additional to that of outputs, as due diligence.

It is indicated from Fig. 4 that total project proposal amounts of PTAs are generally compatible with the number of calls seen in Fig. 2. Moreover, these values also inferred that researchers in Turkey have a tendency to study on the areas, which are also prioritized by developed and emerging countries according to the literature: ICT, Energy, Agriculture, and Health. Average amount of projects proposed for a call, however, is the highest for Chemistry and SSH, opposite to the call amount. The reason of this fact is probably that these areas are relatively newer and they have fewer amount of call. However, average proposal amount for aerospace and machine production is low. This situation might be due to lower amount of qualified researchers in Turkey, studying on these subjects. As opposed to the project proposal amount, total and average number of supported projects per call are lowest for Chemistry and SSH, as seen in Fig. 5. Higher supported project amount for ICT, Energy, Agriculture, and Health indicates that researchers in Turkey studied on the areas are qualified enough to conduct a project in the areas. These areas have also importance all over the world, as summarized in Table 1.

Fig. 4
figure4

Total and average number of proposed projects for each PTA

Fig. 5
figure5

Total and average number of supported project for each PTA

Figure 6 states that distribution of project proposal amounts with respect to each institution type are similar for different PTAs, with some exceptions. Share of public institutions is higher and that of university is lower for Agriculture than for other areas. This is due to the existence of General Directorate of Agricultural Research and Policies, and Food Institute of TUBITAK conducting R&D studies on Agriculture as the major representatives of the public research institutions in Turkey working on this area with their high competence. Besides, share of private sector is higher for the areas of ICT, Automotive, Machine-Production, and Aerospace. It is reasonable since the private sector intensely engages with these areas in Turkey. However, fewer applications from private sector for Energy calls, the area on which private sector is also intensely worked in Turkey, is questionable. Likewise, researchers from public institutions do not generally prefer applying the Program for Health and SSH calls despite the active role of those institutions on these areas. This might be because such institutions are engaged with these areas not for R&D, but for marketing and trading purposes.

Fig. 6
figure6

Distribution of project proposals with respect to the institution types for each PTA

It is inferred from Fig. 7 that for Automotive and Energy, projects of public institutions, specialized on this area, are not qualified as those of universities and private sector due to their lower success rate. For Chemistry and ICT, on the other hand, private sector projects having the highest success rate indicate that they are much more qualified than those of universities and public institutions.

Fig. 7
figure7

Success rate with respect to institution types for each PTA

As seen in Figs. 6 and 7, nearly all of proposed and supported projects are from universities for all PTAs. Additionally, the rate of having sub-project for proposed and supported projects is low, as seen in Figs. 10 and 13. It can be concluded from these facts that attempts to provide university-industry cooperation within the scope of the Program are not as successful as intended.

According to Fig. 8, small-scaled project proposals constitute the majority of them for all PTAs, except SSH regardless of specific funding requirements of PTAs. Fewer small-scaled projects for SSH despite their lower machine-equipment cost are interesting. On the contrary, small-scaled projects dominate the project proposals for ICT and Machine-Production projects, machine-equipment costs of which must be high. The same analysis on supported projects indicates that majority of supported projects are also small-scaled, except Agriculture, Health, and Water. Even supported Aerospace projects are all small-scaled (Fig. 9).

Fig. 8
figure8

Distribution of project proposals with respect to scale for each PTA

Fig. 9
figure9

Distribution of supported projects with respect to scale for each PTA

As stated before, medium and large scaled projects may have sub-projects up to three unless any other restriction exists in the call text. Such projects are detected from second-stage proposals since the information about whether a project has a sub-project or not is not available for the first stage. As indicated in Fig. 10, the rate of having sub-project approximately ranges from 20% to 30% for all PTAs, except Agriculture. Moreover, if the restriction on minimum sub-project amount for three of SSH and two of Water calls is considered, it is seen in Fig. 11 that researchers prefer proposing project having as few sub-project as possible.

Fig. 10
figure10

Rate of projects having sub-project to second-stage project proposals for each PTA

Fig. 11
figure11

Distribution of project proposals having sub-project with respect to their sub-project amount for each PTA

These results indicate that issues in call texts about the obligation of having sub-project or restriction on sub-project amount for medium and large-scaled projects do not affect the overall statistics. For ICT, Health, and Water cases, this may be because the rate of calls having such obligation or restriction is too low. It can also be argued for other PTAs having such restrictions that this situation prompts researchers to propose small-scaled projects rather than medium- and large-scaled ones. The rate of medium and small-scaled project proposals given in Fig. 7 and distribution of project proposals according to sub-project amounts shown in Fig. 11 also promote this claim. The reason of this might be the additional bureaucratic procedures during application and operation processes for the projects having sub-projects and difficulties in management of a project having crowded team in multiple institutions.

According to Fig. 12, having sub-project is more advantageous to get support from the Program for all PTAs, except Aerospace and SSH, having no supported projects with sub-projects. However, the rate of projects with sub-projects among all supported projects is still low for majority of PTAs, except Agriculture, Chemistry, Machine-Production, and Water calls, as given in Fig. 13. If restrictions on having subprojects for medium- and large-scaled projects for SSH and Water calls are considered, it can be concluded that these restrictions are not effective with the same degree for all PTAs to increase success rate of projects for all PTAs.

Fig. 12
figure12

Success rate of all projects and projects having sub-project for each PTA

Fig. 13
figure13

Rate of projects having sub-project to supported projects for each PTA

Figure 14 indicates that the success rate of projects having different sub-project amount differs with PTAs. For Energy and Automotive calls projects having three sub-projects are more successful while those having one sub-project are supported more for Agriculture calls. This means that restrictions on existence of sub-projects and their amounts should be used for calls of some PTAs, but not the same ones for all to increase their effectiveness (Fig. 14).

Fig. 14
figure14

Success rate of projects having sub-project with respect to their sub-project amount for each PTA

According to Fig. 15, distribution of total requested funding to institution types is proportional with that of total project proposal amount, given in Fig. 6 for all PTAs. In addition, project from universities constitutes the largest part of total requested budget for all PTAs while that of projects from private sector has the lowest share for all PTAs except ICT. Besides, the average requested budget of projects proposed from each institution type differs with PTAs. Moreover, the average requested budget of projects proposed from public and private institutions are relatively much higher than that of other projects for Energy, Aerospace, and Automotive.

Fig. 15
figure15

Total and average amount of fund requested for proposed projects with respect to institution types for each PTA

It is expected that funded budget for the projects from private sector should be higher due to the payments of project team included in the budget, but it is not the case except for Automotive and Health as seen in Fig. 16. Even average-funded budget of public institutions is higher than that of private ones for Health projects. The reason of this situation is probably that machinery and equipment costs of projects, half of which are provided by institutions for projects proposed from private sector, dominate funding budget. Higher funding amounts for the projects from public institutions with poorer R&D infrastructure also promotes this claim. This situation points out the requirement of a mechanism to provide machinery and equipment infrastructure for some institutions.

Fig. 16
figure16

Total and average amount of fund given to supported projects with respect to institution types for each PTA

When the total values of given funds are compared with total requested budgets given in Fig. 15, it is observed that the distributions of these values with respect to institution types are proportional with each other. Similarly, proportions of average-given funds with respect to institution type are similar with those of average requested budgets with the consideration of success rates seen in Fig. 12. Moreover, it is observed that average values of given funds are lower than that of requested ones for all PTAs, especially for Aerospace, Chemistry, Automotive, and SSH.

Figure 17 indicates that the rate of having output ranges from 24% to 45% for each PTA while projects of Aerospace, Chemistry, Machine-Production, and SSH do not have output since they are new areas, projects of which have started recently. The highest rate of having output belongs to Energy projects while the lowest one belongs to Health projects. This result indicates that at most, only half of the supported projects are qualified enough to get a tangible solution for the national needs in the prioritized areas.

Fig. 17
figure17

Rate of projects having output to all supported projects for each PTA

Total and average output amounts are quite different with respect to PTAs, as indicated in Fig. 18. Average output amount per project is highest for Automotive while the total is highest for Energy. Besides, the rank of PTAs with respect to average and total output amount is so different from that of PTAs with respect to the rate of project having output, given in Fig. 17.

Fig. 18
figure18

Total and average number of output for each PTA

Output types existing in the system are scientific paper, presentation (verbal/poster), book, patent application, registration, thesis (master/PhD), dissemination, prize, and new project. Although, information about being national/international exists for the outputs, quality, and recognition of them, such as published in a journal being indexed or not and number of citation, could not be obtained from the available data.

If the total output amount is examined with respect to output types, it is seen that majority of outputs are presentations for all PTAs, as seen in Fig. 19. In addition, diversity of the outputs with respect to their types and distribution according to this are different for each PTA. For instance, for ICT and Water projects, there are only four different output types while Automotive and Agriculture projects have nearly all types of outputs.

Fig. 19
figure19

Distribution of outputs with respect to their types for each PTA

To summarize the results of the analyses on program indicators, it is observed that number of proposed and supported projects fluctuate with PTAs due to the huge gap between the number of researchers working on different areas. Moreover, the rate of having sub-project and number of sub-project is so low nearly for all PTAs since these features do not contribute to getting support despite their additional bureaucratic requirements. Similarly, most of the project proposals are small-scaled, due to the restrictions on minimum sub-project amounts for medium and large-scale ones. When these results combine with the fact that most of the proposed and supported projects are from universities, it is also inferred that university-industry cooperation desired from the Program cannot be sufficiently obtained. However, equivalent prioritized R&D grant programs in other countries aiming to contribute economic competitiveness and satisfy the needs of industry, as discussed in Table 1, could achieve this goal. Besides, the public institutions, having relatively poorer R&D infrastructure, request and get more funds. Lastly but most importantly, output amounts and rate of having output are so few and most of them are at basic-research level, which is not sufficient to meet the product-oriented targets of the Program.

After examining the current situation of the program, relation of output amount with project and call characteristics is investigated with the help of econometric analyses. Two different models are estimated under the scope of these analyses.

Firstly, the weighted total output amount of projects is regressed on project characteristics by using OLS. On the obtained raw model, diagnostic tests are applied and it is observed that there are breaks at the points where PTAs change. This indicates that output amount is related with different characteristics of projects at different levels for different PTAs. Thus, OLS estimation is repeated for each PTA, separately. Table 5 indicates the regression results of weighted output amounts for each PTA.

Table 5 Estimation results of selected project-based model for different PTAs (* for p < 0.1; ** for p < 0.05; *** for p < 0.01)

In addition, to eliminate the structural breaks observed for Energy and Health projects, different structural forms are used for estimation equations of these PTAs. If the final regression models estimated for each PTA are compared, it is inferred that weighted output amount of health projects is significantly related only with teamsize additional to timeafterstart. Thus, it is the least explained one with available variables, which makes R2 and R2adj values the lowest for model of health projects. Moreover, it is seen than time elapsed after projects start affects the output amount much higher for Health projects. Being large-scaled, however, has higher negative effect for model of ICT projects than for model of Energy projects but insignificant effect for Health projects. Besides, having a researcher from privatesector and being medium-scaled have effect on estimated weighted output amount for Energy and ICT projects, respectively, but not for others. This indicates the inefficiency of scaling for all PTAs. Additionally, team size has negative effect only for Health projects while privatesector has negative effect only for Energy projects. Thus, these two characteristics are also inefficient and ineffective in terms of output additionality.

In the second model, on the other hand, relation of call features with average weighted output amount is regressed on characteristics of calls, additional to average characteristics of projects belonging to that call. Diagnostic tests indicate that output amount is related with different characteristics of calls at different levels for different PTAs. Thus, the selected model is estimated for Energy, ICT, and Health calls separately. The final estimation results of these models are given in Table 6. In addition, different structural forms are used for Energy calls to eliminate structural break detected as a result of diagnostic tests. Weighted output amount of health calls is significantly related only with meanfund additional to timeaftercall while that of Energy calls is related with, positively related with mainprojects, too. For ICT calls, on the other hand, mean weighted output amount is significantly related with mainprojects and scalerest.

Table 6 Estimation results of selected call-based model for different PTAs (* for p < 0.1; ** for p < 0.05; *** for p < 0.01)

If the final regression models estimated for each PTA are compared, it is seen that the time elapsed after projects start and mean of given fund to supported projects of a call affect the mean weighted output amount much higher for Health calls than for Energy ones. The effect of supported main project amount, on the other hand, is the lowest for Energy calls.

Restriction on the scale of proposed projects has significant and positive effect only for ICT calls, which makes it effective and efficient only for ICT calls but not for Energy ones. It should be noted that non-existence of this effect as a significant one is reasonable for Health calls since there is no Health call including such a restriction. In addition, negative significant relation of mean funds with output amounts makes funds given to Energy and Health calls inefficient while its insignificant relation for ICT calls makes it ineffective. Similarly, meanteamsize is also inefficient and ineffective for ICT calls due to its high correlation with meanfund. Besides, insignificant effect of privateparticipation makes the restrictions and enforcements on participation of researchers from private sector ineffective. Low rate of proposing to the Program and low passing first-stage rate of projects proposed from private sector also supports this claim. Similarly, minimum peer-review grade used as supporting criteria is also ineffective for all PTAs.

Finally, answers of the project coordinators participated in the interviews are analyzed not only to find out input, output, project, and behavioral additionalities but also to learn opinions of the stakeholders about the targets and characteristics of the Program.

Firstly, the projects, team members of which did not have project before the funded one, have at least one researcher proposing project after it, which may indicate the project additionality of the Program. In addition, for Health and Energy projects but not for ICT ones, many projects have team members studying together after the funded project. This shows the contribution of the funded projects to scientific cooperation for Energy and Health.

It is observed that if proof-of-concept and basic research activities exist before the funded projects, these projects are more likely to have output for all PTAs. Thus, to increase the efficiency and output additionality of the Program, the basic research activities for the subjects of a prioritized area project should have been completed or at least started before it. To ensure this, starting and target technology readiness levels (TRL) should be decided for each call and stated in call texts clearly. Then, the proposed projects should be expected to satisfy these levels as an application or supporting criteria. Besides, if any project as a basic research of PTAs, sub-technology fields and prioritized subjects has not been studied before, such projects should be supported by means of an additional funding program, before launching a call under the Program.

Opinions of interviewees about scaling are not affected by the situation of having output for Energy and Health projects, but for ICT ones. Thus, it can be concluded that scaling has different effects on output additionality for different PTAs and restrictions related with scaling should be different for each PTA, as suggested by interviewees. In addition, it is indicated that the rate of projects having sufficient funding amount does not change with scale for all PTAs. Besides, it does not affect the situation of having output, too. This means that funding amount is inefficient for output additionality.

The interviewees mostly prefer the Program due to its relatively much higher funding amount, although most of the supported projects are small-scaled. It is also remarkable that opportunity of conducting R&D study with university-industry cooperation and being and product-oriented program, which are the most important and distinctive features of the Program, are stated as the reason of choosing the Program only by few researchers. This means that target group of the Program does not handle special targets, expectations, and requirements of it, which reduces the quality and support rate of this program.

Nearly all projects have the effect of increasing the competitiveness of Turkey and decreasing the foreign-source dependency economically and technologically. However, only half of the projects contribute to economic growth and creation of social welfare, according to the claims of interviewees. Similarly, contribution to the conscious usage of the technology and university-industry cooperation remains at one third. Indeed, projects including researchers from private sector could not be entirely contribute to the university-industry cooperation. Coordinators of the ones, which cannot contribute to the university-industry cooperation claims that this cannot be achieved with the current situation of industry and technology transfer offices. Nonetheless, projects, which are likely to provide university-industry cooperation, can contribute to reduce foreign-source dependency and increase competitiveness of the country with their support on the least studied areas. This infers that the Program can serve some targets of Vision 2023, but not all of them, since it could not fully convert advances in R&D to economic and social benefits.

More than two-third of interviewees claim that they satisfy not only aims and targets of the calls but also the scientific and social effect they expect to create. Projects of the ones not thinking in this way belong to Energy and Health calls. They state that if the time and budget became more flexible, they could achieve these targets. This shows the requirement of different time and budget restrictions for different PTAs, even for different calls.

As seen in Fig. 20, more than half of the projects provide knowledge to their team members and qualified R&D personnel to coordinator institutions while only three of these projects increase the prestige and familiarity of them. Figure 21 indicates that for half of the projects, coordinators of which participate to interviews, there is no opportunity cost and for only four of them forgo from the time and money which can be used for other R&D activities. This means that most of the capacity used for the funded projects would be idle if these projects were not conducted.

Fig. 20
figure20

Number of projects providing given opportunities to project teams and coordinator institutions

Fig. 21
figure21

Number of projects causing given opportunity costs to project teams and coordinator institutions

In conclusion, it is pointed out that supported projects of ICT, Energy, and Health have high contribution in terms of project additionality. Input additionality is also provided with the help of new qualified R&D personnel and the increasing knowledge of existing ones additional to improvements in the R&D infrastructure of coordinator institutions. Moreover, output additionality resulting from decreasing economic and technological foreign-source dependency and increasing competitiveness of the country is observed. In addition, behavioral additionality significantly exists for Health and Energy. On the contrary, contribution to the university-industry cooperation is so low even for the projects having team members from both of industry and university.

Concluding Remarks and Suggestions

Within the scope of this study, Prioritized Areas Grant Program is evaluated with the examination of its qualitative and quantitative effects for different PTAs. It aims to figure out the ineffectiveness and inefficiencies in terms of output, input, behavioral, and project additionalities of the program. To achieve this, both quantitative and qualitative methods, which are descriptive statistics of program indicators, econometric analysis, and interviews, are conducted. As a result of these analyses, ineffective and inefficient features of the calls and projects, affecting the application amount, success rate, output amount, and quality of the outputs negatively, are detected. Considering these results, some policies, summarized in Table 7, are suggested to obtain more effective and efficient Program.

Table 7 Summary of policy recommendations and policy tools to obtain effective and efficient Prioritized Grant Program

The suggested policies could be classified as micro-, mezzo-, and macro-level policies. Micro-level policies are suggested for the applicants and their institutions while mezzo-level ones should be applied on the processes and regulations of the Program by TUBITAK. Macro-level polices, on the other hand, require national intervention, which could be achieved with general tools out of the Program. The mezzo-level policies could be helpful for other national and international institutions having responsibility of supporting R&D studies while macro-level policies could be a guideline for the countries, which have an intention to prioritize their R&D incentive policies.

To increase the quality of proposed and supported projects, characteristics of both projects and calls should contribute more to output additionality. Firstly, different restrictions on the sub-project amounts and team size could be applied for each PTA, even for each call, at mezzo-level. Moreover, to increase the efficiency and effectiveness of peer-review in terms of output additionality, grading system including evaluation and supporting criteria should be revised. Participation of the project coordinators in peer-review panels could contribute to the evaluation process with interactive discussions on the prospective changes in the project proposals to eliminate their unclear points and deficiencies. These recommendations about the evaluation process will also eliminate the disparities on the supporting rate between PTAs, and create positive impact on the quality and quantity of supported projects, and thus, their effects.

To solve the problem of insufficient knowledge on R&D, Grant Program, and bureaucracy, the briefings which will be given by experts from TUBITAK on these issues could be provided by the related institutions, as a micro-level policy. Simplification of rules and regulation to eliminate their discouraging impact could also be suggested as a mezzo-level policy tool. Moreover, there is also a need for the policy aiming to raise R&D personnel and increase their competences on the prioritized areas before launching call. As such a policy tool, basic research activities on the subjects, which will be prioritized in the future could be supported by additional funding mechanisms at macro-level. Conducting such activities could also help researchers to prepare the prospective calls, which will be launched in the future at micro-level.

International cooperation supports for the prioritized areas could also be a beneficial tool at macro-level to increase the speed of improvements in these areas. The cooperation with the countries competence enough in an area for which acceleration is required in Turkey could be helpful to increase the quality and knowledge level of R&D personnel studying on this area. This could also contribute to the process of converting knowledge-based outputs to real products with the help of the advanced R&D infrastructure in more developed countries. For the side of the areas on which innovation capacity of Turkey is strong, having the researchers in less-developed countries conduct the relatively more basic level research activities could be promoted with such a support mechanism. This provides saving time for more advanced and product-oriented researches in those areas.

To increase the quality of outputs obtained from supported projects, more contribution of these outputs to social, scientific, and economic targets of Vision 2023 and the Prioritized Areas Grant Program should be provided. Firstly, setting starting and target TRLs as application criteria, which could be directing projects to these aims, could be used for this aim, as a mezzo-level policy tool. However, if basic research of a subject does not exist, this decreases the amount of proposed projects. To eliminate this, the generation of basic research knowledge before the calls which will be launched on that subject should be provided. The basic research activities on the areas and subjects detected as not having such knowledge by the SCST with the foresight studies could be subsidized by additional funding mechanisms at macro-level. These subjects could reach the expected starting TRL with these subsidies. In addition, to convert the outputs of the funded projects to real products having high competence in global basis, university-industry cooperation should be provided more effectively. To achieve this, outputs of the funded projects should be shared with the industrial and public sector institutions, which are capable enough to commercialize them as a real product. Such institutions should also be subsidized by an additional funding mechanism.

In conclusion, the analyses conducted in this study and the policies recommended as a result will be helpful to obtain more efficient and effective prioritized R&D Support mechanism, additional to their contribution to the literature. However, some further studies, such as impact assessment and evaluation studies for other R&D support mechanisms and analyses in relation with the Program, are required to maximize the benefit from these R&D support mechanisms.

Notes

  1. 1.

    Average exchange rate in the time horizon of this study is approximately 3.70 TRY/USD

References

  1. Conte, A., Schweizer, P., Dierx, A., & Ilzkovitz, F. (2009). European Commission (September, 2009). Retrieved from Munich Personal RePEc Archive.

  2. Czarnitzki, D., & Hussinger, K (2004). The link between R&D subsidies, R&D spending and technological performance. Centre for European Economic Research (ZEW), Mannheim.

  3. Eilat, H., Golany, B., & Shtub, A. (2008). R&D project evaluation: an integrated DEA and balanced scorecard approach. Omega, 36, 895–912.

    Article  Google Scholar 

  4. European Commission (2011). Impact assessment. Brussels.

  5. Falk, R. (2007). Measuring the effects of public support schemes on firms' innovation activities survey evidence from Austria. Research Policy, 36, 665–679.

    Article  Google Scholar 

  6. Fedderke, J. W., & Goldschmidt, M. (2015). Does massive funding support of researchers work?: evaluating the impact of the South African research chair funding initiative. Research Policy, 44, 467–482.

    Article  Google Scholar 

  7. Feldman, M. P., & Kelley, M. R. (2006). The ex-ante assessment of knowledge spillovers: government R&D policy, economic incentives and private firm behavior. Research Policy, 35, 1509–1521.

    Article  Google Scholar 

  8. Garrison, L. P., Bauch, C. T., Bresnahan, B. W., Hazlet, T. K., Kadiyala, S., & Veenstra, D. L. (2011). Using cost-effectiveness analysis to support research and development portfolio prioritization for product innovation in measles vaccination. The Journal of Infectious Diseases, 204(1), 124–132.

    Article  Google Scholar 

  9. Linton, J., Walsh, S., & Morabito, J. (2002). Analysis, ranking and selection of R&D projects in a portfolio. R&D Management, 32, 139–148.

    Article  Google Scholar 

  10. Tandoğan, V. S. (2011). Impact analysis of industrial research and development subsidy programs in Turkey: an appraisal of quantitative approaches. Ankara: Middle East Technical University.

    Google Scholar 

  11. TUBITAK (2017). Who we are?. Retrieved from https://www.tubitak.gov.tr/en/about-us/ content-who-we-are on 14.10.2017.

  12. Volinskiy, D., Veeman, M., & Adamowicz, W. (2011). Allocation of public funds to R&D: a portfolio choice-styled decision model and a biotechnology case study. 34, 121-139.

  13. Wonglimpiyarat, J. (2008). Building effective research evaluation systems to assist R&D investment decisions. International Journal of Business Innovation and Research, 2(2), 123–140.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to express their thanks to TUBITAK administration for their permission and support to use data obtained from TUBITAK database.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Kübra Gürbüz.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix – Interview Questions

Appendix – Interview Questions

This interview is conducted for the impact analysis study of TUBITAK Prioritized Areas Grant Program. The study aims to make funding mechanism more efficient and effective by detecting the current impacts of funded projects and their PTAs in terms of output and behavioral additionality. During the interview, you will be asked questions about both the pre-funding, funding, and post-funding processes of your project, as well as your opinion about the program itself.

  1. a.

    Information about background of the project and current situation

The purpose of the questions in this section is to get information about both your opinions on the Program and other TUBITAK supports before your prioritized area project as well as the features of the project you are funded.

  1. 1.

    Did you have a joint work/TUBITAK project together with these institutions/organizations?

  2. 2.

    If you had worked work in another institution before and/or after the projects, which are these institutions? Is there anybody in the project team working in these institutions? Is there any study/TUBTIAK Project you conduct together with anybody whom you have studied with in these institutions?

  3. 3.

    What are your studies on call subject before your prioritized area project?

  4. 4.

    Did you have the idea of your prioritized area project before the call?

  5. 5.

    How did you hear Prioritized Areas Grant Program and the related call? Why did you prefer this Program to get support for your project?

  6. 6.

    How much did the basic research idea of the project evolve when you applied for the grant?

  1. a.

    There was no proof-of-concept/basic research before my project. (or equivalent to this)

  2. b.

    Proof-of-concept/basic research studies partially existed while applying to the Program. (or equivalent to this)

  3. c.

    The concept was just proven and supported by basic researches before my project. (or equivalent to this)

  4. d.

    I have no idea.

  5. a.

    Impact and SWOT analyses of the project

Aim of the questions in this part is learning your opinions about both the application-evaluation-operation processes of the Program and the support you get with its outputs and impacts.

  1. 1.

    Could you evaluate application process of the Program in terms of procedures, bureaucracy, duration, and transparency?

  2. 2.

    What do you think about the expected university-industry cooperation and application of all public/foundation universities, public institutions, and private sector to the same calls? To what extent university-industry cooperation can be provided via the Program? How could your project and overall Program contribute to bringing the new technologies based on information produced in universities and research centers into use of industry and public institution?

  3. 3.

    Does your project have any of the following social, economic, and scientific impacts?

  1. a.

    Reducing foreign-dependency in technology /increasing global competence of the country (reducing current account deficit)

  2. b.

    Contributing to economic growth

  3. c.

    Contributing to structural reforms which could be reduced fragilities in the economy

  4. d.

    Contributing to social welfare

  5. e.

    Contributing to conscious use of the technology

  6. f.

    Contributing to an area studied relatively less

  7. g.

    Formation of R&D projects within the frame of university-industry cooperation

  8. h.

    Creating employment and contributing to raise of qualified R&D personnel (reducing unemployment)

  1. 1.

    Is there any contribution of your project to the literature? If yes, choose these contributions from the following cases?

  1. a.

    Improvement/use of a new approach

  2. b.

    Improvement/use of a new dataset

  3. c.

    Improvement/use of a new theory

  4. d.

    Improvement/use of a new method/model

  5. e.

    Improvement/use of a new process

  6. f.

    Improvement/use of a new material/product

  1. 1.

    Did/will your supported project significantly contribute to aims and targets stated in the text of related call? What are these contributions? If you think that, expected impact did not/may not be produced, what are the factors causing this situation?

  2. 2.

    Could you evaluate the outputs obtained from the project activities and your expectation while applying? If you think that you could not/may not able to create outputs and impacts you desired, what are the factors causing this situation?

  3. 3.

    Which opportunities are emerged for you, project team members and your institutions as a result of support you get?

  4. 4.

    Is the funding amount sufficient? Is there any difference between requested and given fund? If yes, how has this revision affected your project? What do you think about the scaling applied for the Program?

  5. 5.

    If you had not been supported in the scope of this program, what would have you thought about conducting this project? Would there be any changes in your Project when you conducted this project without this support?

  6. 6.

    Are evaluation and tracing processes of the Program sufficient? Are the performance indicators suitable and objective?

  1. a.

    About the policy behind the prioritized areas grant program

The aim of the questions in this part is getting your opinion about the policy and the Prioritized Areas Grant Program and your improvement suggestions.

  1. 1.

    Has the support you get from the Prioritized Areas Grant Program met your expectation? If you design this program, how would you change it?

  2. 2.

    Which of the technology areas would you prioritize, except your area, in the scope of the Prioritized Areas Grant Program? For which subject would you prefer to launch call in your area? Why?

  3. 3.

    If you prepared the text of the call you applied, how would you change it in terms of aims, targets, content, specific issues, and etc.?

  4. 4.

    Is there any need for a new national or international support mechanism following the Prioritized Areas Grant Program or at the same time with it? If yes, what kind of support mechanism should it be?

  5. 5.

    Are information activities about TUBITAK supports and the Prioritized Areas Grant Program sufficient?

  6. 6.

    Would you recommend other scientists in your area to apply for the Prioritized Areas Grant Program and other TUBITAK supports?

  7. 7.

    Please, evaluate the importance of output types given below in terms of your field of study and PTA of the call you get support

  1. a.

    Scientific paper

  2. b.

    Presentation

  3. c.

    Book/book chapter

  4. d.

    Prize

  5. e.

    Patent application/registration

  6. f.

    Product/model

  7. g.

    Company

  8. h.

    Dissemination

  9. i.

    Thesis

  10. j.

    New projects

  11. k.

    Others

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gürbüz, K., Erdil, E. Prioritization and R&D Support Mechanisms: Turkish Case. J Knowl Econ 12, 962–991 (2021). https://doi.org/10.1007/s13132-020-00648-y

Download citation

Keywords

  • Impact assessment
  • Additionality
  • Resource allocation
  • Prioritization