1 Introduction

Globalisation, the constant transformation of the labour market, the process of massification of higher education, the rising costs of the educational process, coupled with the limited resources from public funds, as well as the increasing demands for accountability and the need to generate immediate, measurable and quality results, are just some of the elements that shape the national and international context in which higher education institutions (HEIs) are functioning (Herbst 2020, Herbst 2020). The interdependence between the current social, economic and political system and the tertiary education system is acknowledged especially in Western countries. A skilled workforce, as well as an accelerated process of knowledge and innovation, are necessary elements for economic progress that fall within the competence of universities (Jongbloed 2020). The higher education system is considered a public good (Marginson 2014), which in many cases is supported by the allocation of public funds in order to operate optimally and sustainably. The latter is evidenced by the fact that, at least in the case of European countries, “public funding represents between 50 and 90% of the universities’ income structure” (Pruvot, Claeys-Kulik and Estermann 2015, 155).

Given the significant share of the public funds in university funding, one aspect that becomes relevant to explore is how public funds are allocated among universities. The funding systems are quite diverse, being based on the characteristics of each country’s specific political and economic institutional systems but also on the needs identified at a national and international level (Jongbloed 2020).

In order to guide the actions and behaviour of universities to achieve their public objectives, policymakers use various steering methods, with the funding mechanism being one of the most important tools. Thus, in order to stimulate universities to act in a certain direction to achieve common nationally agreed objectives, practices such as regulations or financial restrictions imposed by governments have become the norm (Jongbloed 2008). At the European level, we can discuss about funding systems where the distribution of public financial resources is based either on the distribution of funds using formulas (applying input or output indicators) or by negotiation or contract with the state (Pruvot, Claeys-Kulik and Estermann 2015).

In the case of Romania, the institutional funding mechanism for higher education is a mixed one, mainly based on formulas, with two dimensions covering about 97% of the allocated funds: basic funding (BF), including the fund for doctoral grants, allocated according to the number of students, i.e. doctoral grants, by fields of study at university level (72%) and supplementary funding (SF) which is distributed according to the results obtained by universities on a series of performance indicators (26.5%), weighed with the financial-budgetary dimension of the university in terms of the number of budgeted students the university enrols. The main role of the SF is linked to boosting performance in several major areas: education, research, internationalisation, community engagement of the university.

The main objective of this paper is to analyse the progress of this scheme implementation in the five years since its adoption (2016–2020) and assess its impact. We also try to identify some possible challenges and risks and explore the extent to which certain characteristics of universities, such as their size or dominant field of study, have an impact on the distribution of the resulting funds.

The analysis makes use of the preliminary results of an ongoing study of the POCU 126766 project “Quality in higher education: internationalisation and databases for the development of Romanian education”.

2 Funding Systems in Higher Education and the Performance-Based Mechanisms

Jongbloed (2008) discusses funding mechanisms that involve the distribution of public funds according to the input recorded or the output achieved by universities (it is also possible to combine the two types of methodologies).

In the case of input-based funding schemes, the distributed funds are intended “to cover distinct costs such as staff salaries, material means, building maintenance costs, investment” (Jongbloed 2008, 3). Essentially, through these funds, the states provide universities with the basic resources needed to carry out their activities. In most cases, the amount of funding universities receive depends on the number of students enrolled, as well as their fields of study, and is distributed according to a formula. This type of funding mechanism is recommended because it ensures a transparent and predictable process for universities: you get money according to how many students you enrol. Such an approach also reflects, to some extent, the objectives assumed at the state level regarding the role of the university system, namely to increase the number of graduates (Teixeira et al. 2014, 224).

In terms of output funding schemes, universities’ “budgets depend on performance measures, and there is reason to believe that those who receive the budgets will pay more attention to their performance” (Jongbloed 2008, 3). For this type of system, funds are distributed according to a series of “teaching and research outcomes of the institutions’ activities” (Jongbloed 2008, 3). This category of indicators focuses on “progress to or completion of final outputs (e.g. study credits, number of degrees awarded, publications, competitive research funding awarded, citations, patents, amount of competitive/external research funding, student satisfaction)” (Kivisto and Kohtamaki 2016).

This type of mechanism is called performance-based funding, and its adoption indicates that the state has intended a distribution of limited resources to universities based on principles of competitiveness and performance (Orr 2005).

Performance-based funding for universities has become popular since the 1980s, especially in Western countries (Orr 2005; Sorlin 2007). The appeal of this mechanism is explained by the fact that it would imply a financial reward only for a certain type of behaviour and results (pre-determined by the funder), and implicitly a more efficient use of budgetary financial resources, because “the weakest institutions in the system are penalised” (Teixeira et al. 2014). However, [Orr 2005, 34] points out that “it is difficult to design funding methods, which accurately reflect the plethora of activities carried out in a university”. Thus, the process of identifying a set of standards/common indicators that provide comparable results across the higher education system is a complex one, based on trade-offs between policymakers and universities. This process includes determining the data set gathered, as well as how to calculate the indicator, which in turn is a challenging and potentially problematic dimension of indicator implementation. An example in this regard can be the criticism of the mechanism with which the JCR ranking of Web of Science indexed journals is carried out, an indicator used to measure the relevance and quality of research activity, including the case of Romania (Vîiu and Păunescu 2021).

As mentioned above, at the European level, there is a great diversity of funding systems and indicators used. An important contribution to this issue is the study carried out as part of the DEFINE project (European University Association) in 2015 on the funding mechanisms implemented by European countries.

The report systematically presents performance indicators as follows:

  • “Teaching: BA/MA degrees obtained; degree completion in the standard time of study

  • Research: Doctoral degrees/theses completed; research evaluation; successful patent applications; external research funding obtained; scientific activities; research contracts obtained; publications/citations; income from science and technology transfers; publishing researchers

  • Other: External funding obtained; EU/international funding obtained (can be linked to teaching and research); rankings outcomes” (Claeys-Kulik and Estermann 2015, p. 26)

While researchers recognise the important role that quality indicators play in the distribution of public funds, they have also identified a number of challenges and risks. One such risk is the ‘Matthew-effect’ which has the potential to create inequalities between universities. Given that output indicators take into account past performance, financial rewards for good performers will have the effect of increasing their performance, leading to a virtuous spiral. On the other hand, universities with lower performance, which receive less funding, will be less likely to develop positively in the future (given their limited resources), and they enter a spiral of mediocrity which leads to underfunding. This is more evident in the case of research indicators (Claeys-Kulik and Estermann 2015).

In addition to this ‘Matthew-effect’, another aspect to consider is that the way universities react and change their institutional behaviour and strategies may be influenced by variables such as ‘their reputation, their size, their disciplinary profile, their location or their mission orientation’ (Teixeira et al. 2014, 218), all of which are more difficult for universities to control in certain institutional contexts.

2.1 Funding System and Performance-Indicators in Romanias’ Higher Education

The transformations of the funding system are relevant to understanding the evolving objectives concerning the role of the higher education system but also relevant to learning how universities respond to the various incentives applied to achieve the established objectives. In the case of Romania, over the last three decades, the funding system has undergone important changes, the funding mechanism (the distribution of budget allocations to universities) representing the main strategic instrument for the operationalisation and implementation of the objectives assumed at a system level, especially in the last 20 years.

Following an initial period of expansion and structural reform of higher education after 1990, in which the funding system was essentially a result of the pre-‘89 funding system based on the level of personnel costs and administrative and teaching space, the beginning of the millennium was marked by the shift to global funding, which was a way to achieve a better performance in education and research through a better management of financial resources (CNFIS, 2007).

In addition, the introduction of per-student funding, starting with 1999, represented a major change for the distributing budget allocationsFootnote 1 to universities for their core activity. It was followed, since 2002, by the introduction of a differentiated funding, based on quality indicators, distributed at institutional level, with successive refinements (both in terms of indicators’ definition, but also in terms of the share they had in the total core funding, until 2012Footnote 2).

Since 2012, a new component has been introduced distinctly to provide additional funding to stimulate the excellence of institutions and study programmes (based on quality criteria and standards applied at study programme levelFootnote 3), which was essentially a more complex mechanism corresponding to the institutional quality component applied in the previous period. Even if the basic principles of the budget allocations distribution for this supplementary funding component (SF) have been maintained since then, there have nevertheless been two distinct periods of implementation: the 2012–2015 timeframe, in which the results of the ranking of study programmes were applied separately, by study cycle, and the 2016-present period, in which a specific, complex set of quality indicators was adopted and applied mainly at the level of the branch of science. The second period, between 2016 and 2020, is the subject of a detailed analysis,Footnote 4 with some of its main preliminary results being presented in this article.

Studies and analyses of the Romanian funding system, based on quality indicators, have not been few, especially in the last ten years. Particularly noteworthy are the studies and analyses that conducted an assessment of the funding policies operational at that time (and/or compared to the one that had been implemented previously), emphasising the ongoing challenges caused by the chronical underfunding of Romanian higher education system, the need for predictability in funding policies, and the need to improve and strengthen institutional mechanisms focused on quality assurance in the education and research process (CNFIS 2015, CNFIS 2016, 2017,CNFIS 2015,CNFIS 2019) (Miroiu and Vlasceanu 2012), (Miroiu et al. 2015) (Vîiu et al. 2016).

For the period 2003–2011, an important overview and general analysis of the results of the influences determined by the quality indicators application can be found in Vîiu (2015). The researcher mentions that the impact of this funding model is rather weak (at a global, summative level, for all nine years) and can be influenced both by the homogeneity of higher education institutions in Romania, as well as by a possible mutual cancellation of the influences determined by certain indicators (G. A. Vîiu 2015).

The significant changes that have taken place in the implementation of the performance-based funding model, through the supplementary funding subcomponent (SF), as well as the annual adjustments and refinements, are detailed in the annual public reports on the State of Higher Education and the Optimisation Measures Needed, produced by the National Council for Higher Education Funding (CNFIS).Footnote 5

Proposals to adapt, improve and add to the system of quality indicators are periodically formulated both within CNFIS and by the academic community while the Ministry of Education annually debates the funding allocation mechanism. During the period under review, supplementary funding was allocated on the basis of 15 quality indicators grouped into four classes,Footnote 6 described briefly below:

Class 1. Teaching/Learning—The indicators in this class aim to measure the dynamics of the educational process carried out in universities, the potential for development, as well as the preparedness of the human resources involved in the educational process. The following indicators were selected as being representative: the ratio of the students’ number to the teaching staff number (IC1. 1), the ratio of the master’s degree students to undergraduate students (IC1.2), the ratio of the teaching staff under 40 to the teaching staff (IC1.3), and the ratio of the Ph.D. supervisors to the teaching staff (IC1.4);

Class 2. Scientific Research/Artistic Creation/Performance in sports—The indicators have as their main objective the systematic and multidimensional assessment of the university performance in terms of scientific output and its impact on the quality of the teaching process, through the following indicators: IC2.1. The quality of human resources in universities, assessed in an overall sense through the CNATDCUFootnote 7 score of each faculty member; IC2.2. The impact of scientific activity/artistic creation, assessed through a scientometric index - the Hirsch index of the faculty members; IC2.3, the performance of scientific activity/artistic creation, which is selectively quantified by counting only the papers defined as most important and relevant in terms of international visibility; IC2.4, the funding for scientific research/artistic creation, which is an indicator aimed at measuring institutional success in attracting financial support for specific scientific research activities;

Class 3. Internationalisation Performance—Quality indicators associated with class C3. Internationalisation Performance are a direct measure of the universities’ capacity to support student mobility, as well as to sustain study programmes in international languages. Thus, indicator IC3.1 measures the share of student mobility, while indicator IC3.2 reflects the share of international students enrolled at universities;

Class 4. Regional focus & social equity—The quality indicators associated with class C4 focus on measuring activities such as integration of people from disadvantaged backgrounds (IC41), the university’s contribution to the scholarship fund (IC42); internship activity for undergraduate studies (IC43); capacity for provision of student accommodation (IC44); grant funds attracted by universities (IC44)”. (CNFIS 2017)

The formula operates by generating independent rankings of universities on each of the quality indicators, mostly by fields of study,Footnote 8 which are determined by the universities’ scores on these indicators. In fact, there are nine indicators operating at the field of study level (for each of the 40 fields of studyFootnote 9), as well as six indicators at the institutional level. One of the stated aims of the funding allocation mechanism was to make comparisons between similar university domains, and therefore, the option was made to consider the indicators at the level of field of study, where possible, an innovative approach compared to the overall institutional quality indicators that were used before 2011. From a methodological point of view, based on data reported by universities and checked by Executive Unit for Financing Higher Education, Research, Development and Innovation (UEFISCDI), a set of independent university rankings is generated for each indicator in each field of study, plus institutional rankings for each institutional indicator. It is important to note that the intention has not been to generate rankings, but only partial operational hierarchies for each indicator within each study field for the purpose of budget allocation. Thus, being contingent on a single indicator and a single field, the rankings have no significance beyond the purpose of allocating the funds corresponding to the share of the respective indicator in supplementary funding subcomponent.

Thus, the logic of this subcomponent is to differentiate the core funding based on the universities’ performance in terms of the indicators previously agreed on.

Globally, at the level of higher education budget, SF represents 26.5% of the institutional funding.Footnote 10 However, this funding is also distributed, the same as the basic funding (BF), per student, and it is thus not only a function of the university performance but also of its size in terms of the number of students. The formula also considers the different costs of study programmes per fields of study, so it employs the concept of unitary equivalent student to weigh the different study programmes and university degree level. A student in social sciences at bachelor level is equal to a unitary equivalent student, whereas a student at master level in social sciences equals two unitary equivalent students or a student at bachelor level in engineering equals 1.75 unitary equivalent students and so on. Thus, a neutral allocation would distribute the 26.5% equally to all institutions merely on the basis of the number of students in various programmes (weighted as unitary equivalent students). From this 26.5% allocation, a lower percentage is equivalent to a gradual loss up to a theoretical 0 if it is last ranked. Going above the 26.5% threshold means gradually gaining up to approximately double the basic funding equivalent, which reflects the university’s top position of the indicators by which it is ranked. The hypothetical range is between 0 and 53% (due to the formula that takes into account the number of students, the extremes are practically excluded).

However, the allocations are cumulative, being grouped into four categories of indicators, each with a different share, as it follows in Table 1.

Table 1 Share of indicator categories from the institutional funding
Full size table

According to the funding allocation mechanism, supplementary funding is a zero-sum game: for each indicator, for each field of study, the gains of the universities are reflected in the losses of the others. Thus, the allocations of supplementary funding are determined by the relative quality of a university based on an indicator, on a field of study, compared to the other universities in the same field of study and also taking into account their relative size in terms of the number of state-funded students.Footnote 11

3 Research Design

In this article, we will present the allocation of financial resources for supplementary funding, received by each university, as a share of institutional funding (IF), both by indicator classes and university categories. As we are referring to percentages of financial allocation (and not amounts) for each indicator class, the data presented are comparable between universities and not affected by their size.

Indicators are calculated by CNFIS in order to distribute financial allocations for additional funding. The results are obtained from the data reported annually or bi-annually by the state universities, according to the funding allocation mechanism for state universities in use.

Although allocations are made by fields of study, with each being allocated an amount proportional to the number of state-funded students enrolled, we believe that, for a better global understanding of the dynamics of budget allocations,Footnote 12 it is appropriate to present aggregate data per category of universities and class of indicators. The method is particularly useful to compare the rankings of universities with the same general profile and to observe whether there are notable differences between university categories. The classification is based on the university’s main fields of study (even if most universities also have educational programs in other fields of study/domains). For example, we have considered a university as ‘comprehensive’, taking into account the diversity of study programmes offered by the university, or as ‘specialised’ such as medical or technical, based on its dominant field of study. Thus, in presenting the data, a classification of 47 Romanian state universities financed by the Ministry of EducationFootnote 13 under six main categories was used: agronomic-veterinary, medical, technical, comprehensive, social & economic sciences and arts. It is important to mention that these categories are used only to allow an easier presentation and understanding of the data, but they did not play any role in the funding allocation process, which only followed relevant data at a field of study and institutional level. At the same time, in the analysis, we also took into account the size of universities according to the total number of students enrolled at Bachelor, Master and Ph.D. level. Therefore, we have decided to split the 47 universities into three categories,Footnote 14 considering the concentration of students as follows: large (with more than 19,000 students), medium (between 7,000 and 19,000 students) and small universities (with less than 7,000 students).

3.1 Analysis

In order to highlight the dynamics of the financial allocations across the four indicator classes, in the table below, we present some descriptive data and the central trend, i.e. minimum, maximum, median and standard deviation percentages for each indicator class over the five years of implementation (presented in Tables 2 and 3). Also, the column “Formula” in Table 2 shows the theoretical minima, maxima, and median that can be achieved through the overall performance-based formula application, when compared with the core funding formula results, based on students number. Obviously, the Romanian higher education institutions are quite diverse as regards their size in terms of number of enrolled students. The size of the competing universities within a field of study (or at the institutional level for some indicators) affects the granularity or the increments of percentage variation from one university to another for each class of indicators, as well as for the overall allocation; thus, the variation in percentages is not uniform, nor similar from one ranking to another, the increment varying according to the size of the universities compared, more precisely the number of their students. However, the median calculation relates to the value that includes 50% of the universities in one field of study, regardless of their size. Thus, a lower median than the theoretical one (which is the neutral level of the respective indicator weighed within the total supplementary funding of 26.5%) reveals that, in general, large universities (relative to the number of students) have higher indicator values than small universities. This is particularly noticeable in the case of Class 2, the research performance indicators, where the median is considerably lower than the theoretical median for each year. For Classes 1 and 3, the actual medians are similar to the theoretical ones indicating a size-independent distribution, while Class 4 also shows slightly lower annual medians than the theoretical one.

Table 2 The descriptive data and central trend for each class of indicators
Full size table

At the same time, the standard deviation calculated overall, between or within universities (across the five-year period), indicates that universities tend to have stable results over time (the standard deviation being lower within universities), but there are higher standard deviation scores between universities, which may indicate that these indicators (regardless of class) tend to differentiate more between universities rather than within universitiesFootnote 15 across time, as shown in Table 3. Looking at the level of indicator classes, it can be seen that the differentiation between universities is more significant at classes 2–4 than at class 1, as can also be concluded from Table 2.

Table 3 Standard deviation for each class of indicators (2016–2020)
Full size table

In addition, to identify an annual dynamic of the financial allocations received by universities, we used a dichotomous variable for each year, with the following values: value 1, if the university obtained more than 26.5% for SF in FI, and value 0 if they received less than 26.5%. The tables below show (per total and separately, by the two main categories of universities: according to the main field of study or domain, and size of the university) what is the share of universities that from year “T” (regardless of year) to year “T+1” (next year) are likely to win (1) or lose (0). Therefore, per total, approx. 91% of universities that lost in the year “T” tend to lose in the following year, and only approx. 8% of universities that lost in the year “T” tend to gain more than 26.5% in the next year. Regarding the categories of universities (by the main field of study), the lowest dynamics are observed among Comprehensive and Technical universities, while the highest dynamics (a higher share of universities tending to gain in the next year) are observed among Agronomic-veterinary and Humanities and Social Science universities. Taking into account the other category, based on the size of the university, we observe a low dynamic of annual gains among medium and small universities and a relatively higher dynamics among the large universities. Also, the large universities are in the winning positions for approx. 75% of the occasions, while the medium and small universities only 37% and respectively 32% of the occasions are net winners (Table 4).

Table 4 The dynamics of gains/losses in the share of SF in IF
Full size table

By analysing the results at the level of universities and types of universities, the graphs below show the results-based allocations of financial resources to quality indicators as a share of institutional funding (IF) for all 47 state universities in Romania for a period of five years, starting with 2016 (which was the first year of implementation of these indicators). Universities are divided a) into six categories based on their main fields of study and b) in three categories considering their student population, as mentioned above.Footnote 16 The data are presented at the overall level of application of the SF (Figs. 1 and 2), as well as at the level of indicator classes (Figs. 3, 4, 5, 6, 7, 8, 9 and 10). At the same time, by discussing financial allocation percentages, it is easier to see the “gains/losses” achieved by universities and their evolution over the period analysed.

According to Fig. 1, which shows the overall allocations of the SF (cumulated across all four classes of indicators), one can observe a relatively weak dynamic in terms of the annual results obtained by universities, and higher variations from year to year are found in the case of a small number of universities, most of which obtain similar annual values (they do not change their earnings significantly as a share of the IF).

Fig. 1
figure 1

% of SF from institutional funding, by category of universities (2016–2020)

Full size image

An important point to note is that there are 11 institutions (from all university categories, the majority of them being located in the major Romanian university centres such as Iasi, Bucharest or Cluj-Napoca) that maintain a competitive advantage each year, managing to obtain allocations higher than 26.5% of the IF (over the five-year period). From this group of universities, technical, socio-humanities or architecture/art/sports universities stand out, which implicitly also indicates high results on performance indicators, at least in relation to other universities enrolling students in the same fields of science. At the same time, of the universities that fail to exceed the 26.5% threshold in all of the five years, we find a significant number of higher education institutions, most of which are in the comprehensive category. A noteworthy observation concerning these universities is that they do not manage to approach the 26.5% threshold, in practice obtaining lower financial allocations than they would have obtained in a performance neutral “per capita” only mechanism of allocation.

It can also be noted that variations are higher at smaller universities, while larger universities tend to have more stable “winning” allocations (Fig. 2).

As a general conclusion, we can state that more specialised universities, with fewer fields of study tend to be more competitive than comprehensive universities. The latter compete in more study fields, but with lower overall results. Also, the size of universities (especially if we refer to large universities compared to the rest) tends to be an important characteristic that can indicate their winning/losing position (as can be seen in Fig. 2).

Fig. 2
figure 2

% of SF from institutional funding, by university size (2016–2020)

Full size image

As regards the distribution of funds on the basis of Class 1—Teaching/Learning indicators (shown in Fig. 3), we note that there are no significant changes at the university level in terms of the percentages obtained from IF across the five years. Actually, in this class more than any, the rankings of universities do not register any important variation from one year to another. This is explainable as this class includes, more than any other, indicators that refer to the institutional capacity, such as staffing indicators, as well as other human resources or students indicators (share of teaching staff under 40 years of age or of Ph.D. coordinators). These indicators have greater stability and do not significantly change from one year to another, so as to be reflected in the quality indicatorsFootnote 17 and thus alter the rankings. Among the universities that manage to obtain annual allocations higher than the weight of this class (16 in total), institutions in the arts category stand out (more than half of them—5—are constant in terms of their share in this class of indicatorsFootnote 18). In the case of Class 1, the size of universities (Fig. 4) does not seem to be relevant in explaining the dynamics of the results obtained by universities.

Fig. 3
figure 3

% of C1—SF from institutional funding, by categories of universities (2016–2020)

Full size image
Fig. 4
figure 4

% of C1—SF from institutional funding, by university size (2016–2020)

Full size image

The financial allocations to Class 2 Scientific Research/Artistic Creation/ Performance in sports (shown in Fig. 5) seem to indicate the most marked differences,Footnote 19 both between and within universities, over the period analysed. Compared to the results obtained by universities in Class 1, these indicators show a smaller number of universities (11) that manage to maintain their competitive advantage over the five years. More specialised higher education institutions perform significantly better in attracting funding for indicators measuring the scholarly activity of university teaching staff compared with comprehensive universities. Thus, for this important set of indicators, comprehensive universities appear to be less competitive than more specialised universities (whether agricultural, technical, medical, socio-humanities or arts). Also, large universities tend to benefit more from this class of indicators, as they obtain values above the neutral allocation almost every year (as can be seen in Fig. 6).

Fig. 5
figure 5

% of C2—SF from institutional funding, by categories of universities (2016–2020)

Full size image
Fig. 6
figure 6

% of C2—SF from institutional funding, by university size (2016–2020)

Full size image

The set of indicators measuring the degree of internationalisation of universities (Class 3—International Performance, Fig. 7) have a positive impact on a larger number of universities, including comprehensive or art universities (although there are important differences between universities within this class). At the level of universities grouped by their size, there are no significant developments in this class of indicators during the reference period, the results being rather stable. (as shown in Fig. 8).

Fig. 7
figure 7

% of C3—SF from institutional funding, by categories of universities (2016–2020)

Full size image
Fig. 8
figure 8

% of C3—SF from institutional funding, by university size (2016–2020)

Full size image

Universities largely benefiting from indicators focused on measuring regional orientation and community engagement (Class 4, Fig. 9) are those with technical and agronomic-veterinary profiles. This class of indicators has minor budget allocation effects for comprehensive universities, as well as architecture, arts and sports universities. In the case of the latter, in addition to the fact that they did not perform good/well enough to attract additional financial resources (with very few exceptions) during the period under review, significant annual variations were also observed. Also, taking into account the size of universities (Fig. 10), there are no significant changes at the university level during the reference period, and small universities tend to lose quite a lot from the available allocations almost every year.

Fig. 9
figure 9

% of C4—SF from institutional funding, by categories of universities (2016–2020)

Full size image
Fig. 10
figure 10

% of C4—SF from institutional funding, by university size (2016–2020)

Full size image

4 Discussion and Conclusion

There is clearly a differentiation between the universities’ budget allocations, in terms of percentages related to each class of indicators (IF), as well as overall. This result suggests that the implementation of performance indicators leads to a differentiation between higher education institutions according to their relative performance, even if, in some cases, this is based on minor differences in the quality indicators. This is also a consequence of the formula that actually conceals the real differences in quality indicators by using a simple hierarchical ordering of the universities for each respective indicator. The allocations are then a function where the number of enrolled students in the competing universities is also an important variable.

The longitudinal distribution of percentages for the quality indicators also shows important stability of results over time (as in the case of Class 1), most of them being dependent on more structural aspects, such as the number of teachers or the volume of facilities, which are related to institutional capacity and are not only more difficult to change, but some are not fully within the universities’ decision (as the state investments for instance). Also, due to regulations, especially those related to quality assurance (that do impose specific standards related to student/teacher ratio, for instance), the differences between universities, especially within the same field of study, could not be very large and are rather stable.

Indicator Class 2. Scientific Research/Artistic Creation/Scientific Performance seems to discriminate stronger between universities in terms of their research performance. Not only is it the most competitive indicator class, but it also has the highest weight among performance indicators and is therefore responsible to a greater extent for the differences in financial allocations between universities. For this class of indicators, the variance of the allocation percentages between universities is greatest.

Furthermore, particularly within the Class 2 of indicators, the median is below the theoretical value, which indicates that larger universities tend to be more competitive.Footnote 20 Although, in general, the size of the university (in the sense of the number of students enrolled in a field of study) does not seem to have an impact on financial allocations in indicator classes 1 and 3, the size of the university appears to be an important factor in indicator class 2 and partly in indicator class 4. Taking into account that research indicators (class 2) discriminate the strongest, we can say that, eventually, university size has impact on the overall financial allocations.

In addition to aspects related to the size of the university, the profile of the university can also prove to be relevant, strongly highlighting certain categories of universities in a positive sense (the example of class 4 for technical and agronomic-veterinary universities) or in a negative sense, if we analyse the situation of comprehensive universities, which seem to be the most disadvantaged by the implementation of these indicators. Comprehensive universities systematically have poorer relative results for all classes of indicators than more specialised ones (one of the possible explanations for this could be the ‘Matthew-effect’, universities that consistently underperform are less likely to break out of this circle in the future). Also, relatively younger universities compared to more established ones have a disadvantage when it comes to endowments and capacity in general, and this is also reflected in the indicators from Classes 1 and 4. In general, comprehensive universities are relatively newer compared, for example, to technical or medical schools. However, it should be considered that these are aggregated results for all fields of study, making it more difficult for a comprehensive university to be equally competitive across all fields of study compared to more specialised universities.

Nevertheless, a detailed longitudinal analysis of fields of science level and quality indicators is needed in order to highlight the competitive advantages of universities in certain fields and on certain indicators, but also whether there are particular trends at university level, both in terms of fields of science (thematic specialisation) and indicators (specialisation in terms of types of services offered). At the same time, such a detailed analysis would indicate the extent to which there might be mutually cancelling influences driven by certain indicators (as observed by Vîiu (2015) for the 2003–2011 allocation period), with the university “gaining” on certain indicators in a specific class, but at the same time “losing” on the total class.

Also, it should be taken into account that the implementation period is relatively short, as universities need more time and stability to “react” to these incentives. Due to how indicators are calculated and to their changing weights over time, a longitudinal analysis proves difficult. Also, the incentives that universities had also changed, which makes it difficult to analyse trends at a system level. There is certainly a need for more stability in the application of indicators and for a more detailed impact analysis to reveal more subtle trends in the response of the higher education system to this type of funding policy. Our analysis reveals, based on the available data, that the degree of university specialisation (comprehensive or more specialised) and the relative size of universities within a field of study have an impact on performance-based funding allocations.

At the same time, one of the recent studies regarding the effect of performance-based funding in two states of United States, Ohio and Tennessee, mentions that even if “there is clear financial incentive to improve outcomes in response to these policies, it is worth considering theoretical reasons why outcomes may not improve” (Ward and Ost 2021). And their arguments are related to the effect of the principal–agent model that motivates the performance-based funding (when the state and university have very different objectives, these incentives should alter university behaviour, and when the universities share the same objectives as the state, then theoretically the incentives will not have their intended effects) or to the university capacity to (re)allocate resources for improving their outcomes.

The policies pursued by the performance-based funding must be seen in a broader context and cannot be isolated from other measures applied at a national level (e.g. some of the measures to increase equity and access to university education are part of the National Strategy for Tertiary Education and are tracked not only through the PBF model, but also through FDI or other national strategic programmes). Future research will also consider qualitative analysis in the form of institutional feedback. The perception and institutional responses to performance-based funding model are important and may not be obvious where the university follows the implementation of national strategic measures anyway, without additional financial incentives or with some internal constraints.

Annex

Categories of universities

N

Universities size

N

Agronomic-veterinary

4

Large

6

Arts, Sports

9

Medium

17

Comprehensive

16

Small

24

Medicine

6

  

Humanities and Social Science

6

  

Technic

6

  

Total

47

Note: N is the number of universities