Introduction: Higher Education System Governance in Transition

The allocation of public funding to higher education has been increasingly subject to debates and change in recent decades. The changes have often been linked to changing beliefs and conceptions about how the public sector should be steered and managed. The backdrop to this was the New Public Management (NPM) approach to governing public organizations (Ferlie et al. 1996) which argues that the public sector should be addressed with similar management tools as the private sector.

Under NPM, the predominant steering approach in European higher education systems has emphasized decentralization, with higher education institutions (HEIs) enjoying a large autonomy and receiving a lump sum budget from their funding authorities. To a large extent, HEIs are autonomous in areas such as the provision of educational programmes, managing their research portfolio, their human resources and their asset and property portfolio. This governance approach may be characterized as “state supervision steering” (Van Vught 1989). The government limits itself to a restricted number of “framework steering” elements: setting the tuition fees and distributing student financial support; organizing quality assurance of education and research and determining whether new education providers and new degree programmes qualify for public funding.

Barr and Crawford (2005) argue that “the days of central planning have gone”, stating that a mass higher education system requires differentiation and a greater reliance on markets. In their view, the government’s role is to act as a facilitator. The idea is that more institutional autonomy will produce higher levels of quality, diversity and efficiency because a more diverse set of HEIs will better respond to student demands and societal needs. The competition and search for prestige by HEIs will produce better educational and research performance and more distinct educational and research profiles. However, the question is whether more autonomy combined with more market-based approaches will indeed produce more diversity and better performance.

In light of the latter question, concern has been expressed about this governance model that stresses autonomy and decentralisation. Criticism was targeted at how quality assurance and accreditation were shaped, with some arguing that accreditation was too inward-looking, carrying too few incentives to enhance quality and achieve excellence. In addition, there were concerns about HEIs becoming increasingly identical in their race for academic reputation. And yet, others were pointing at tendencies such as fragmentation, duplication and diseconomies of scale resulting from marketization.

The attention for the less-desirable effects of marketization has contributed to the emergence of new forms of accountability and new interventionist policies. An example is the reshaping of accreditation and quality assurance mechanisms, directing them more towards students’ achieved learning outcomes. A related example is the introduction of information tools that try to make higher education more transparent. A third example—very much in line with the NPM approach—is to influence HEIs’ behaviour by concluding contracts between the public authority and each HEI to guarantee that the services expected from the HEI and their quality will be delivered. In another contribution to this volume (Jongbloed et al. 2018) these examples are discussed from the point of view of transparency and accountability in higher education.

In this chapter, we will analyse performance contracts as a way to combine institutional autonomy with new forms of steering and accountability. Performance contracts, and its related concept of performance-based funding imply a new approach to steering, with a contract model replacing state supervision. The contracts are “individualised” agreements, embedded in a clear accountability context, that allow governments to steer on specific societal targets. This may be understood as the next stage in NPM. Performance-based funding (PBF) is a frequently cited example of a new regulatory policy instrument. While some may interpret it as governments moving away from input-steering and preventing intrusion into the HEIs’ internal affairs, PBF schemes may also be seen as a means to allow governments to force universities into certain desired directions. This raises questions about the impact of PBF and performance contracts on institutional behaviour. Does performance steering matter for the performance of a national higher education system? Will it stimulate HEIs towards behaviour that is better aligned with national goals? And do performance-based approaches have any unintended effects, e.g. a return to forms of bureaucratic oversight?

In the next section, we will present some characteristics of PBF systems in several OECD countries. We then (in Sect. 3) move on to the Netherlands where an experiment with performance agreements was concluded recently. The outcomes of the Dutch performance agreements in terms of their impact on performance and diversity are discussed in Sects. 4 and 5. Section 6 presents some lessons that can be drawn from the Dutch experiment and formulates some overall conclusions on performance agreements.

Funding Mechanisms in Higher Education: The Move Towards Performance Agreements

Models for public funding of HEIs vary across countries/jurisdictions. Most countries employ funding formulas that link the core (recurrent) grant that an HEI receives from its funding authority (a ministry or funding council) to input indicators such as student enrolments (Jongbloed and Vossensteyn 2016). In recent years, many countries have introduced measures of performance in the funding arrangements. PBF was introduced in the belief that it would steer HEIs’ behaviour towards producing higher levels of performance, quality and efficiency. In a recent overview of performance indicators included in funding formulas of OECD countries (see De Boer et al. 2015), it is illustrated what the activities are where governments want HEIs to perform better on:

  • Number of Bachelor and Master degrees: Austria, Denmark, Finland, Netherlands, Germany, United States (e.g. Tennessee)

  • Number of exams passed or credits earned by students: Austria, Denmark, Finland, US (e.g. Tennessee, Louisiana, South Carolina)

  • Number of students from underrepresented groups: Australia, Ireland, Germany, US (e.g. Tennessee)

  • Study duration: Austria, Denmark, the Netherlands, US (Tennessee)

  • Number of doctoral degrees: Australia, Denmark, Finland, Germany, Netherlands

  • Research output (e.g. research quality, impact, productivity): Australia, Denmark, Finland, Hong Kong, United Kingdom

  • Research council grants won: Australia, Finland, Germany, Hong Kong, Ireland, Scotland, US (e.g. Tennessee)

  • External income (i.e. non-core revenues): Australia, Denmark, Finland, Germany, Hong Kong

  • Revenues from knowledge transfer: Australia, Austria, Scotland

Obviously, what exactly is understood as performance varies across higher education systems, as well as between subsectors of the higher education system (e.g. research universities versus universities of applied sciences), depending on the challenges and ambitions of the country.

What are the characteristics of performance contracts (or performance agreements)? For one thing, they are ex-ante funding. Formula-based funding arrangements are backward looking, with indicators in the formula referring to the recent past (ex-post funding). In performance contracts, funds are based on a bilateral agreement between the funding authority and the HEI that includes performances that an institution promises to deliver in the (near) future and the budget that the HEI will receive in return for this. In this case, the HEI’s budget is (partly) based on a specification of its goals for the future (ex-ante funding).

In performance contracts (or performance agreements), each HEI is invited by the funding authorities to specify its ambitions. The agreement usually includes a financial penalty or sanction of some sort if objectives are not achieved.

A performance contract seeks to redress the one-size-fits-all nature of formula-based funding that rewards all HEIs on the basis of the same formula and the same indicators. With performance agreements, there is more room for HEIs to have additional aspects of their performance reflected and connected to financial rewards. Performance agreements can handle situations where HEIs have multiple objectives and—within nationally-set boundaries—can set their own target levels, given their particular mission and strengths. A funding agency or independent committee usually oversee the drawing up of the agreements to guarantee that agreements are in line with national objectives and monitor progress during the contract period.

Table 1 shows the characteristics of the performance agreements in place in several countries.

Table 1 Performance agreements across the World
Full size table

The performance agreement in some countries is not always directly linked to (a separate portion of) the budget (see middle column). For example, in Australia, entering into a performance contract—a compact—is one of the quality and accountability requirements that a university must meet as a condition for receiving a grant.

The middle column of the table illustrates that performance agreements are not only meant to strengthen performance but also have aims like encouraging HEIs to strategically position themselves (institutional profiling), improving the strategic dialogue between the government and HEIs, or informing policy-makers and the public at large about HEIs’ performance, thus improving accountability and transparency.

The share of the HEIs’ public recurrent grant that is based on performance (see Table 1) is difficult to determine exactly because both the funding agreement and the formula often mix input and output elements. For instance, in the Netherlands, the performance agreements constitute on average 7% of a university’s teaching grant, whereas 20% of the (separate) formula-based teaching allocation is based on degrees, and another 40% of the (separate) research allocation is also based on degrees. Thus, on average, a quarter (for universities) to a third (for universities of applied sciences) of funds is based on performance measures.

The benefits of a diversified higher education system are well-recognised, and performance agreements are expected to help achieve this goal in, e.g., Austria, Ireland, Germany, Finland and the Netherlands. The broader set of objectives and indicators facilitated by the performance agreements are expected to promote institutional diversity. Performance agreements may prevent one of the risks of formula funding, namely that all HEIs respond to the formula’s indicators in the same way, which would result in more homogeneity instead of more diversity in the system (Codling and Meek 2006; Van Vught 2008).

Performance Agreements in the Netherlands

The Netherlands has a binary higher education system, which means there are two types of programmes: research-oriented education, traditionally offered by research universities, and professional higher education, offered by universities of applied sciences (UASs). University and UAS programmes differ not only in focus but also in access requirements, length and degree nomenclature.

There are eighteen research universities in the Netherlands, including one Open University, and 36 UASs. The UASs have a more regional function and focus in particular on their education mission, although in recent years they also began to strengthen their practice-based research, partly thanks to dedicated public funds for research and research-oriented staff positions.

In 2009, an independent expert committee (the Committee on the Future Sustainability of the Higher Education System, also known as the Veerman Committee) was tasked by the Minister of Education to advise about system performance and diversity in Dutch higher education. The Veerman Committee concluded that, overall, the quality of the education provided is in good order and that the binary structure should be maintained. However, at the same time, it perceived many risks for the future. It called for a threefold differentiation in higher education (Veerman et al. 2010):

  1. 1

    differentiation in institutional types (meaning: universities versus UASs)

  2. 2

    differentiation among institutions of the same type (i.e. institutions distinguishing their own profile)

  3. 3

    differentiation in the range of programmes offered.

The third dimension points at the need for a more tailored approach and smooth transfers between programmes at different levels, to enable students to successfully complete their studies. The Committee stated that student dropout was too high and completion too low. It also opined that some students’ talents were not properly addressed and there was too little flexibility in the system to serve the various needs of students and the labour market. Especially the large difference between the success rates of “ethnic minority” and “native Dutch” students was considered to be problematic.

Increasing diversity was regarded as an important part of the solution and, partly as a result of the recommendations of the Veerman Committee, performance agreements were introduced in 2012. The agreements were signed between the Education Ministry and each individual HEI and formulated in terms of quantitative indicators and qualitative ambitions, with ambition levels chosen by the institutions themselves. The agreements intended to encourage and reward performance in terms of the following goals:

  • Improving the quality of education and the success rate of students in universities and universities of applied sciences;

  • Enhancing differentiation within and between HEIs, encouraging them to exhibit distinct education profiles and more focused research areas (including the creation of Centres of Expertise by UASs);

  • Strengthening the valorisation function in universities and UASs (i.e. knowledge dissemination, commercialization, promoting entrepreneurship).

For the period 2012–2016, 7% of the core grant for education (some €130 million for the research universities and €170 million for the UAS sector) was tied to performance agreements. The remainder of the core grant of HEIs continued to be based primarily on a funding formula that, already since the early 1990s, included a significant performance orientation. An independent Review Committee was installed by the Minister of Education in 2011 to oversee the performance agreements, to develop criteria for assessing them, to monitor each institution’s progress in realizing its ambitions during the contract period, and, at the end of the period (i.e. in 2016), to make a recommendation to the Minister about whether the goals in the agreement had been met. If an HEI did not achieve its agreed goals, it risked losing part of the core grant for subsequent years. The performance agreements arrangement was set up as a policy experiment. Depending on an external evaluation, performance agreements could be continued (perhaps after some adaptations) and be included in the legal arrangements determining the funding of HEIs.

Enhancing Study Success Through Performance Agreements

At the start of the performance agreements process, the HEIs jointly agreed to use seven mandatory indicators measuring their ambitions with respect to student success and educational quality. Two indicators, completion rates and dropout rates, received the most attention in the annual monitoring and, eventually, in the final conclusion of the performance agreements. Ambitions with respect to differentiation and institutional profiling were stated in more qualitative terms, relating to topics such as starting new degree programmes and phasing out old ones, introducing student mentoring programmes, setting up research centres, engaging in partnerships with local business, et cetera.

In 2016, the agreements came to an end. The Review Committee assessed each institution’s performance in the light of its performance agreement, based on information presented to the Review Committee through the institutions’ annual reports and meetings with the institutional governing boards.

The Review Committee published a summary of the results of the performance agreements in its 2016 Annual Monitoring Report (Review Committee 2017a). The Committee concluded that many research universities had achieved substantial success in reducing dropout and increasing completion rates. The average completion rates in universities had risen from 60 to 74%, and dropout rates declined from 17 to 15%. In professional higher education (i.e. the UASs), major efforts had been undertaken to achieve the targets set in the performance agreements; nonetheless, some of the UASs had failed in their attempts to improve completion. The average completion rate in UASs fell from 70 to 67%. However, dropout was pushed back slightly, from 27 to 25.6%.

The disappointing results in the UAS sector regarding student completion can be attributed partly to the trade-offs between access, quality, and completion. These trade-offs manifested themselves in particular in the large UASs that have a highly diverse student population. Being confronted with the accreditation agency’s requirement to set more stringent academic standards, the UASs saw themselves facing more challenges than they had expected at the start of the performance agreements experiment. At the conclusion of the experiment, many UASs argued that they had prioritised quality over completion and, in addition, felt a need to stick to their institutional mission of facilitating access, also to students who, from an academic point of view, may be less academically prepared than other students.

The Review Committee had to advise the Minister that six UASs had not fully achieved their targets, despite all their efforts. The Minister decided to impose a financial penalty on these six institutions. However, she decided to apply only half the envisaged penalty in appreciation of their efforts.

In Fig. 1 (for the 13 research universities we have data for) and Fig. 2 (for 36 UASs), we show the results for two key performance indicators: completion rate (on the vertical axis) and dropout rate (on the horizontal axis). Definitions are as follows:

Fig. 1
figure 1

Completion rate and dropout; initial situation (year 2011) and realisation (year 2015); research universities

Full size image
Fig. 2
figure 2

Completion rate and dropout; initial situation (year 2011) and realisation (year 2015); universities of applied sciences

Full size image

  • completion rate: the proportion of full-time Bachelor’s students who, after the first year of study, re-enrol at the same HEI and earn a Bachelor’s degree at that same HEI in the standard time to degree plus one year;

  • dropout rate: the proportion of the total number of full-time Bachelor’s students (only first-year HE students) who, after their first year, are no longer enrolled in the same HEI.

During 2012–2016, most universities (Fig. 1) achieved higher completion rates and lower dropout rates, though some showed higher completion with slightly higher dropout. In the UAS sector (Fig. 2), the dominant movement is towards lower completion and higher dropout. The opposite movement also occurs but only for a few UASs. To show the scores of individual HEIs better, we used colour codes for various types of HEIs. Type is primarily determined by the student cohort size of new entrants and the scope of programmes offered.

The average completion rates in universities have risen considerably. The sharpest rise can be observed among the technical research universities (green; see Fig. 1): from on average of 42% to 68% in 2015. At most research universities, the 2015 completion rates equalled or exceeded the ambition set for 2015. At two of the four universities that fell short, the 2015 completion rates were close to the target values.

In the UAS sector, the average completion rate fell from approximately 70% to 67%. Large differences between the UASs can be observed in this respect (cf. Fig. 2). For example, among the large UASs in the largest cities of the Netherlands (the blue arrows in Fig. 2), the average completion rates over 2015 (55%) were not only substantially lower vis-à-vis 2012 (63%) but also in many cases showed a persistent downward trend, which only recently had appeared to take a turn for the better. At thirteen UASs, completion rates in 2015 were significantly lower than in 2011 whereas they had formulated ambitions to reach higher rates.

Around the conclusion of the performance agreements (during the second half of 2016), most attention in the popular press and among relevant stakeholders in higher education was given to the indicators shown in Figs. 1 and 2. This was only natural because the financial sanctions attached to the performance agreements were very much tied to whether an institution had met its agreed ambition levels on the seven mandatory indicators, among which completion and dropout proved most challenging.

In particular, challenges were big for UASs that wished to serve a large variety of students. Compared to universities, UAS institutions have a much more challenging and heterogeneous student population, with many “ethnic minority” students or students holding a vocational education and training (VET) degree. For UASs in the large cities, the proportion of incoming “ethnic minority” students averages 26%.

Encouraging Diversity in Dutch Higher Education Through Performance Agreements

The performance agreements aimed to create a higher education system “fit for the future”, offering increased quality and diversity. Recognising that a one-size-fits-all approach tends to create uniform reactions, performance agreements were seen as the way to create more diversity. Diversity is a prominent theme in higher education (Birnbaum 1983; Marginson 2017) and science and technology policy (Nowotny et al. 2001). It is seen as a means to enhancing rigour and creativity, offering flexibility in the face of uncertain future progress and promoting learning across programmes (Stirling 2007).

While five-sevenths of the budget tied to the performance agreements depended on seven mandatory indicators, the remainder (two-sevenths) was awarded to HEIs in the form of competitive funds. This was known as the selective budget and it was awarded in proportion to the quality of an HEI’s plans for programme differentiation and research concentration. HEIs that submitted the best plans according to the Review Committee received relatively more than other HEIs.

The concept of diversity is difficult to grasp because diversity is naturally multidimensional and its nature is essentially qualitative—making it subjective and susceptible for criticism. In its annual monitoring reports and its assessment of the individual performance agreements, the Review Committee operationalised diversity through the range of programmes offered by individual HEIs (e.g. two-year Associate degrees, Bachelor’s degrees, Master’s degrees, broad-based bachelor programmes, two-year research master’s, selective honours programmes, professional Master’s programmes offered by UASs) and by the disciplines covered by their research.

To assess aspects of diversity, the Review Committee analysed three partly overlapping features of an HEI’s profile: (1) the range of programmes offered by an HEI, to see whether it is broadening the scope of its programmes and covering more or fewer disciplinary areas, (2) whether an HEI focuses on particular programmes within its programme range, and (3) the market share of the programmes provided by the HEI. The Review Committee looked at a number of diversity indicators to analyse (de-)differentiation over the period of the performance agreements. Sophisticated diversity measures were computed (see Review Committee 2017a for more on this topic), including the Herfindahl index—often used to measure concentration in a market on the basis of the institution’s market share. Another diversity indicator is the Gini inequality index, indicating the balance of an institution’s student intake across its programmes.

In this chapter, we must limit ourselves to the question whether HEIs differ in terms of their emphasis on particular disciplinary areas—both in their education and in their research activity (see Review Committee 2017a for analyses of other aspects). To analyse this, the Review Committee used information on student intake across the institution’s respective degree programmes, respectively the dispersion of research publications across the 250 sub-disciplines that can be distinguished in the world of science.

The Review Committee noted that, from 2006 onwards, student intake in the majority of research universities became increasingly even across the Bachelor’s programmes on offer and that this trend largely continued during the period of the performance agreements (Review Committee 2017a). This was even stronger the case among Master’s programmes. The Review Committee interpreted this as a tendency towards less focus in education. Also in the UAS sector, over time, the distribution of students across Bachelor’s programmes became more even, showing no indication of a focus on particular education areas. However, in Master’s programmes in the UAS sector, more institutions appeared to develop more clearly visible focus areas during the period of the performance agreements.

On the research side of higher education institutions, the performance agreements aimed at the promotion of focus areas (research concentration is the English term; “focus and mass” is the Dutch term). Since research in the UAS sector is still a rather small part of UASs’ activities, robust judgements can only be made for the research universities. To achieve the goal of research concentration, the performance agreements included many initiatives related to (inter-institutional and intra-institutional) collaboration, selective professorial appointments and internal reallocations of research capacity and resources. It obviously takes time for such initiatives to be reflected in research publications or other research outputs. Nevertheless, the Review Committee has sought to monitor the development of focus areas in research by analysing the distribution of the scientific publications per university across the various sub-disciplines; an unbalanced distribution across disciplines indicating research concentration.

The Gini index that was used for this analysis indicated that over time, research publications were spread more equally across disciplines. This proved to be the case, in particular, for the four technical universities. The Review Committee interpreted this as a decline in the number of focus areas in universities and as showing a decline in the diversity of the Dutch university research system. For each university, the committee witnessed an increase in the number of sub-disciplines covered by the research publications. The differences between the various categories of universities (e.g. comprehensive versus technical universities) appeared to diminish. This suggests a decrease in differences between university research profiles. However, the Review Committee found it difficult to attribute these tendencies to the performance agreements alone as they already started to manifest themselves much earlier.

Initiatives by HEIs to enhance differentiation and diversity cannot be evaluated only in terms of programme offerings or research publications. Therefore, the Review Committee also analysed non-quantitative differentiation initiatives. Based on its conversations with HEIs and a study of their annual reports, the Committee concluded that HEIs had undertaken substantial efforts for institutional profiling in the areas of education, research, and knowledge valorisation, but that their impact was not yet visible in terms of its diversity indicators.

Reflection: Lessons from the Dutch Performance Agreement Experiment

The question whether performance agreements actually have made a difference is a very complex one. Before we try answering it for the Dutch performance agreements, we turn briefly to some of the results and experiences that may be found in other countries.

According to the OECD (2017), evidence from several OECD countries has shown that performance agreements:

  • are not solely meant to strengthen performance but also have aims such as encouraging HEIs to strategically position themselves, given their particular mission and strengths

  • can handle situations where HEIs have multiple objectives (education, research, innovation, entrepreneurship) and—within some nationally-set boundaries—can set their own targets;

  • improve the strategic dialogue between the government and the HEIs

  • help inform policy-makers and the public at large about the HEIs’ performance, thus improving accountability and transparency

  • can be used to promote horizontal collaboration between different actors.

The evidence also points at the following lessons for an effective design of this type of agreements:

  • Performance agreements are taken more seriously by all parties and have greater impact if financial consequences are attached. They should include a mechanism to reward “overachievement” and not just be focused on budget cuts as a result of failure to meet indicator-based targets.

  • The nature of financial incentives must be carefully chosen. The budget linked to the agreements must be sufficiently large to have an impact, yet not so sizeable to the extent that the incentive becomes a goal in itself or could lead to perverse effects.

  • Agreements must primarily pertain to goals and results. The indicators related to the targets should meet the requirements of validity, relevance, and reliability. Organisation-specific performance indicators can sometimes limit the scope for horizontal collaboration, with HEIs focusing solely on meeting the performance targets assigned to them.

Overlooking the outcomes of the performance agreements experiment in the Netherlands, the general conclusion must be that the results in terms of the two overall objectives are mixed. In the previous sections, we showed that degree completion in the research universities significantly improved while the UAS sector experienced difficulties in achieving its ambitions regarding degree completion. In terms of the second key objective, increasing diversity in terms of degree programmes and more distinct education and research profiles, results again are mixed. In terms of programme diversity, there is no clear sign of institutional differentiation: most institutions exhibit a more equal distribution of activity over their degree programmes. On the research side, we notice a clear tendency towards decreasing differences among universities, giving no indication of increased research concentration.

Although the results in terms of their general objectives are mixed, the public reactions have been largely positive. Leading newspapers reported the substantial increases in educational quality, student satisfaction and completion rates; employer organisations applauded the results of the experiment, and some politicians argued that the results show that performance agreements are a very effective steering instrument. The evaluations that were undertaken by three different committees also were largely positive. The Review Committee produced its own evaluation (Review Committee 2017b). The association of UAS institutions also produced an evaluation (Slob et al. 2017). Finally, the Minister of Education ordered an independent committee (the Committee Van de Donk) to evaluate the experiment and make recommendations for a future system of performance agreements (Evaluatiecommissie Prestatiebekostiging Hoger Onderwijs 2017).

The three committees agreed on many issues. They concluded that the performance agreements (PAs) had contributed to the following outcomes:

  • Putting improvement of study success more prominently on the institutions’ agendas

  • Intensification of the debate about drivers of study success (both among and within HEIs)

  • More attention for profiling (differentiation, focus areas) of HEIs

  • Improvement of the dialogue between stakeholders in higher education (executive boards of HEIs, Ministry, department heads, associations of HEIs, Review Committee, representatives of business and community)

  • Increased transparency and accountability, thanks to the setting of targets and the use of indicators

  • Appreciation of the possibility for HEIs to share their “story behind the numbers” with the Review Committee, especially in meetings with the Committee.

The associations of the universities and the UASs and the national student organisations were more critical. Their concerns regarded:

  • Decline of the HEIs’ autonomy, due to setting national targets and prescribing uniform indicators

  • Additional bureaucracy due to the emphasis on uniform indicators

  • The financial penalty for not achieving goals

  • Choice and definition of the seven core indicators, which in some cases contributed to unintended effects

  • Lack of time available for a well-considered construction of the rules surrounding the experiment

  • The fact that the experiment was managed largely by stakeholders that are quite distant from the “shop floor level” (executive boards, managers, ministry, national committees and organisations), with a small role only for students in this process.

In the three evaluations of the performance agreements and in the political follow-up discussion, the need to incorporate a performance-oriented component in the funding mechanism for HEIs was reaffirmed. Already earlier, the Minister of Education had expressed her intention to continue with some form of performance agreements but was keen to stress that the agreements should ultimately be about the quality of higher education and that quantitative targets regarding study success should not receive priority over qualitative ones. The future agreements will, therefore, have the label “Quality Agreements”. There still is no clarity about potential financial sanctions tied to future quality agreements. The associations of HEIs showed little enthusiasm for financial sanctions. However, the Review Committee in its evaluation (Review Committee 2017b) concluded that attaching financial consequences to agreements fosters effectiveness. It argued that both the international literature and the Dutch experiment have shown that agreements are taken more seriously by all the parties and have greater impact if financial consequences are at stake for each individual higher education institution.

Despite concerns expressed by associations of universities and students, there is political support for connecting budgets to the quality agreements. Most likely, there will be some form of rewarding (but not punishing) HEIs for meeting self-stated ambitions about quality and differentiation. The fact that those ambitions are to be agreed in close dialogue with the HEIs’ relevant (local) stakeholders implies that the agreements will lose more of their New Public Management character and develop more into a steering instrument that fits the Network Governance and Public Value Management paradigms (Stoker 2006; Vossensteyn and Westerheijden 2016).

Whether performance agreements matter for the performance of higher education is a question that cannot be answered on the basis of the Dutch experiment with performance agreements alone. Causality is difficult to prove. First of all, because the experiment was integrated in a larger policy framework with connections to other policy instruments and policy domains. Second, one needs to be aware of the fact that national policies and their associated incentives have to trickle down from the ministry (i.e. system level) to the HEI and, then, the level of the student or the academic (say teacher, researcher) to have an effect. The individual HEI and its specific characteristics is an important intermediate layer where many intervening (either obstructing or facilitating) factors are located (Jongbloed and Vossensteyn 2016). And, thirdly, one needs to understand the concept of performance much better (de Boer et al. 2015); it is a multi-dimensional concept and highly subjective for that matter.

Despite these methodological concerns, the effectiveness of the Dutch experiment can be judged favourably, and many of the positive results that performance agreements had in other countries (OECD 2017) can also be found in the Dutch system. The Dutch performance agreements are an experiment on the way towards finding an effective design for funding mechanisms and their associated incentive structure. While we have perhaps not definitely answered the crucial question “Do Performance Agreements Matter for Performance?” we have summed up some of the evidence that may be used to inform the design of future funding mechanisms.