Introduction to the special issue: “Performance measurement systems in universities: Threats or opportunities for governance?”

Introduction

The change in the public university setting in the last few decades, with the wave of New Public Management (NPM) logics and neo-liberal imperatives supported by a mode of regulation based on “fabrication” (Ball, 2000), commodification (Radin, 2001) and performativity (Lyotard, 1984), has led to a drastic reduction of public funds, managerialization and dramatic cut in costs, while obnubilating the idea that public universities produce public goods. “The relationship of the suppliers and users of knowledge to the knowledge they supply, and use is now tending, and will increasingly tend, to assume the form already taken by the relationship of commodity producers and consumers to the commodities they produce and consume—that is, the form of value. Knowledge is and will be produced in order to be sold, it is and will be consumed in order to be valorized in a new production: in both cases, the goal is exchange. Knowledge ceases to be an end, it loses its “use-value” (Lyotard, 1984, p. 4). The language of performativity has been gradually shaping the reality of public universities. The market rhetoric became dominant, and this generated a sea change in the way both internal (members of academic staff, above all) and external (students, social environment, the government, other universities, etc.) relationships are conceived and thought of. The narratives by which public universities are requested to legitimize their role in contemporary society have been transformed by the above-mentioned changes. Public universities’ functions have been “translated” into services to be rendered (and quite often sold) that is an output to be measured in quantitative terms and as much as possible in terms of costs and revenues. Performativity as a means of social legitimation explains immediately both the present generally accepted mode of considering and interpreting the achievement of public universities and the present state of the art in the field of performance measurement and evaluation of public universities: “The goal is to become an entrepreneurial university, a generator and a promoter of knowledge that bases its development on innovation and problem solving for the benefit of the society and the economic growth” (García-Hurtado et al., 2022, p. 3).

Consequently, performance measurement and management reforms have strongly affected universities, driven by resource cuts, a process of marketisation and public–public/public–private competition (Parker, 2013). While there is a growing need for measurement systems assessing research, teaching and third mission activities (i.e. service to society or outreach) to inform governance bodies and support decision making, such systems are affected by the specificities of the new ‘must’ introduced in higher education, such as the urgency of publishing, the need to attract financial resources, multiple institutional logics, among others, which make effective performance measurement systems (PMS) increasingly challenging.

A wide strand of research has touched upon performance measurement and management systems in universities and possible limitations and contradictions (among others: Dobija et al., 2019; Guthrie & Neumann, 2007; Ter Bogt & Scapens, 2012). However, reflections addressing how measurement systems assessing research, teaching and third mission activities can serve overcoming competing logics, supporting decision-making and sound governance, ensuring the creation of public value are still necessary to solve the contrasting discourses in play.

The articles in this special issue offer a multifaceted and nuanced analysis of effects of performance management in universities so far and how they function. While much of the literature has focused on policy, the public management perspective offers a fresh view of the main problems and possible solutions.

Our editorial aims to formulate some critical considerations on the expected and actual use of performance measurement (and management) in public universities, also based on results emerging from the four articles included in this special issue (Biondi & Russo, 2022; Pilonato & Monfardini, 2022; Salemans & Budding, 2022; Tandilashvili & Tandilashvili, 2022), providing some reflections on the need for research to promote rethinking PMS in search of a mode of assessment of public value creation.

Performance measurement in universities: for better or for worse?

The introduction of performance measurement in the public sector has been largely connected with the NPM pillars, which imposed to some extent the adoption and implementation of managerial and accountability tools, already in use in the private sector (ter Bogt & Scapens, 2012). More specifically, performance measures, coupled with reporting systems, have been deemed able to increase public sector organization accountability in the use of public resources, to keep cost under control while recapturing efficiency, to stimulate responsibility and entrepreneurship by public managers in the achievement of assigned objectives (Mussari & Sostero, 2014). However, the increased focus on output measurement, and the scarce ability in assessing the outputs, may lead—together with some of the expected benefits—to some unintended consequences, in particular in knowledge-based organizations, such as universities. In fact, the use of simplistic measures and an attitude to “governance by numbers” (Osterloh, 2010) seem to neglect that quantitative measures are not always the most appropriate ones to assess scientific research, teaching quality and contributions that universities can offer the community (Kallio et al., 2017). The managerial ethos that is currently characterizing universities governance has overshadowed academics’ ownership of research and teaching (Cook et al., 2022). Scholars have also warned about the unintended consequences of performance assessment in public universities, where a weak correlation between performance indicators and expected output and outcomes persists, and the institutional and cultural context may play a pivotal role, undetected by standardized measures. (Vakkuri & Meklin, 2003). Almost 20 years ago van Thiel and Leeuw (2002) suggested that the misuse of performance measures can bring about severe unintended consequences, where reaching predefined targets and measures can be considered an objective per se, detached from the initial aims calling for the introduction of PMS. For instance, the tension towards publishing in highly ranked journals, to accomplish promotion decisions and requests from research funding associations, may compromise local research history and traditions (van Helden & Argento, 2020). In the same vein, the obsession with publications may shift the attention of academics towards research topics that are trendier and more easily publishable (Mussari & Sostero, 2014). The purpose of student surveys to assess simultaneously teaching quality and student satisfaction can create contradictions and cause stress to academics, leading to insufficient reward and recognition of academics' performance (Cook et al., 2022). Furthermore, the extreme measurement pressures in universities have been accelerated by the use of IT platforms supporting the development of “governance modes that oppress the scholarly voice of university academics” (Nørreklit et al., 2019, p. 872). The progressive adoption of performance measures produced on the base of artificial intelligence techniques and Big Data dramatically increase the risk of uncontextualized decision-making, where people and context become irrelevant (Cinquini, 2019). As earlier as 1996, Mintzberg warned of the side effects connected with aligning public sector organizations to a managerial fashion based on the idea that performance can be fully and properly evaluated by objective measures, despite the intrinsic incapacity of such measures to capture different nuances or to be adapted to specific contexts. He suggested that there was no single best model for managing governments, but that there was a need to re-think the role of values and beliefs.

In recent times, the public value perspective and the need to put back ‘publicness’ at the center of research on public sector organizations (including public universities) have come to the fore (Bracci et al., 2021; Steccolini, 2019). It can offer a useful perspective in re-considering the role of performance measurement and management in public universities in monitoring the creation of public goods and the related implications for governance. Shifting from neo-liberalism and exacerbated managerialization in the use of performance measures requires first to recognize the need for multifaced measures, that have to be adapted in accordance with specific contexts, encompassing the needs of a variety of stakeholders and then to reconsider how multiple and sometimes competing values can be reflected in designing new PMS, in support of sound governance.

How to change performance measurement systems? Reflections from the special issue

The main issues raised on the functioning and effects of performance management in universities are similar to other typologies of public sector organizations. These include: an implementation gap, value conflict and a decline in the motivation and morale of human resources. The most serious limitation of performance management systems, however, seems to be the output and competition orientation, frequently the cause of the other shortcomings.

A focus on the individual level helps to understand the role behavioral factors play in the successful implementation and use of performance management systems (Viale, 2021). Both in France (Tandilashvili & Tandilashvili, 2022) and in Italy (Pilonato & Monfardini, 2022), the perception of academics and managers have an impact on performance management.

Tandilashvili and Tandilashvili (2022) describe the shift from the traditional academic logic to the new market-driven logic, which has impacted the academics’ professional identity and the common missions of research, teaching, and service to society. New rules introduced by the performance-oriented control systems have come at the cost of traditional values such as democracy, collegiality and freedom (Argento & van Helden, 2021; Grossi et al., 2020).

Through an analysis of French universities, they show how the impact of New Public Management-style performance management systems have empowered the university presidents, creating value conflicts.

More in general, Pilonato and Monfardini (2022) explore how individuals face managerial reforms using the frame of institutional logics. The authors look at the performance management model applied to teaching in Italian universities. The reforms put internal contradictions between multiple individual goals and identities in the spotlight so that the same reform contains a plurality of organizational and managerial consequences. The implementation gap, more than a decade after the reform was introduced, is attributed either to individuals not recognizing some logics as fully legitimate or to logics referring to apparently incompatible values.

The individual level analysis provides a novel view of the effects of institutional complexity within the organization. In Italy, as in France, empirical results show that resistance to performance management will probably come from academics who perceive the new system mostly as a constraint to their academic freedom. The pre-existing set of strong values related to freedom and self-reference teaching, the role of the national and international scientific community in the individual evaluation, and the principles of free access to university education, need to be integrated with values more consistent with the managerial logic, which include quality, assessment and efficiency. While there is a common reaction of resistance, some respondents also emphasized that the managerial logic empowered by the new performance management system can give them opportunities to operate in their organizations in a different nuance, contaminating the professional logic (Pilonato & Monfardini, 2022).

Besides the key problems, the articles in the special issue offer important recommendations on possible solutions. The main one seems to be the urgency of reclaiming the specificities of the public sector vis-à-vis business organizations. This translates in:

• Adopting outcome-based performance management systems;

• Balancing the different interests and logics by including all relevant stakeholders in the performance measurement and management processes.

Biondi and Russo (2022) point towards the need of integrating strategic management with performance measurement. Measurement supporting the achievement of strategic objectives (e.g. expected outcomes; Bracci et al., 2019; Talbot, 2008) could help avoid counterproductive and opportunistic effects experienced with output-oriented systems.

Similarly, Salemans and Budding (2022) analyze the unsuccessful efforts of Dutch universities in incorporating outcomes (or the public value dimension) in performance management and measurement. While all the higher education institutions analyzed aim to deliver public value, they still use performance indicators that have a narrower orientation, and are primarily focused on processes, outputs, and service delivery quality. More than 25 years after Moore’s first introduction of the public value concept in 1995, its operationalization is still considered challenging (Moore, 1995). The picture that emerges is that although Dutch universities express their ambitions in public value terms, their use of performance indicators in the annual reports does not match that. Customer satisfaction and efficiency indicators are used, which were part of former performance agreements between the universities and the Dutch Ministry of Education, Culture and Science, but are no longer applicable. Nevertheless, the institutions decided to continue using these showing difficulties in operationalizing outcomes and public value dimensions. However, the authors also observe that universities use narratives to demonstrate the public value that has been created.

This struggle towards outcome-oriented management highlights several points. First, standards and formats imposed by central government may play a negative role, hindering the natural evolution of performance management systems from the New Public Management to the New Public Governance principles (Cepiku, 2005; Osborne, 2006). Another obstacle may be the very nature of outcomes: long term, risky to achieve as they depend on different stakeholders, difficult to measure. Measuring what can be measured rather than what is relevant has strong consequences in terms of implementation and behaviors.

The second solution put forward by the special issue authors calls for tempering the different interests cohabiting universities: administrative staff, professors, students, international scientific community and also local communities and politicians.

The impact of performance measures and management on governance in universities is still understudied.

The cohort of stakeholders whose participation can improve the effectiveness and benefits of performance management systems, however, is wider and should be considered in future studies, starting from the students. Participatory planning and evaluation approaches are gaining ground in other public sector areas and can be beneficial to universities as well.

Another interesting finding from the special issue is that, while research and teaching have been affected by performance management reforms, the third mission seems so far less affected.

Future research should explore how institutional logics influence the opportunities and constraints perceived by key actors when a new performance management system is introduced.

A way forward in public universities: final reflections and possible research avenues

Research on performance measurement and management in public universities has so far been unable to identify possible solutions to satisfy in one fell swoop the multiplicity of actors interested in those measures. Rather, possibly the evident contradictions between expected results and unintended consequences deriving by the adoption of PMS in different contexts and for different actors, highlight how the realizations of certain public values (e.g. ensuring high quality research; provide students with high quality teaching; ensure universities’ contribution to the development of society) may be in contrast with some people’s values (e.g. ambition for academic career, obtaining research funds, experimenting new research approaches, etc.).

National strategies adopted in monitoring higher education institutions need to consider possible ambiguity that may derive from top-down designed PMS and contrasting discourses that may derive from organizational level. At the same time, at micro-level, governance bodies have to consider how to pursue institutional goals, to be made accountable through a suite of indicators, without creating conflicts with individual goals both at managerial and academic levels, safeguarding the interest of students as the ultimate recipients of universities’ services. International funders may carry out evaluation procedures including not only quantitative measures on which to base the distributions of their resources, but also qualitative measures, in an attempt to allocate their funds to research teams that ensure the creation of value for the benefit of the community. To synchronize all these efforts is far from easy, especially because it requires the identification of indicators that match the different aims, escaping the performativity traps. Academics can support these efforts continuing their attempt to analyze and interpret the reality, making use of alternative interpretative lenses and research methods attuned to consider effects induced by the use of current measures. But above all, all actors should bear in mind that universities are mainly complex organizations, working for the benefits of multiple stakeholders, in the pursuit of public value. Consequently, performance measurement and management systems, suitable to support sound governance, may be better designed by identifying the common values around which common efforts should be made, and the tension inherent in the public value creation in the specific context. In the same way, performance measures should not be adopted to support ranking activities and determine who (or what) is the best and who (or what) is the worst, but rather should support improvements by those organizations and actors that strive for better performance.

A further research avenue to be developed may also consider how universities are incorporating the United Nations Sustainable Development Goals (SDGs) in their strategies and actions. These ambitious goals may promote greater attention to the achievement of public value creation, and it would worth noticing if and how (and to what extent) the adoption of SDGs is affecting performance measurement and governance in universities.

The papers in this special issue appear to show that a way forward in shaping PMS in public universities makes it necessary to carefully consider the varying expectations and needs of a cohort of stakeholders, including managers, academics, researchers, students and the community in which these institutions are located. Future research may explore if and how involving all actors in a collaborative effort for combining different values may result in an alternative combination of performance measures, addressing the measurement of public value produced by universities.