Keywords

1 The Need for Informed Higher Education Policies

The articulation of European Higher Education Area (EHEA) and related policy processes has brought into attention a series of challenging “transition” paradoxes, mainly to the strategic steering of higher education systems and institutions, but also to the key actors in this field. High among them are the following:

  • From the elite to the massification of education and training;

  • From the local/regional approach to the global approach;

  • From the use of mainly governmental funds to attracting private investments;

  • From collegial management/steering to corporate management/steering;

  • From national to cross-national standards;

  • From regional to international mobility;

  • From national cultural frameworks to cultural diversification;

  • From accreditation to quality enhancement mechanisms—trust, transparency, impact;

Starting from the last point above, namely the need for trust, transparency and impact assessment, we will try to illustrate the role and function of transparency tools (with a specific case-study) in the new conceptual picture and social mission of higher education in Romania.

In the following chapter we will undertake an ex-ante analysis of the implementation of one tool (U-Multirank) in Romania, as an example of a relevant policy that aims to address the information asymmetry at the level of the higher education sector. The U-Map sets to develop a more elaborated and versatile model through which institutional performance can be measured and compared, and also a series of new indicators, considered as being more appropriate and valid to capture the perspectives of beneficiaries by the authors of this approach. Our analysis is therefore a contribution to the current debate regarding the tools for impact assessments at system level, suppliers, professional practice and student and other direct/indirect beneficiaries level and their role in assuring a higher transparency and information symmetry (Hazelkorn 2012; Usher and Jarvey 2010; Van Vught and Ziegele 2011). The aim of the chapter is to identify the policy measures that best address a set of external criteria that refer to the scope, the impact, the depth and the associated costs of the intervention. We will approach this ex-ante impact analysis by estimating the advantages and disadvantages of each of the policy options identified, based on the secondary analysis of the existing data. In the end we will propose a policy measure that best addresses the transparency issues given the criteria that we set.

Higher education represents a set of institutions through which high-level qualifications are provided following professional and academic training processes (Witte 2006; Maasen and Stensaker 2011). This set includes educational suppliers (higher education institutions), stakeholders (students, academic staff, parents, employers etc.), regulatory and supervisory organisations (the ministry, consultative committees, quality assurance agencies etc.), regulations, social rules and practices. Higher education is organised similarly to a market, in a broad sense of the word, a market in which educational services suppliers offer study programs at various levels (bachelor, masters, doctorate) to current students and potential beneficiaries. The market runs mostly on the principle of demand and supply. The state is an important social stakeholder which plays a key role in the transactions between various actors (suppliers, clients, beneficiaries) in order to correct the dysfunctions caused by market imperfections. Government interventions are made through educational policies in the higher education sector (Curaj et al. 2012; Păunescu et al. 2011). The need for these policies is justified by the following core reasons:

  1. (1)

    Higher education is a public (or partially public) good or it exhibits the features of such a good. The benefits of higher education are non-exclusive, since they spill over to the society at large and do not only affect the direct beneficiaries (graduates, their families). The development of the human capital is crucial for an economy, allowing its organizations to recruit highly qualified workforce. Therefore it plays an essential role in attracting capital investments and economically developing the society as a whole. In addition, the innovation and the technological progress facilitated by the universities are beneficial to the society in general. The development of citizenship through the higher education study programs also has a positive impact on the democratic functioning of a nation. As it does produce a lot of positive externalities (the benefits go beyond those who pay the direct costs for its provision), it displays the features of a public good, higher education tends to be produced and delivered at suboptimal levels, entailing that generally higher levels of provisions and consumption would pay the additional costs. This leads to state intervention through public policies in order to increase participation or consumption (measured, for example, through transition rates from secondary to tertiary education, higher education enrollment rate; the percentage of degree holders in the 30–34 years age group, etc.) or to enhance the quality of the results and the scope of impact (graduation rates, labor market insertion rates). Public policies in higher education generally pursue such objectives, aiming at raising the overall participation in the sector, as well as the effectiveness of the educational sector (Vlăsceanu et al. 2011).

  2. (2)

    Secondly, at the level of the higher education sector, there is an information asymmetry between providers and beneficiaries of educational services, which leads to systemic distortions of the market. Student choices are distorted because of the lack of standardized and comparable information on the higher education providers. Students most often rely on surrogate information such as peers’ choices or family guidance. This situation is actually to the advantage of education providers. The lack of state or other type of regulations indirectly encourages higher education suppliers to publish only the favorable pieces of information that drives their competitive advantage or to process and analyse the data to their own advantage. A certain degree of approximation can be found in all these types of information, since it does not always reflect the quality of a study programme/supplier. Consequently, decisions are made without relevant information and inefficient social services do not easily find their way out of the market (Hazelkorn, 2011; Păunescu et al. 2012).

State intervention through public policies enforces a certain standardization of the information provided by the higher education suppliers in order to make it usable and comparable. Transparency tools such as the multi-dimensional higher education institutions classification (U-Map) or the multi-dimensional ranking (U-Multirank) are examples of recent policies assisting future students by enabling evidence-based individual decisions. Seen from a public policy perspective, the use of these tools contributes also to three important dimensions/functions (Ciolan et al. 2009):

  • The accountability function, responsible for determining the quality of the public policy and for accountability to beneficiaries and financiers, which have to be informed in regard to any intervention performance and impact;

  • The management function, which provides early warnings, solutions and recommendations for the continuous improvement of the public policy during its implementation; impact management is an adaptive approach that is aware of the contextual factors that influence success;

  • The learning and developing function, which refers to the institutional capacity to capitalise on the successes and failures of an intervention in order to perform better in that same intervention and especially in future similar ones.

2 U-Multirank as a Transparency Policy—a Case Study on the Romanian Higher Education Sector

2.1 Context

U-Multirank is a recently developed transparency tool that aims at producing rankings of European higher education institutions and beyond, both in terms of overall institutional characteristics and also specific rankings on fields of study. U-Multirank is intended as a user-centered tool, democratic, abandoning the principle of alleged, implicit, epistemic authority in alternative, unitary rankings. Other important principles that stand at the basis of this model are: avoiding simplistic league-tables, wide applicability, approach based on empirical data, involvement of multiple stakeholders in providing information and tracking several dimensions, robust scale and special attention given to collecting additional data. All the activities are based on the voluntary decision of universities to be part of this exercise. It is expected that the first results on the multi-criteria classification of 500 higher education institutions, as well as the study fields of mechanical engineering, electrical engineering, business administration and physics, will be published in 2014. In the following years U-Multirank is planned to be progressively extended in what concerns the number of institutions and fields of study. For example, sociology, psychology, social work, computer science and music will be introduced in the ranking process by 2015.

The basic assumption from which this transparency tool begins is that there is no universal definition of quality in higher education on the basis of which to build a unique methodology for ranking. Conversely, higher education fulfills multiple functions for different stakeholders and therefore requires a multidimensional assessment to empower stakeholders in producing different rankings, according to their areas of interest. Classic rankings give arbitrary weights to different performance assessment criteria (generally giving substantial weight to research) and produce composite scores based on which they can generate unique rankings. The problem with this approach is that by aggregating the scores obtained on various indicators arbitrarily weighted, reductionist and artificial results are produced. Furthermore, such rankings are often irrelevant for different categories of stakeholders that do not pursue excellence in research, but rather in the quality of teaching, in the relations with the external environment, community engagement, etc.

In the case of U-Multirank, the dimensions/criteria taken into account when generating hierarchies, besides the classical one regarding research, are teaching, knowledge transfer, international orientation and community/regional commitment. Based on these, multiple rankings can be generated, taking into account the interests and values of the user of the resulted classification.

U-Multirank starts from a comprehensive definition of higher education. The general characteristic of higher education and research institutions is the processing of knowledge (Clark 1983; Becher and Kogan 1992). These institutions are engaged in different ways, in breakthrough efforts, in the development, conservation, transmission and application of knowledge (Clark 1983). On the other hand, U-Multirank assumes both vertical and horizontal institutional diversity corresponding to different levels of quality of educational services provided (which also corresponds to different educational levels), as well as to various institutional profiles in accordance with institutionally assumed missions and also to different audiences of higher education institutions. With regard to quality measurement of the provided services, the U-Multirank project includes indicators regarding the institutional capacity of higher education institutions, so-called input indicators but also the articulation of processes for knowledge building and transfer, as well as the short, medium and long term results (impact). Furthermore, there are indicators regarding different higher education institutions capacities and different missions like: education, research and knowledge transfer. Also, some indicators refer to objective, quantifiable data regarding inputs, processes and results, while other indicator categories refer to subjective aspects such as opinions, perceptions and attitudes of beneficiaries or other interested stakeholders.

In the following we will try to make an ex-ante assessment of U-Multirank as a transparency tool addressed to Romanian stakeholders.. We will thus approach this ex-ante impact analysis by estimating the advantages and disadvantages of each of the policy options identified against a set of a priori criteria. The evaluation will be based on the secondary analysis of the existing data at the higher education sector level.

2.2 An Ex-ante Assessment Case-Study for the Introduction of a Transparency Tool

U-Multirank is a public policy initiative aimed at reducing or eliminating the information asymmetry offered by different providers and needed by various stakeholders (beneficiaries/founders of higher education services). An ex-ante analysis must start from defining the problem that the proposed public policy is meant to address. The questions the analysis has to answer at this stage are:

  1. 1.

    What are the nature and the scope of the problem that the public policy aims to address?

  2. 2.

    Who are the stakeholders involved and directly affected by this problem (organizations, institutions, individuals)?

  3. 3.

    Which are the main types of options, scenarios and measures that should be considered?

All these questions will be approached and analyzed below.

1. Analyzing the need for a transparency policy in the Romanian landscape of higher education, the identified problem refers to the lack of valid, standardized and comprehensive data which will enable different stakeholder categories to make multidimensional comparisons based on specific aspects of higher education. Higher education is not a commodity that users can evaluate a priori (for example based on competitive prices). Higher education is an ‘experiential’ service (Nelson 1970): users can evaluate the quality of the service provided only after they have ‘experienced’ it, but this type of ‘experience’ is an ex-post knowledge. Other authors consider that not even the learning experience is sufficient for a correct evaluation of the service provided. Dulleck and Kerschbamer (2006) consider that the value of the service provided becomes clear only after graduation (or even long after) depending on how the gained competences improved the graduates work and social status. However the most important characteristic of the evaluation of higher education services is its’ multidimensionality and the difficulty of weighting indicators/dimensions according to the various needs of different users/beneficiaries. Different categories of stakeholders need specific information to enable informed decision-making processes. Prospective students and their parents might be interested in the global performance of universities in education, employability of graduates from a particular field of study, study conditions etc., thus aiming at choosing a specific supplier for educational services. The state or other financing organizations might be interested in the overall performance of universities, as well as knowledge transfer and the social value of research in choosing the most effective public investment. The different nature of the goals pursued by different stakeholders requires for relevance valid and specific data, collected in a strandardized manner that allows relevant comparisons. The problem is thus the lack of a set of comprehensive, standardized and valid data for the different dimensions of higher education, both for the institutions and the study programmes, that would allow both transversal and longitudinal comparisons between programmes as well as institutions, relevant for different types of stakeholders.

2. There are different categories of stakeholders that might have interests in a public policy on transparency of information regarding higher education institutions. On the one hand, institutional stakeholders as the state, employers, evaluation and quality certification agencies and consultative councils of the ministry are interested in the development and consolidation of comprehensive multi-annual databases on which to fundament different types of public policies for higher education. On the other hand, higher education institutions themselves can benefit from longitudinal comparative data as well as transversal data collection would allow setting of benchmarks against which to compare their own performance in order to improve the services offered. Finally, beneficiaries, individual stakeholders, students, parents and also organizational stakeholders and employers need information regarding learning outcomes and higher education impact in terms of employability, the exploitation of competences and skills on the labor market etc.

3. The option types or various policy scenarios vary depending on the nature and the level of the public intervention. In what concerns the nature of the intervention, in solving a problem there are at least two possibilities: public intervention or non-intervention and the self-regulation of the higher education system itself. When the second case is preferred, one starts from the assumption (if not empirically demonstrated) that the public objectives set out and pursued by public policies at system level can actually be better achieved following the stakeholders’ individual choices (individuals or organizations) without state intervention. This option starts from the assumption that the market offers enough incentives for higher education institutions to publish information about their own performance, while the beneficiaries/financiers have the capacity to identify, select and analyze the information available in order to make rational decisions. Equally, the higher education institutions could also identify a common interest and therefore be able to cooperate by initiating and sustaining a collective action without state intervention (for example establishing a common methodology for reporting results or establishing their own benchmarking system). In both cases, public intervention would become obsolete (and probably also harmful by distorting the market incentives) as long as the competition conditions or the cooperative social capital of organizations in that sector provide sufficient incentives and guidance for universities to publish relevant pieces of information. It is further assumed that beneficiaries can easily access and use this information at a low cost. In this case, the intervention can be minimal by encouraging universities to publish certain pieces of information, offering data presentation recommendations, facilitating exchanges of good-practice examples on ensuring transparency or supporting collective actions—self-regulations between higher education institutions. However, the intensity of public intervention could vary on a continuum having as milestones the following:

  • Major intervention—using several tools of intervention, taking many public objectives, highly institutionalized means of achieving objectives, restricting individual choices;

  • Moderate intervention—intervention tools that guide individual options through sanctions and rewards, assuming a certain degree of self-regulation, incremental policy change according to the evolution of the expected and unexpected consequences of policy implementation;

  • Minor intervention—soft tools of intervention, based on informing stakeholders and facilitating collective action (for self-regulation purposes).

In what concerns the level of intervention, it can be local, state or supranational.

2.3 Current Situation. Baseline Scenario

The current situation in Romania is that of moderate, multiple public interventions, both at state and supranational levels, oftenly perceived as being poorly coordinated and most often redundant interventions. Different stakeholders, usually public authority bodies, have imposed different data collection mechanisms which are mandatory for most higher education institutions and create a lot of strain on them, as they differ from other methods used for reporting to different bodies. This is easy to observe as there are different mandatory methodologies for data collection: a) the legal provisions on the financing of higher education have produced informational flows managed by the National Council for Higher Education Financing (CNFIS) together with the Ministry of Education; b) the national education reports that are in line with the Eurostat methodological requirements are managed by the National Institute of Statistics (INS). The Law no. 87/2006 approving the Government Emergency Ordinance no. 75/12.07.2005 regarding the higher education quality assurance establishes the principle of internal and external evaluation of the quality of institutions and study programs using its own taxonomy of criteria, standards and indicators. The institutional and programme data are managed by the Romanian Agency for Quality Assurance in Higher Education (ARACIS) that has also the legal power to propose to the Ministry the accreditation or non-accreditation of an institution or a study programme. In addition, following the provisions regarding the university classification and study programs ranking mentioned by the National Law of Education no. 1/2011, the new national assessment exercise for classification and ranking was established, its’ methodology being approved by the Government Decision No. 789/10.08.2011.

The situation is characterized by a marked lack of coordination of the various data collection initiatives regarding the higher education system. The UEFISCDI analysis regarding the various information flows in the Romanian higher education system mentions (pp. 3–5):

Currently there is no integrated information system that can produce a unitary data collection and generate useful data in basing higher education public policies. Moreover, there are several reporting and data collection exercises at the national level (from higher education institutions) for the same type of data only that they request presenting it in different formats or machetes. This endeavor squanders resource and is time consuming for all the stakeholders involved. […] Data collections are systemically done (at intervals clearly defined in time), on one hand by The Ministry of Education through the General Department for Higher Education (DGIS) in collaboration with CNFIS—UEFISCDI and, on the other hand, by the National Institute of Statistics (INS). Other institutions either make periodic data collection on their own depending on their legal provisions (the National Research Authority, UEFISCDI—National Council for Scientific Research-CNCS for research centers), or do not directly collect data, but make use of the ones which are already public being published by the National Institute of Statistics in its publications (for instance the Institute for Educational Sciences).

In conclusion, we can state that the comparability of data collected through different methodologies is very low and most often than not the availability of primary data for generating of customized reports suited for the needs of different users is nonexistent. Data collection methodologies often change over time depending on the specific criteria from the financing methodologies (for example, the evolution of quality indicators for the allocation of the competitive primary financing). This leads to the lack of comparable data regarding the dynamics of some specific aspects of higher education institutions.

In addition to central authority agencies requiring data on higher education institutions, another data source is represented by the self-evaluation reports (both institutional reports and, respectively, study programmes reports) that the higher education institutions must submit to the Romanian Agency for Quality Assurance in Higher Education (ARACIS), before the Agency’s external evaluation, for accreditation and/or mandatory periodic evaluation purposes. In this context, quality assurance also performs the function of a transparency tool, enforcing the accountability for education providers and offering interested stakeholders with reliable information validated by an external evaluation agency. The collected data are both objective—measurable performance indicators and subjective—the results of peer review evaluations. The problem with this data is that the information presented is either, descriptive and little comparable, or in many cases only the evaluation results are publicly available, but not the data that lead to the evaluation agency’s decision. Most often than not, the evaluation results are poorly differentiated, most institutions being accredited by ARACIS and awarded with a ‘high confidence’ label; thus, there is little vertical differentiation and, to the extent it is, it is only one-dimensional (ultimately generating a single-ranking of institutions and/or study programs). Furthermore, as shown in the Quality Barometers published by ARACIS (Vlăsceanu et al. 2010; Paunescu et al. 2011) internal quality assurance is mostly a ritualistic and conformist process, often decoupled from the university management processes. Data reliability is, in this case, reduced. In addition, the autonomous and independent use of the disclosed information by various individual/organizational stakeholders is limited to the conclusions/evaluation reports of various assessment exercises. The primary data is often unavailable in usable and user-friendly forms in order to generate customized reports in accordance with the need of information and relevant, specific criteria of different stakeholders’ categories.

The data collected during the national exercise for university classification and study programs ranking were gathered in 2011 only and were, to a limited extent, public as aggregate higher education institutional reports. Users did not have any possibility of autonomous data management for relevant comparisons, based on their own criteria. The data collected were processed through a national methodology approved by Ministry Order no. 5.212/2011 which reflects a certain vision on higher education (centered mostly on research) and resulted in a single ranking by providing weights and aggregating indicators specific to different dimensions. Eventually, the ranking is single-dimensional and reflects a particular view of higher education, mostly centred on research; while this could be beneficial for producing a vertical differentiation, it severely limits the horizontal differentiation. However, the most important shortcoming remains the narrow autonomy of users in selecting and using the data in order o generate customized comparisons that are relevant for the respective user’s criteria. The public nature of the data collected is also affected by the lack of availability of an online tool with a simple interface through which data of interest to each individual user can be accessed and processed.

2.4 Setting of Policy Objectives

The objectives of public intervention on increasing transparency, reducing the information gaps on the higher education market and consequently increasing the public confidence in the higher education services are as follows:

O1. Setting of a comprehensive and standardized data set with relevance for the performance of higher education institutions and the study programs they offer;

O2. Enabling cross comparisons (between institutions at a specific moment) and longitudinal comparisons (between different moments in time regarding the evolution of the same institution/program and its characteristics) that capture the relative quality of the services provided;

O3. Increasing the autonomy of various stakeholder categories in higher education data usage and making relevant reports, according to their own informational interests.

2.5 Option/Scenario Identification

The current situation/baseline scenario is sub-optimal in several respects. On the one hand, it is characterized by a persistent informational asymmetry between users and beneficiaries of educational services. On the other hand, the inflation of informational flows puts a lot of strain on higher education institutions due to the multiple and, in the same time, redundant reporting requests; these are also time and resource consuming and eventually lead to a sub-optimal data usage. Data that cannot be compared and used in various assessment exercises because of the incompatible methodologies and collection methods used.

There can thus be various scenarios that address in different ways the objectives mentioned above. The public policy options for the achievement of the above stated public objectives would be:

  1. 1.

    Integrating the existing informational flows into a single centralized consistent process through which to collect the data requested in both primary legislation (quality assurance, education law) as well as secondary legislation (norms associated with the financing methodology, EUROSTAT methodology). In this scenario the information would also be centrally processed and unitary institutional and study programmes reports would be generated. The state will assume the role of data collector and processor and would standardize specific type of reports by replacing them with a unique framework that collects data useful to various stakeholders, usually agencies and public bodies.

  2. 2.

    Standardizing the mandatory institutional reports of higher education suppliers; the reports are actually mandatory as requested by the Law of National Education no. 1/2011, but this scenario will further elaborate secondary norms imposing the form, type of information and of data to be submitted regularly by each education provider. It will thus force the suppliers of education services to publish regularly a precise set of data (as defined by the norm) and not as a freely written report as it is happening. This will conduct to a standardized set of institutional reports that would comparable both transversal and longitudinal.

  3. 3.

    Carrying on with the national evaluation exercise for university classification and ranking of study programs. In this scenario a novelty would be the integration of collected data (at least some parts of them) into an accessible database through an online tool that can generate customized reports, according to user information needs. The centralized classification and ranking exercise could thus become transparent for all users’ categories.

  4. 4.

    Improving the ARACIS external evaluation methodology for higher education institutions and programs/fields of university studies in order to increase the relevance of self-evaluation reports (for example through the continuous development and adaptation of benchmarks). In this scenario, the self-evaluation reports would likely become more reliable and richer in useful information and comparable data and would be usable by various stakeholder categories;

  5. 5.

    Assuming the implementation of U-Multirank at the national level by encouraging the higher education institutions to participate in the European project. In this scenario, the policy objectives would be followed by assuming and streamlining a European project, generating types of data and procedures that are comparable and compatible at European level.

  6. 6.

    Assuming the mandatory participation of all Romanian higher education institutions in U-Multirank. This scenario will enforce a European initiative through national authority and will foresee institutional penalties for failure to comply.

The above options are clearly not mutually exclusive. Some of these can be implemented at the same time, but they have different advantages and disadvantages associated with various stakeholders´ categories. On the other hand, the nature of the intervention differs as well as its level: options 1 and 6 represent major interventions, options 2 and 3 represent moderate interventions and options 4 and 5 represent minimal interventions. Options 1 to 4 are formulated at the national level while options 5 and 6 are formulated at the supranational level.

2.6 Options Analysis

The following option analysis will be conducted by taking into account the following criteria:

  • the scope: degree to which various stakeholders’ needs are included and addressed;

  • the impact regarding the fulfillment of the assumed policy objectives;

  • the type of intervention: the criteria of proportionality and subsidiarity—when the objectives of the public intervention are achieved equally effective, minimal or moderate interventions are preferred to major ones, more intrusive and restrictive of the individual liberty; also, the principle of subsidiarity requires that the intervention level should be kept to a minimum providing the effective achievement of the policy objectives;

  • the balance between the main costs and benefits associated with various stakeholder categories.

Option 1 effectively addresses objectives 1 and 2, but does less in the case of objective 3—the autonomy of various stakeholder categories in using data and generating reports relevant to their informational interests. The scope or degree of stakeholders’ coverage is limited to institutional actors that use data to elaborate public policies, the option being less relevant to students, future beneficiaries and parents. Also, being a major public intervention (centralized and enforced by authority), it is less preferred to others that fulfill the same objectives. Individual stakeholder costs are greater because they have to identify the relevant information in the various standardized reports for the public agencies/authorities and consequently to compare the information from different sources. However, the advantages arise from the increased validity and increased amount of the data verified by national authorities and consequently the public confidence.

Option 2 is the moderate version of the above option. Reports, as transparency tools, are made public by the institutions themselves, but are susceptible to data manipulation from institutional stakeholders. Also, the costs associated to generating relevant data by the information users are high because of the individual researching efforts to find and further process the available data. The advantages come from the establishment and effective usage of institutional reports as transparency tools, the enhanced institutional responsibility and fostering of a quality culture at the level of higher education providers, the increased communication and direct information paths between suppliers and beneficiaries of educational services.

Option 3 is beneficial both for the institutional users (funding or evaluation agencies) and for the individual users due to the online interface that allows data selection and generation of meaningful comparisons, relevant to the specific interests of each user. There is a high scope (degree of coverage) due to the fact that all the institutions take part in the exercise while data access and availability is guaranteed by a user-friendly interface. Data access could be made different according to each user’s category. The disadvantages are related to the fact that the specific national methodology of the classification/ranking exercise does not always allow the comparison of Romanian institutions/study programs with their European or international counterparts. At the same time, such a measure implies the risk of formalizing and ritualizing the reports and even data manipulation in the context of a lack of rigorous control; on the other hand, controlling and verifying the reports implies significant costs for the evaluating agencies. It may also discourage the emergence of a true quality culture as the standards, indicators, processes and procedures are imposed from top-down. Alternatively, the previous option (option 2) as well as the next option (option 4) encourages universities and the other providers to take ownership of the reporting process and the quality assurance mechanisms.

Option 4 leads to increased institutional autonomy and public responsibility of higher education providers, also fulfilling the quality enhancement function. Besides achieving social information functions, the collected data will also be relevant to the specific missions of each higher education institution (through their own choice of a set of relevant indicators for their assumed objectives/missions, in addition to a minimal set of mandatory indicators for all types of providers). The disadvantages consist in the high costs experienced by stakeholders in the selection and analysis of specific data; there can actually be information barriers for the stakeholders that are not experienced in the documentation of various sources and data analysis. This option is more appropriate as an effective quality enhancement tool for providers, but less as a transparency tool for beneficiaries—this function is fulfilled rather indirectly through the high validity of the data reported by the universities and through the increased trust between providers and beneficiaries. This option also fosters the emergence of sustainable quality cultures within the educational providers.

Option 5 meets all the policy objectives set; it is also a moderate intervention. The scope or degree of coverage can however be limited to the universities/study programs that decide to participate in the European project. It is also an option that yields long-term benefits: only after a critical mass of institutions and study programs participate in the project, the tool becomes effective for appropriately informing the potential beneficiaries. Also, only following the large-scale inclusion of providers and programmes in this project, the self-regulatory mechanisms at the field level become effective through the high image costs incurred by the non-participant institutions and programmes. The disadvantages consist in the excessive standardization of certain indicators and lack of flexibility in the design of alternative indicators. Also, the lack of data required by the national authorities in the design of public policies (the methodology and types of indicators are set out at the level of the U-Multirank project without consulting the national authorities) is also a disadvantage.

Option 6 also meets all the objectives set, but being at the same time a major intervention method that could take to a faster achievement of objectives; this comes with the cost of limited institutional liberty. On the other hand, U-Multirank design philosophy is centered on the voluntary participation of higher education institutions. Mandatory provision in the national legislation would lead to a disproportionate and excessive measure with regard to the other European higher education institutions participating in the project. In addition, such a measure implies the risk of formalizing reports and even data manipulation associated with if not properly enforced by control specific to a major intervention. Such a control would not be appropriate for the philosophy of the project.

On the other hand, given the specific context of the higher education market in Romania, both option 5 and option 6 may lead to a limited impact as concerns the student choices regarding the education suppliers/services offered. Bachelor students’ mobility in Romania is relatively low according to the study The university graduates and labor market recently conducted by UEFISCDI. The majority of students (66.5 %) do not choose to travel to a different study region from the region in which they passed the Baccalaureate; this fact points out that the main criteria in choosing an education supplier is still that of residence proximity rather than any other reason. The impact of the measure could be more significant only for the minority of students (33.5 %) that chose to migrate to another region (0.3 % to another country) to attend the courses of a faculty/university. The percentage of students who chose an international mobility is insignificant, and that actually points out a very low need for an international transparency tool. On the other hand, a transparency tool that includes higher education institutions from abroad can provide comparative information to foster greater future mobility, both national and international. However, this advantage is weakened by the fact that the actual undergraduate mobility is quite low as it can be seen from the table below. The data presented below represents a transversal view of the graduates of 2009. The dynamics may suffer changes, also fostered by the information made available by a comprehensive transparency tool, but at this stage it gives an image of the level of students mobility in Romania (Table 1).

Table 1 Mobility of Romanian students
Full size table

Concluding, options 2, 4 and 5 build on the assumption of more responsibility of higher education institutions and also assume the capacity for self-governance of the sector if the state intervention provides the push and the conditions for collective action. Conversely, options 1, 3 and 6 are more centralized and assume a more active and paternalistic role of the state in its relation to the higher education institutions. Options 2, 4 and 5 could be implemented at the same time as these are complementing each other, but there needs to be consistency in the policy implementation as the main benefits will only be visible on the long-term. The development of a sector-owned benchmarking system and the emergence of a true quality culture enhancing the institutional responsibility will be the main positive achievements on the long run.

3 Conclusions

From the beneficiaries’ perspective, U Multirank seems to directly answer their requirements/needs to have at ease relevant valid data and to make informed decisions. These tools prove to be effective in assisting prospective students to select their academic path and decide between different higher education institutions offers, contributing to a broad movement for a greater transparency and accountability (Hazelkorn 2012).

Mainstreaming the use of transparency tools for Romanian higher education system faces however various challenges, as indicated by our analysis of possible scenarios. The table below highlights some of the main advantages of accountability and transparency tools, with specific examples of challenges, from institutional-administrative, academic, social and financial perspective.

Table 2 Examples of the advantages and risks of implementing accountability and transparency tools in Romania
Full size table

Moreover, as suggested by Miroiu (2010), “Romanian higher education currently exhibits a variety of internal differences. Yet there are, at the same time, strong incentives towards the homogenization of existing HEIs on several dimensions. To the extent to which institutional diversity is desirable, one must first define the dimensions of diversification while avoiding, at the same time, absolute dimensions or the reduction of the entire process to a single type of diversification. The public policies which may be advanced and then implemented by the authorities have a number of available instruments to reach their objectives. Each of these instruments has its specific virtues and drawbacks and any efficient policy application able to limit the range and impact of unintended consequences must consider them”

Various transparency and accountability tools in higher education system were developed in recent years, in particular related to accreditation, quality assurance and national ranking for resource allocation. However, the state was the main driver of these developments, universities having so far limited initiatives and experiences of independent use outside specific public policies. Having the comparison of characteristics and uses of transparency instruments as a reference (Hazelkorn 2011), benchmarking seems to be, in particular, underdeveloped and could offer universities a space for cooperation in the near future.

At institutional level, benchmarking added value is linked with the instrument capacity of using institutional comparisons processes for assisting strategic decisions and with the voluntary dimension of the approach. It provides a framework both for systematic comparison of practices and performances with peer institutions and for systematic cooperation in specific areas, based on mutual needs and interests.

The current context provides a good opportunity for a wide use of this instrument, since both educational authorities and universities themselves are still in search for effective and efficient transparency and comparative approaches, formats and mechanisms. It could directly contribute to enhance quality, performance and competitiveness and also create a community of practice of institutions jointly sharing data, best practice, peer-to-peer development and mentoring (Labi 2011; Hazelkorn 2011). As a bottom-up process and not a top to down, it could lead to the creation of a group of peer institutions that, at a later stage could be the main drivers of mainstreaming accountability and transparency tools within higher education system.

This approach could be particularly relevant in Romania, where for the last decades the policy process was ‘suppressed’ by the uncontested dominance of regulatory tools and very diffuse presence of real impact assessments. Impact evaluation and the availability/use of transparency tools on a large scale could enhance the evaluative culture in higher education institutions, stimulate decision-makers towards more evidence-based processes and bridge this missing link of the between policy process stages and actors, namely evaluation.

The international scene offer insightful examples on how trustworthy data exchange directly impact the strategic decision-making and promote innovative tools for boosting initiatives for higher education modernisation and competitiveness (i.e. example of the Canadian universities partnership). U-multirank demonstrates the wide area that could form the basis of the cooperation, guided by the diversity of higher education missions and activities and is fully compatible with this approach. As one of the core design principles is the peer-group comparability whereby higher education institutions of similar mission are looked at together, benchmarking could offer a concrete framework for Romanian universities to accurately diagnose bottlenecks and fundamental restructure in a continuous improvement process.