1 Introduction

Since EURECA-PRO was established in late 2020, a variety of educational offerings have been developed and implemented by the partner universities. This includes new study programmes at Bachelor’s and Master’s level as well as credited modules and courses on Responsible Consumption and Production (RCP), European values and culture, European languages, and others. Most of these actions were implemented within, at least partially, new conditions, considering the participation of the seven partner universities and the need to find common solutions for special issues resulting from different national rules and regulations. That is why for TU Bergakademie Freiberg, as the Education Lead in EURECA-PRO Phase 1, it was particularly important to create a basic Quality Assurance (QA) framework for educational offerings and to evaluate the educational programmes and activities during and after the implementation. For this purpose, and based on investigation and evaluation data, a number of documents on the evaluation process and their results were completed during this phase including analysis of existing conditions, possible common standards and document templates. QA development including evaluation is a continuous process. At this time, during the transition to the second phase of EURECA-PRO, we summarise European QA standards in education and our experience in implementation in the EURECA-PRO Alliance.

2 EURECA-PRO Educational Concept and Offerings

To realise the end goal of a multinational and multilingual European University that offers among other activities undergraduate education, EURECA-PRO has developed a stepwise approach to the implementation of study programmes. This acknowledges the difficulties faced by many partners in developing new joint programmes, especially in a short time period and with strict regulatory environments, whilst allowing for progress towards this final goal by partners that have greater study programme freedom.

An essential achievement resulting from three years of cooperation in the alliance is the joint programme at master’s level entitled International Master in Responsible Consumption and Production, developed during the first project phase and led by Montanuniversität Leoben (Austria) with TU Bergakademie Freiberg (Germany) and the University of Leon (Spain).

Other RCP programmes individually realised at partner universities are designed as the basis of future joint degrees within the EURECA-PRO alliance. These new programmes aim to cut across topics and disciplines to provide a holistic series of courses on RCP, languages, and European values and culture. New programmes are designed with student mobility options and requirements from the start and form integral aspects of the programme of studies. However, these programmes are still based at and managed by a single partner university (see Table 1).

TABLE 1 Summary of Bachelors and Masters’ programme implemented by EURECA-PRO partner universities
Full size table

The aim of all partners in EURECA-PRO is to engage in joint educational activity to develop new study programmes in RCP which will go on to form the basis of double, triple, and eventually fully joint degrees, with a single certificate (plus supplements) issued upon graduation. For some programmes, mobility to partner universities in the form of “tracks” will be mandatory for all students at some point during their studies. For other programmes, mobility will be optional, but highly encouraged. Tracks are defined as a series of courses and modules (mandatory, electives, and free electives) that make up a semester’s worth of higher education (normally 30 ECTS) on a particular RCP topic. Tracks allow student choice whilst ensuring that students are provided with a joined up educational experience and guidance in defining their own, personal education path. Partners can offer tracks in their national languages or in English.

Within EURECA-PRO a number of modules and courses have been developed that supplement the study programme offerings of the European University. These modules provide students the opportunity to educate themselves on the challenges of United Nations Sustainable Development Goal 12 (SDG12), not only students who are taking part in EURECA-PRO Bachelor’s or Master’s programmes but all alliance and society members. Educational activities are offered on RCP, European Values and Culture, for development of language, digital and presentation. These are the credited study modules, also in the format of Lecture Series, Summer Schools, Challenge-based interdisciplinary courses, and research internship developed according to European quality standards that can be integrated in the curricula and recognised by every partner university.

2.1 European QA Standards for Programme Evaluation Development

The criteria EURECA-PRO applies to the evaluation of study programmes, tracks, and modules are given in the document “Standards and Guidelines for Quality Assurance in the European Higher Education Area” [1], which forms a basis for “the European Approach for Quality Assurance of Joint Programmes” [2], adopted by the ministers responsible for higher education in the European Higher Education Area in 2015. To develop QA standards for EURECA-PRO educational offerings, the criteria for internal QA as well as the results of the European accreditation reports for study programmes [3] were analysed and summarised with the most essential criteria defined.

2.1.1 For Governance Processes:

  • A jointly agreed QA system through development of common educational offerings should be applied, with document subject to formal institutional approval processes;

  • Data on students’ evaluation results (both course and programme) should be collected regularly on a comparable level at all partner universities based on a joint systematic QA system for collection of information;

  • The admission requirements and selection procedures should be fair and align with the programme’s level and discipline;

  • Recognition of qualifications and periods of study (including recognition of prior learning) should be applied in line with the Lisbon Recognition Convention and subsidiary documents [4];

  • Agreed fair and clear procedures for dealing with students’ complaints.

The implementation of a student-centred learning and teaching approach requires special attention to EDI polices in education including consideration of the diversity of students, their differing needs, different cultural background, study requirements in a foreign country, and in a foreign language environment. The evaluation process involves constant interaction and dialogue with stakeholders around the process; students, university management, teachers and staff.

Special attention should be paid to the career path of the graduates, including the evelopment of alumni associations to ensure that graduates have the best start in professional life and have positions for which the study programme aims. Constantly adaptation of these aims is needed relative to requirements of labour markets and occupational standards in the respective fields.

2.1.2 For the Structure of Study Programme:

  • Compliancy with overall programme objectives and intended learning outcomes;

  • Definition of the expected student workload, e.g. in ECTS;

  • Description of module objectives in the individual module descriptions, full integration of the curriculum, presentation of teaching content and definition of learning outcomes;

  • Alignment of the intended learning outcomes with the examination regulations and assessment, applied consistently across partner institutions.

2.1.3 For the Teaching Process:

  • Adaptation of teaching and learning methods to meet societal challenges, especially relative to digital learning;

  • Development of co-teaching activities;

  • Appropriate qualifications, professional and international experience of teaching and technical staff;

  • An interdisciplinary approach in applying challenge-based teaching methods;

  • Support of the development of intercultural competences, transversal skills including multilingualism;

  • Monitoring the achievement of objectives set in the programme and continuous improvement to meet societal needs;

  • Regular revising programmes, involving students and other stakeholders, to analyse their content in relation to cutting-edge research in the given discipline ensuring maximum programme relevance.

2.1.4 For Students

  • The sufficient autonomy of the student, alongside guidance and support;

  • Appropriate students’ workload for progression;

  • Effective procedures for assessment of students;

  • Meeting the student expectations, needs and satisfaction in relation to the programme;

  • Effective learning environment and support services.

3 National Quality Standards in Education of EURECA-PRO partners

EURECA-PRO partner universities use different approaches to evaluate their study programmes. To investigate the system of accreditation and partner universities, a survey was conducted that sought to answer the following questions:

  1. 1.

    At what level (local, regional, national etc.) are programmes approved?

  2. 2.

    What indicators are used to evaluate study programmes?

  3. 3.

    Who takes part in the evaluation process?

3.1 At What Level Are Programmes Approved?

A significant degree of difference exists in how study programmes are approved at EURECA-PRO partner universities. Some partner universities require no approval from government ministries or other bodies to commence new study programmes, with faculty or rectorate boards. In other partner universities, government ministries and agencies are highly involved in the approval of study programmes.

Formal accreditation of programmes is by and large conducted within legal frameworks either defined at the regional or national level. Evaluation of programmes is commonly conducted within university regulations, but some partners also need to refer to national laws and regulations.

3.2 What Indicators Are Used to Evaluate Study Programmes?

What programme performance and quality indicators each partner university uses in measuring success is provided in Table 2.

TABLE 2 Summary of the performance and quality indicators used by EURECA-PRO partner universities in assessing study programmes
Full size table

All, or almost all, partner universities use the number of students joining a programme, the number of students who successfully complete their studies, the student dropout rate, the length of time a student needs to finish a study programme. Students are expected to pass examinations to progress their studies, a majority of partner universities use student exam pass rate data in evaluating their programmes. To ensure that their education is as inclusive as possible for all students, metrics of student diversity are collected by a small majority of partner universities. Similarly, a majority of partners also use the international student ratio to evaluate their programmes. The analysis of other indicators used across EURECA-PRO partner universities reveals that the majority of partner universities use almost all of the indicators defined.

3.3 Who Is Involved in the Evaluation Process?

The process of study programme evaluation is performed by different individuals at different stages. For example, a student may be asked for formative opinions, lecturers could be required to implement an evaluation of their module, or programme directors to prepare internal reports for overall evaluation. Table 3 summarises the results of different stakeholders in the evaluation process across EURECA-PRO partner universities.

TABLE 3 Stakeholders engaged during the evaluation process by EURECA-PRO partner universities
Full size table

The majority of the partner universities of EURECA-PRO makes use of all the identified stakeholders during the evaluation process. A small number of partners do not involve programme study deans, rectorates, or government authorities/ministries in this process at all based on their local evaluation framework. Students are commonly involved as participants in generating evaluations of individual courses. Graduate students are also commonly asked to provide feedback on their experiences during their studies and alumni networks to investigate graduate job prospects.

Professors or lecturers form part of the evaluation process at all partner universities, although they have no consistent role in evaluation. In some cases, they work within internal university structures such as education management teams or they directly participate in external evaluation procedures. The role of Study Deans is similarly varied, with not all partner universities having a defined role of Deans of Studies within their structure. In some cases, study deans are responsible for day-to-day programme evaluation or involved in annual evaluations. Not all partner universities have a role of rectorate within their structure or do not assign programme evaluation tasks to them. Where there is rectorate input, their role is focused on programme approval or programme accreditation. The rectorate role is commonly not concerned with assessing study outcomes but in monitoring student satisfaction in the general learning environment.

External experts are used by all partner universities as part of the evaluation process of study programmes. Some partners use external experts to review university-produced evaluations, within advisory boards or to verify information provided to a national agency. In some cases, external experts have multiple roles within the evaluation process such as providing feedback as an external reviewer or as review committee members. Other stakeholders involved in the evaluation process include the Board of Governors or university Senates, which usually set the terms of the evaluation process or define final programme success. New EURECA-PRO joint programmes aim to be externally accredited by relevant EU authorities.

4 EURECA-PRO Evaluation of Semesters

Using the above analysis and details of the European Higher Education Area evaluation concept, a pathway to EURECA-PRO evaluation has been planned. This evaluation process is summarised in Table 4. This evaluation process will be conducted in three cycles that best capture the educational offering of EURECA-PRO, at the programme level, at the semester level, and at the course/module level.

  1. 1.

    Cycle 1: Whole programme evaluation. This assessment will judge the overall success of the study programme. The two key stakeholders at this cycle level will be the programme (lead) study dean and the students themselves. External experts may also be involved in this evaluation cycle.

  2. 2.

    Cycle 2: Semester evaluation. This assessment will judge the success of a single track of RCP education. The two key stakeholders at this cycle level will be the semester study dean and the students.

  3. 3.

    Cycle 3: Course/module evaluation. This assessment will judge the success of a single course or module offered by EURECA-PRO, whether forming part of track or as a standalone educational offering. The two key stakeholders at this cycle level will be the course director(s) and the students.

TABLE 4 Summary of preliminary evaluation procedure for EURECA-PRO programmes, semesters and individual courses/modules
Full size table

Four key sets of stakeholders have been defined in this evaluation process, programme study deans, semester study deans, course directors, and students. Each EURECA-PRO programme has a designated study dean (or programme coordinator) who will be responsible for performing an evaluation of the whole programme. As mentioned above, a key part of the EURECA-PRO educational offering will be the offering of tracks. At partner universities offering tracks, study deans will be defined who have similar responsibilities for the tracks as programme deans have for the programmes as a whole. Education during a semester is delivered in a series of courses and modules by professors, lecturers, and other educators who have been summarised as “course directors”. These stakeholders are the ones that are most in contact with the students and are responsible for planning and delivering the learning outcomes. Students, as the ultimate consumers of teaching and learning activities, are key stakeholders and thus are important to engage in the evaluation process at all levels.

The evaluation process is designed to produce metrics which are easily compared between partner universities. Educators, study directors, and students will be involved in each of these cycles as appropriate. The majority of these evaluation criteria are self-explanatory and already in use by partner universities. A distinct EURECA-PRO criterion that is included for the first time is whether sustainable development principles in educational practice are met. This question consists of two parts. Firstly it considers if the learning content has been designed and implemented through the lenses of the sustainable development goals, particularly SDG12. Secondly it considers what efforts have been made to reduce the impact of the teaching experience itself, for example by reducing paper usage or teaching in energy efficient rooms. In the general comments section, stakeholders are encouraged to record their general observations, experiences, and feelings related to the course, semester, or programme as a whole. The exact data used in compiling these metrics is subject to future discussion and agreement between partner universities.

5 Conclusion

Through analysing current EU evaluation criteria given in various EU documents (particularly “Standards and Guidelines for Quality Assurance in the European Higher Education Area” and “the European Approach for Quality Assurance of Joint Programmes”), and through assessing the current evaluation processes at partner universities, a new EURECA-PRO evaluation framework has been reached. Of these criteria, many are currently already used by partner universities. Others, such as EDI policies, will be the focus of effort in the next phase of EURECA-PRO, with the ultimate aim of unifying quality standards as a prerequisite for developing joint programmes of highest quality. However, this evaluation framework does not represent the end state of the European University. Evaluation is an ongoing process, and as EURECA-PRO develops new study programmes, modules, and courses in its second phase, and as EU authorities update relevant criteria, EURECA-PRO will continuously update this evaluation framework to meet all new requirements.