Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

2.1 Introduction

One of the main goals of the projects MetaMath and MathGeAr was to conduct a range of comparative case studies that would allow the consortium to understand the set and the magnitude of differences and commonalities between the ways mathematics is taught to engineers in Russia, Georgia, Armenia and in the EU. Equipped with this knowledge the consortium was able to produce recommendations for modernisation of existing Russian, Georgian and Armenian mathematics courses during the project.

However, before such comparative analyses can even begin, one needs a clear set of criteria that would determine the nature and the procedure of the planned comparison. Identification of these criteria was the goal of this methodology development. The results of this activity are described in this chapter.

The methodology has been developed by the EU partners (especially, Saarland University and Tampere University of Technology) in consultation with Russian, Georgian and Armenian experts. Dedicated methodology workshops have been organised by the Association for Engineering Education of Russia in the Ministry of Education and Science in Moscow, Russia, and in the National Center for Educational Quality Enhancement in Tbilisi, Georgia, in June, 2014. During these workshops, the first draft of the comparison criteria has been discussed with the partners and invited experts. It has been amended in the future versions of the Methodology to better match the realities of the university education in Russia, Georgia and Armenia and provide a more objective picture of it with regards to mathematics for engineers. Also, during the workshop, the SEFI Framework for Mathematical Curricula in Engineering Education [1] has been introduced to the Russian, Georgian and Armenian partners as the unified instrument to describe and compare the content of the corresponding courses.

2.2 Criteria for Conducting Comparative Case Studies

2.2.1 University/Program Profile

When comparing courses, it is not enough to choose courses with similar titles. The goals and the purpose of the courses should align and the focus of the courses and their roles within the overall curricula should be comparable. Even the character of the university where a course is offered can make a difference. A classic university and a university of applied sciences can have very different perspectives on what should be the key topics within courses with the same name. In a larger university a professor can have a much richer set of resources than in a smaller one; at the same time, teaching a course to several hundreds of students puts a much bigger strain on a professor than teaching it to several dozens of them. The overall program of studies that the target course is part of is equally important for similar reasons. Therefore, the first set of criteria characterising a course profile focus on the general description of the university and the program (major) where the course is taught. These parameters include:

  • Criterion A: University profile

    • Classic or applied

    • Overall number of students

    • Number of STEM disciplines

    • Number of STEM students

  • Criterion B: Program/discipline profile

    • Theoretical or applied

    • Number of students

    • Role/part of mathematics in the study program

2.2.2 Course Settings

The next set of criteria describes the context of the course including all its organisational settings and characteristics not directly related to pedagogical aspects or the content. This is the metadata of the course, which allows us to easily identify whether the two courses are comparable or not. For example, if in one university a course is taught on a MSc level and in another on a BSc level, such courses are not directly comparable, because the levels of presentation of the course material would differ much. If in one university a course’s size is 3 ECTS credits and in the other 7 ECTS, such courses are not the best candidates for comparison either, because the amount of work students need to invest in these two courses will be very different even if the titles of the courses are similar. Sometimes, we had to relax some of these conditions if for particular universities best matches cannot be found. The complete list of course characteristics include:

  • Criterion C: Course type

    • Bachelor or Master level

    • Year/semester of studies (1/2/…)

    • Selective or mandatory

    • Theoretical/applied

  • Criterion D: Relations to other courses in the program

    • Prerequisite courses

    • Outcome courses

    • If the course is a part of a group/cluster (from which it can be selected), other courses in the group

  • Criterion E: Department teaching the course

    • Mathematical/graduating/other

  • Criterion F: Course load

    • Overall number of credits according to ECTS regulations

    • Number of credits associated with particular course activities (lectures/tutorials/practical work/homework/etc.)

2.2.3 Teaching Aspects

In order to describe how the teacher organises the course, we identify three important criteria: use of any particular didactic approach (such as project-based learning, inquiry-based learning, blended learning, etc.), organisation of course assessment (how many tests and exams, what form they take, how they and the rest of the course activity contribute to the final grade) and the resources available to a teacher—from the help of teaching assistants to the availability of computer labs. Teaching aspects include:

  • Criterion G: Pedagogy

    • Blended learning

    • Flipped classroom

    • MOOC

    • Project-based learning

    • Inquiry-based learning

    • Collaborative learning

    • Game-based learning

  • Criterion H: Assessment

    • Examinations (how many, oral/written/test-like)

    • Testing (how often)

    • Grade computation (contribution of each course activity to the final grade, availability extra credits)

  • Criterion I: Teaching resources

    • Teaching hours

    • Preparatory hours

    • Teaching assistants (grading/tutorials)

    • Computer labs

2.2.4 Use of Technology

One of the aims of the MetaMath and MathGeAr projects is to examine and ensure the effective use of modern ICT in math education. Therefore, a dedicated group of criteria has been selected to characterise the level of application of these technologies in the target courses. There are two top-level categories of software that can be used to support math learning: the instruments that help students perform essential math activities and the tools that help them to learn mathematics. The former category includes such products as MATLAB, Maple, Mathematica, or SPSS. These are, essentially, the systems that a professional mathematician, engineer or researcher would use in everyday professional activity. Using them in a course not only helps automate certain computational tasks but also leads to mastering these tools, which is an important mathematical competency on its own. The latter systems are dedicated educational tools. They help students understand mathematical concepts and acquire general mathematical skills. In MetaMath and MathGeAr projects, we apply a particular tool like this—an intelligent education platform Math-Bridge. In both these categories, the number and diversity of available systems is very large. The focus of the criteria in this set is to detect whether any of these systems are used and to what degree, namely what the role in the course is.

  • Criterion J: Use of math tools

    • Name of the tool(s) used (MATLAB, Maple, MathCAD, Mathematica, SPSS, R, etc.)

    • Supported activities (tutorials, homework)

    • Overall role of the tool (essential instrument that must be learnt or one way to help learn the rest of the material)

  • Criterion K: Use of technology enhanced learning (TEL)-systems

    • Name and type of tool used (Geogebra—math simulation; STACK—assessment software; Math-Bridge—adaptive learning platform; etc.)

    • Supported activity (assessment, homework, exam preparation)

    • Role on the course (mandatory component/extra credit opportunity/fully optional supplementary tool)

2.2.5 Course Statistics

Another important aspect of the course is the data collected about it over the years. It shows the historic perspective and evolution of the course, and it can also provide some insights into the course’s difficulty and the profile of a typical student taking a course. Although by itself this information might be not as important for course comparison, combined with other criteria it can provide important insights.

  • Criterion L: Course statistics

    • Average number of students enrolled in the course

    • Average percentage of students successfully finishing the course

    • Average grade distribution

    • Percentage of international students

    • Overall student demographics (gender, age, nationality, scholarships, etc.)

    • Average rating of the course by students

2.2.6 Course Contents

Finally, the most important criterion is the description of the learning material taught in the course. In order to describe the content of the analysed courses in a unified manner that would allow for meaningful comparison we needed a common frame of reference. As the context of mathematical education in this project is set for engineering and technical disciplines, we have decided to adopt a “Framework for Mathematics Curricula in Engineering Education” prepared by the Mathematics Working Group of the European Society for Engineering Education (SEFI) [1]. This report is written about every 10 years; and the current edition formalises the entire scope of math knowledge taught to engineering students in EU universities in terms of competences. The competencies are broken into four levels, from easier to more advanced, and allow for composite representation of any math course. As a result, every course can have its content described in terms of atomic competencies and two similar courses can easily be compared based on such descriptions.

  • Criterion M: Course SEFI competency profile

    • Outcome competencies of the course (what a student must learn in the course)

    • Prerequisite competencies of the course (what a student must know before taking the course)

2.3 Application of the Criteria for the Course Selection and Comparison

This set of criteria should be used (1) for selecting appropriate courses for the comparison and (2) for conducting the comparison itself. During the selection process, Criteria A and B will ensure that only universities and study programs with matching profiles are selected. Criteria C, D, E and F will help to select the courses that correspond in terms of their metadata. Criteria G, H and I will help to filter out courses that utilise unconventional pedagogical approaches or differ too much in terms of assessment organisation and teaching resources available.

At this point, if a pair of courses passed the screening, they can be safely compared; all criteria starting J contribute to the comparison. One needs to note that, in some cases, the strict rules of course selection might not apply, as a particular partner university sometimes presents a very unique case. In such situations, the selection rules can be relaxed.