Pre-service secondary teachers’ pedagogical content knowledge for the teaching of mathematical modelling

Abstract

The article deals with the pedagogical content knowledge of mathematical modelling as part of the professional competence of pre-service teachers. With the help of a test developed for this purpose from a conceptual model, we examine whether this pedagogical content knowledge can be promoted in its different facets—especially knowledge about modelling tasks and about interventions—by suitable university seminars. For this purpose, the test was administered to three groups in a seminar for the teaching of mathematical modelling: (1) to those respondents who created their own modelling tasks for use with students, (2) to those trained to intervene in mathematical modelling processes, and (3) participating students who are not required to address mathematical modelling. The findings of the study—based on variance analysis—indicate that certain facets (knowledge of modelling tasks, modelling processes, and interventions) have increased significantly in both experimental groups but to varying degrees. By contrast, pre-service teachers in the control group demonstrated no significant change to their level of pedagogical content knowledge.

Introduction

The importance of mathematical modelling for mathematics education is internationally accepted (Kaiser, 2020). In this article, we deal with the teaching of mathematical modelling in secondary education (see Blum, 2002, p. 161 Issue 3b). Besides studies investigating how mathematical modelling can be taught at school (Kaiser, 2020; Schukajlow, Achmetli, & Rakoczy, 2019), Hattie (2003) demonstrated that 30% of the variance in student performance can be explained by the knowledge, actions, and beliefs of their teachers. We therefore focus on teacher education and investigate the development of pre-service teacher pedagogical content knowledge as part of professional competence for teaching mathematical modelling. We use the term pre-service teacher, in referring to those training to become teachers and who are presently doing a master’s degree in education.

The concept of professional competence encompasses activities and tasks required to meet professional demands. In general, professional competence entails “knowledge, technical skills, …, emotions, values, and reflection” (Epstein & Hundert, 2002, p. 235). For the purposes of this article, competence is interpreted as an inherently acquirable and context-dependent disposition of cognitive performance (Klieme, Hartig, & Rauch, 2008). This concept of competence can also be considered in the sense of action competence (Weinert, 2001) and therefore also includes motivational, metacognitive, and self-regulatory features, which are regarded as decisive for the willingness to act. Thus, like Baumert and Kunter (2013), we approach professional competence as an interaction between various factors, including “specific declarative and procedural knowledge […], professional values, beliefs, and goals, motivational orientations, professional self-regulation skills” (Baumert & Kunter, 2013, p. 28).

We do not focus on professional competence in general but consider facets of professional competencies for teaching mathematical modelling (Blum, 2015). In particular, we focus on the extent to which suitably designed seminars can promote the pedagogical content knowledge of pre-service teachers for teaching mathematical modelling, as well as the extent to which modelling seminars with different emphases (on tasks or on adaptive intervention) can promote the various facets of their pedagogical content knowledge.

In the theoretical background, we describe the pedagogical content knowledge for the teaching of mathematical modelling, starting from the students’ modelling competence. We focus on using the “comprehensive concept of modelling competencies based on sub-competencies and its evaluation” (Kaiser & Brand, 2015, p. 135), which takes into account an atomistic approach; but we also consider the holistic view of students’ modelling processes (Blomhøj & Jensen, 2003). This leads to the research questions in which type of modelling seminar this special pedagogical content knowledge can be promoted. The presentation of the results is followed by a detailed discussion and a conclusion.

Theoretical background

To promote pedagogical content knowledge for the teaching of mathematical modelling, we first present our understanding of student modelling competence. The understanding of such competence essentially forms the basis for the conception and investigation of pedagogical content knowledge for teaching mathematical modelling. Subsequently, based on professional competences, we explain how we access professional competences for teaching mathematical modelling in general, the central aspect of which is the associated pedagogical content knowledge.

Students’ modelling competence

By modelling competence, we mean “the ability to identify relevant questions, variables, relations or assumptions in a given real-world situation, to translate these into mathematics and to interpret and validate the solution of the resulting mathematical problem in relation to the given situation” (Niss, Blum, & Galbraith, 2007, p. 12). This is not limited to abilities and also includes their reflected use in life and the willingness to use these abilities independently (Blomhøj & Jensen, 2003, p. 126; Maaß, 2006, p. 116). Modelling competence can be seen in a comprehensive overall concept of competences (Blomhøj & Jensen, 2007).

In almost all approaches, the modelling process itself is represented as a cycle comprising different steps or phases (Kaiser, 2020). The ability to carry out a sub-process in the modelling cycle can be regarded as a specific sub-competency of mathematical modelling (Kaiser, 2007; Maaß, 2006). The term competence is used here in a broader sense, whereas (sub-)competency refers to the different constituents of competence (Blömeke, Gustafsson, & Shavelson, 2015). Depending on the modelling cycle under consideration, one can describe different sub-competencies of modelling. Following an approach from Kaiser (2007, p. 111) and Maaß (2006, p. 116), one arrives at the sub-competences of understanding, simplifying, mathematizing, working mathematically, interpreting, validating, and exposing (Greefrath, Kaiser, Blum, & Borromeo Ferri, 2013, p. 19).

In addition to the sub-competencies described above, various studies indicate the importance of metacognitive competences in modelling processes (Kaiser, 2007; Stillman, 2011; Vorhölter, Krüger, & Wendt, 2019; Zöttl, Ufer, & Reiss, 2010). Furthermore, it is known that students typically do not proceed through the modelling cycle in the order given but choose individual modelling routes (Borromeo Ferri, 2018, p. 30). Various test instruments already exist for measuring modelling competencies, which not only assess modelling competence globally but also focus in part on the various sub-competencies (Haines & Crouch, 2001; Hankeln, Adamek, & Greefrath, 2019; Hankeln & Greefrath, 2020; Kaiser & Brand, 2015; Zöttl, Ufer, & Reiss, 2011). In the next section, we therefore focus on the professional competence of teachers and especially pedagogical content knowledge as a basis for acquiring such knowledge for teaching mathematical modelling.

Pedagogical content knowledge of mathematics teachers as part of professional competence

The professional competencies of mathematics teachers and pre-service mathematics teachers have been investigated in detail (TEDS-M study by Blömeke, Hsieh, Kaiser, & Schmidt, W. H. (Eds.)., 2014; COACTIV study by Kunter et al., 2013). There are some key similarities in the concepts of professional competence developed from the COACTIV and the TEDS-M studies. In particular, both concepts of professional competence consider professional knowledge to be comprised of different knowledge areas: content knowledge, pedagogical content knowledge, and pedagogical/psychological knowledge (Shulman, 1986). Alongside these cognitively oriented knowledge dimensions, both approaches also consider professional competence to include affective/value-oriented aspects.

Besides the conceptions of Baumert and Kunter (2013) and Blömeke et al. (2014), there are various other conceptions of pedagogical content knowledge for mathematics teachers, some of which also encompass content knowledge and pedagogical knowledge (Depaepe, Verschaffel, & Kelchtermans, 2013). For example, Ball, Hill, and Bass (2005) focus on mathematical knowledge for teaching (MKT) (Ball et al., 2005, p. 17), which consists of two components. These are general mathematical knowledge that any well-educated adult should possess and mathematical knowledge for teachers that is especially important in the context of teaching. Thus, mathematical knowledge for teaching (MKT) contains references to both content knowledge and pedagogical content knowledge in the sense of Shulman (1986). Even though these diverge, they share the same core of pedagogical content knowledge (Krauss, Baumert, & Blum, 2008, p. 878). In almost every modern conception and study on pedagogical content knowledge, this shared core consists of identifying student (mis)conceptions and difficulties (see also Kaiser, Schwarz, & Tiedemann, 2010, p. 441) and knowledge about teaching strategies and representations (Depaepe et al., 2013, p. 16). In any event, the acquisition of pedagogical content knowledge is widely established as a key objective of teacher education (Peressini, Borko, Romagnano, Knuth, & Willis, 2004, p. 73). Although such knowledge is only one aspect of professional competence, it does appear to be important. Teachers who have high pedagogical content knowledge employ tasks with high potential for cognitive activation and offer good support for individual learning (Krauss et al., 2008; Kunter & Baumert, 2013).

Pedagogical content knowledge for the teaching of mathematical modelling

Pedagogical content knowledge for promoting modelling competences has been investigated by several researchers. Particularly the teaching of mathematical modelling requires specific pedagogical content knowledge. Future teachers should “develop a comprehensive understanding of modelling and its pedagogical value, future teachers need appropriate knowledge and competencies in mathematics, mathematics pedagogy and general pedagogy” (Kaiser et al., 2010, p. 433). Pedagogical content knowledge is the central factor for determining the cognitive activation potential of a lesson (Baumert & Kunter, 2013). This is also important, as Kuntze (2011) shows, given indications that teachers do not have an extensive understanding of meta-knowledge in mathematical modelling.

Accordingly, Borromeo Ferri and Blum (2010) presented four theoretically derived, content-related competence dimensions. Each is concretized by knowledge and ability facets that relate to declarative and procedural aspects of knowledge (see Table 1).

Table 1 Content-related competence dimensions of teaching mathematical modelling (Borromeo Ferri, 2018; Borromeo Ferri & Blum, 2010)

The theoretical model of Borromeo Ferri and Blum (2010) covers a wide range of teacher competencies required for mathematical modelling. For example, the diagnostic dimension also includes meta-metacognition, which refers to the monitoring and support of metacognitive processes of learners (Stillman, 2015) and modelling-oriented noticing (Galbraith, 2015). We therefore use this model, adapt it for pre-service teachers, and focus more on models of professional knowledge (Baumert & Kunter, 2013; Blömeke & Kaiser, 2014), in order to develop a test instrument and a modelling seminar on this theoretical basis. The authors’ preliminary investigations (Klock, Wess, Greefrath, & Siller, 2019; Wess, Klock, Siller, & Greefrath, 2021) suggest that pedagogical content knowledge has a four-dimensional structure (see Fig. 1).

Fig. 1

Modelling-specific pedagogical content knowledge (Wess et al., 2021)

Knowledge about interventions is a facet of the teaching dimension (Borromeo Ferri & Blum, 2010). Suitable adaptive teacher interventions, according to the principle of minimum assistance and various levels of teacher interventions, are used to evaluate support for modelling processes (Leiß & Wiegand, 2005). Such interventions promote independent work by learners and support the metacognitive competencies of students. This includes the characteristics of different intervention concepts (Tropper, Leiss, & Hänze, 2015; van de Pol, Volman, & Beishuizen, 2010; van de Pol, Volman, Oort, & Beishuizen, 2014) and adaptive interventions in the context of modelling days (Kaiser & Stender, 2013; Stender, 2019).

Knowledge about modelling processes is characterized by specific diagnostic knowledge on students’ modelling competences (Kaiser, 2007; Maaß, 2006), which is highly relevant to the learning processes of their students (Brunner, Anders, Hachfeld, & Krauss, 2013). For example, when diagnosing modelling processes (Blum & Leiß, 2007), pre-service teachers should focus on identifying the modelling phase (Maaß, 2006) in which the learner is currently operating (Borromeo Ferri & Blum, 2010). Various studies have also identified specific difficulties in the modelling process of each modelling phase (e.g., Galbraith & Stillman, 2006).

The knowledge and ability to analyse, process, and develop modelling tasks are facets of a task-related dimension (Borromeo Ferri & Blum, 2010). It is considered important that pre-service teachers work on the modelling tasks themselves (Tan & Ang, 2013). The comprehensive classification scheme proposed by Maaß (2010) for categorizing and analysing modelling tasks, together with the explanations about task design from Czocher (2017) and knowledge of tasks with multiple solutions (Schukajlow, Krug, & Rakoczy, 2015), provide a theoretical basis for the facets mentioned here and in particular a criteria-based development of modelling tasks that considers references to reality, relevance, authenticity, and openness (Palm, 2007; Siller & Greefrath, 2020; Vos, 2018).

Knowledge about aims and perspectives consists of selected aspects of theoretical background knowledge. This describes knowledge of modelling cycles (Borromeo Ferri, 2006) and the suitability of these cycles for various purposes and various perspectives of research on mathematical modelling (Kaiser & Sriraman, 2006).

Teaching professional competencies for mathematical modelling is widely regarded as a challenge, due to the large number of skills required (Lingefjärd, 2007). In particular, pedagogical content knowledge about mathematical modelling does not directly ensure an understanding of the difficulties experienced by learners in the modelling process. To this end, practical or working experience is required (Kaiser et al., 2010, p. 441).

Research questions

Developing the professional competencies of teachers is a fundamental aim of teacher education. On the one hand, there are already many empirical findings on pedagogical content knowledge from pre-service teachers in general (Blömeke et al., 2014; Kunter et al., 2013). In this respect, pedagogical content knowledge is seen as a central factor in determining the cognitive activation potential of a lesson (Kunter & Baumert, 2013). This can be promoted through practical experience and reflection on one’s own interventions in courses given by experts in the field (Evens, Elen, & Depaepe, 2015). On the other hand, there are clear descriptions of student competencies to be achieved in class (Maaß, 2006) and also concepts of professional competencies of teachers for mathematical modelling (Borromeo Ferri & Blum, 2010; Kaiser et al., 2010).

When designing modelling seminars with practical experience for the acquisition of professional competences, the question arises as to which particular facets of modelling-specific pedagogical content knowledge should be emphasized. While a comparable framework is chosen in the theoretical part of the seminars, different emphases with particular importance for practical experience—based on two facets from the model in Fig. 1—can be applied in the practical part. In one group, we placed particular emphasis on knowledge about interventions and in another group on knowledge about modelling tasks. This raises the question of which seminars at university can promote pedagogical content knowledge for teaching mathematical modelling. We have designed two different seminars and examined why they are suitable for this purpose. Firstly, the difference with respect to a seminar that does not focus on mathematical modelling is of interest. Secondly, the difference between two seminars on modelling with different focal points is to be investigated. This leads to the following research questions (RQ) for our study:

• RQ 1: Do pre-service teachers attending a modelling seminar improve their pedagogical content knowledge for teaching mathematical modelling more than pre-service teachers attending a mathematics didactics seminar on another school mathematics topic?

Mathematical modelling was not discussed in the seminars which had other topics. These were seminars in which the pre-service teachers carried out practical aspects together with students, similar to the modelling seminars, but for the planning and analysis of lessons or the use of digital tools in school.

• RQ 2: Do pre-service teachers attending a modelling seminar with a focus on tasks or adaptive intervention improve various facets of their pedagogical content knowledge for teaching mathematical modelling?

With regard to the research desiderata described above, it can be assumed that intensive work on subject didactic content for mathematical modelling, together with practical experience, leads to an increase in pedagogical content knowledge specific to modelling. Due to the different focus, different levels of knowledge growth within the experimental groups are expected, which deal with mathematical modelling, whereas no substantial changes within the group are expected, in which mathematical modelling is not dealt with.

In answering these research questions, we first show that the structure of modelling-specific pedagogical content knowledge that we are considering may be described by means of a Rasch model.

Method

Sample

Results from previous studies suggest that teachers who are qualified to teach at a grammar school (Gymnasium¹) have higher levels of pedagogical content knowledge than teachers qualified to teach at other school types (Krauss, Brunner, et al., 2008). Since we expect a large part of the pedagogical content knowledge for teaching mathematical modelling to be acquired in the seminar under consideration, we expect the clearest and most significant effects from pre-service teachers qualified to teach at a grammar school (Gymnasium).

To answer the research questions, data were collected from 198 pre-service teachers qualified to teach at a grammar school (Gymnasium) across a total of twelve mathematical pedagogical modelling seminars, with integrated practical components, at two universities from different parts of Germany. At both universities (Koblenz-Landau and Münster), pre-service teachers in the first semesters study mathematics. There are also courses on their second subject and on educational science. They have already attended an introduction or lecture on the didactics of mathematics. The seminars considered here are the first mathematics didactic seminars in the course of their studies. In addition to the first experimental group (“task group”, 4 modelling seminars in Münster, N = 76) and a second experimental group (“intervention group”, 3 modelling seminars in Koblenz, N = 55), data from a baseline group (5 seminars in Münster, N = 67) were also recorded. The baseline group includes only pre-service teachers from the University of Münster and none from the University of Koblenz-Landau. Since the pre-service teachers were recruited through their attendance at seminars, it was not possible to randomly assign the participants to the study. All of them completed both the pre-test and the post-test.

Since the two experimental groups worked at different sites, sex, age, semester, and Abitur (general higher education entrance qualification in Germany) grade were recorded for each candidate in order to assess comparability (see Table 2). The Abitur grades in Germany range from 1 to 4, with 1.0 being the best possible. The differences in semester are primarily explained by structural differences in the mathematics teaching programmes at the two campuses. This should be taken into account in interpreting the results, as well as the differences in average Abitur grades.

Table 2 General information about the sample

Study design

The quasi-experimental study was performed with a pre-post design measuring the modelling-specific pedagogical content knowledge of the participating pre-service teachers. The treatment consisted of a 12-session seminar to pre-service teachers. This seminar on teaching mathematical modelling, with integrated practical elements, was designed in two variants (task experimental group and intervention experimental group) for this study. There was also a baseline group with no thematic link to mathematical modelling (see Fig. 2). The pre-service teachers completed the same test instrument before and after the treatment.

Fig. 2

Study design

Treatment design

The seminars completed by the three groups consisted of 12 sessions and additional blended learning formats. The seminars were divided into a theory-based preparation phase, a practical phase, and a reflection phase (see Fig. 3).

Fig. 3

Treatment design of the seminars

In the theory-based preparation phase of both experimental groups, modelling-specific pedagogical content knowledge according to Fig. 1, i.e., knowledge about interventions, modelling processes, modelling tasks, and goals and perspectives of modelling, was addressed. The differences relate mainly to differences in session content, but there were no differences in the pedagogical approach of the sessions.

The task group paid particular attention to aspects of the knowledge of modelling tasks (characteristics, analyses, development, and multiple solutions of modelling tasks; see Fig. 2). The pre-service teachers worked on a modelling task (“How much hot air fits into the hot air balloon shown”?) and analyse the solutions using a modelling cycle. They developed their own modelling tasks and did not discuss the effects of suitable adaptive interventions. Criteria for suitable modelling tasks were established (in particular, references to reality, relevance, authenticity, and openness), and the pre-service teachers developed a set of corresponding tasks to be used subsequently in the practical phase, as part of a blended learning format with multiple feedback cycles. Finally, criteria and indicators for predefined modelling sub-processes (in particular, simplifying, mathematizing, validating) were identified to enable the learning processes of students to be observed and diagnosed during the practical phase. In this phase, each participant of the task group attended two appointments with students. Over the course of these 90-min project sessions, teams of three pre-service teachers supervised small groups of students, as the latter worked on the self-developed modelling tasks. The teams focused their observations of these processes on the sub-competencies of mathematical modelling displayed by the students and recorded them on a previously prepared observation sheet. The reflective phase of the task group focused on practical experience from the observation sessions with students and its ramifications with regard to self-designed modelling tasks. Finally, the acquired knowledge was used by the pre-service teachers to professionalize their own teaching activities and evaluate the self-developed modelling tasks. They also reflected on and, if necessary, adapted the criteria for suitable modelling tasks established during the preparatory phase. The experience and knowledge acquired were summarized in a term paper. The pre-service teachers did not discuss the effects of suitable adaptive interventions.

In the intervention group, however, suitable adaptive interventions according to the knowledge of interventions (characteristics and effects of suitable interventions; see Fig. 1) were discussed. The pre-service teachers worked in pairs on predefined and complex modelling tasks, but no modelling tasks were developed. The results were then discussed and the potential solution paths and difficulties anticipated. Criteria for adaptive interventions in mathematical modelling (diagnosis-based, minimally invasive, adapted to difficulty, preserving independence) and a corresponding process model were then developed. Two third-party videos on modelling activities of pre-service teachers with students and their transcripts were then analysed in each case. The adaptivity of the interventions was evaluated. In a subsequent practical phase, the pre-service teachers of the intervention group worked in pairs to supervise a small group of students, as they worked on a modelling task that had previously been worked on by the pre-service teachers themselves. The pre-service teachers reflected on their interventions immediately, using a reflection sheet. They were also recorded on video during the practical phase, when they support the students in their work on modelling tasks. The videos and their transcripts were then also analysed in terms of the adaptivity of the interventions and possible alternative courses of action within the framework of the modelling seminar. In the reflection section, they analysed two selected interventions against the background of the criteria of adaptive interventions.

Seminars with the same scope and target group were held in the baseline group. These seminars can be chosen by the students as an alternative to the modelling seminars. They also consist of 12 sessions and a theory-based preparation phase, a practical phase, and a reflection phase. There are various themes on offer (beyond mathematical modelling). These seminars include either planning and analysing lessons or planning, creating, and implementing digital learning environments in mathematics teaching. Here, either a digital learning path with inner-mathematical tasks for derivation was developed or teaching units on the vertex form of quadratic functions were planned. All learning arrangements were tested with students and then reflected upon.

Test instrument

To answer the questions stated above, a test instrument was developed (Klock & Wess, 2018). The items were first developed by the authors. The items of this first version were qualitatively pre-piloted in a small sample (N = 8) of mathematics educators, who specialize in mathematical modelling. The aim was to revise incomprehensible or imprecise items. The items were subsequently optimized by the authors through a process of consensus. The four knowledge dimensions of modelling-specific pedagogical content knowledge (knowledge about interventions, knowledge about modelling processes, knowledge about modelling tasks, knowledge about conceptions/dimensions/aims) were operationalized for piloting with a total of 103 dichotomous test items in multiple-choice and combined-single-choice formats. The items from the areas of knowledge about modelling process and about interventions relate to modelling tasks supplemented by text vignettes about specific working processes of students in order to be able to check these facets authentically (for an example item, see Fig. 4). Following such a text vignette, the pre-service teachers were asked in multiple-choice format which phase of the solution process the students were in, which problem occurred, and which intervention was appropriate (see Table 3: Knowledge about modelling processes). The pilot was conducted in the 2017/2018 winter semester with the data of 156 participants (67% female) studying to be teachers at Gymnasium schools (higher track) or comprehensive schools (middle track) at three universities in Germany (Münster, Koblenz-Landau, and Duisburg-Essen). At the time of data collection, the pre-service teachers were either at the end of their bachelor’s degree (13%) or currently completing a master’s degree (87%). After the pilot, four of the ten vignettes and four items on knowledge about conceptions/dimensions/aims were removed, so that the test ultimately included 71 items. Example items are shown in Table 3 for each scale. The test instrument was used as a pre- and post-test.

Fig. 4

Example of item text vignette (Klock & Wess, 2018, p. 22; Idea by Maaß & Gurlitt, 2011, p. 634)

Table 3 Example of test items for determining the modelling-specific pedagogical content knowledge

Data analysis

The dichotomous items were scaled using a simple Rasch model (Embretson & Reise, 2000, p. 67 ff.; Fischer, 1997). A classical confirmatory factor analysis was not an option for scaling the data, due to the lack of multivariate normal distribution of the data. Since the simple Rasch model does not require such an assumption, it was used to scale the dichotomous data. Based on the response patterns of the participants, the item difficulty parameters and person ability parameters were calculated using a multidimensional random coefficients multinomial logit model (Adams, Wilson, & Wang, 1997) implemented in ACER ConQuest software. To do this, both the pre-test and post-test data were subsumed into a single dataset (virtual persons approach).

Before calculating the model, one item relating to knowledge about interventions, two items relating to knowledge about modelling processes, and two items relating to knowledge about conceptions/dimensions/aims were excluded from the evaluation due to insufficient discrimination. To evaluate the model fit, the theoretically founded four-dimensional model was compared with a one-dimensional model. For the one-dimensional model, all items were loaded on a general factor—the modelling-specific pedagogical content knowledge. In the four-dimensional model, all items were loaded on the knowledge facet that was operationalized in each case (see Fig. 5).

Fig. 5

One-dimensional and four-dimensional models

The model fit was evaluated using the deviation, the AIC (Akaike information criterion), the BIC (Bayesian information criterion), and the correlation between the dimensions (global fit) and the weighted mean squares (local fit). All results indicate a better fit of the four-dimensional model, considering the higher number of parameters. Since the four-dimensional model was found to have a better fit, it was used to scale the data. The scaling analysis confirmed the results previously established by structural equation modelling (Wess et al., 2021).

The personal ability parameters were estimated as Weighted Maximum Likelihood Estimates (WLE; Warm, 1989), based on an eight-dimensional model using the item parameters determined earlier. According to the virtual item approach, a four-dimensional model was used for each of the two measurement points T1 and T2. Table 4 shows the EAP reliabilities of the scales, which can be interpreted in a similar fashion to Cronbach’s α. For almost every scale, the reliabilities are greater than 0.70 and therefore acceptable to good. The low reliabilities at the first measurement point T1 in the scales for knowledge about modelling tasks and knowledge about conceptions/dimensions/aims are presumably explained by a lower variance of personal ability in the pre-test. Since the post-test reliabilities are excellent, we may consider the measurements to be reliable. The WLEs determined at the first and second measurement points were investigated with univariate and multivariate one-way and two-way ANOVA in IBM SPSS Statistics.

Table 4 EAP reliabilities of the scales in the pre-test and post-test

Missing values

The evaluation of missing data showed that the test could be completed by all participants in the time allocated. The missing data were therefore not due to lack of time (not reached) but due to lack of motivation or lack of ability of a test subject (omitted) (Ludlow & O’leary, 1999). Following Mislevy and Wu (1996), these were considered incorrect, since the test design was always aimed at reducing the probability of guessing, and based on a “correct-for-guessing” evaluation, it was assumed that the reason for omitting an item was its perceived difficulty.

Results

The following sections report the development in the facets of the modelling-specific pedagogical content knowledge described above during each seminar. The dimensions of the test model represent the facets of modelling-specific pedagogical content knowledge. First, we present descriptive statistics of the person estimators of the test groups for each facet and measurement points T1 and T2. We then use ANOVA methods to analyse the changes in the test groups over time (Research Question 1) and the differences in these developments between groups (Research Question 2).

The mean values and standard deviations of the ability values observed for each seminar at measurement points T1 and T2 are listed in Table 5. Outliers that deviated from the arithmetic mean by more than three standard deviations were identified in advance, both for each seminar and across all test groups. These values were excluded from the subsequent evaluation.

Table 5 Descriptive statistics of the facets of modelling-specific pedagogical content knowledge

Similar mean values were observed in the test groups across all facets at the first measurement point T1. Multivariate one-way ANOVA only yielded a slightly significant difference between task group and intervention group for the scale “knowledge about modelling tasks” using the Games-Howell post-hoc test (p = .025). For the other scales, there were no significant differences at the first measurement point T1. Prior knowledge about the participants can therefore be considered largely comparable (see Fig. 6).

Fig. 6

Facets of modelling-specific pedagogical content knowledge (T1, measurement point T1; T2, measurement point T2; EG1, task group; EG2, intervention group; BG, baseline group)

Changes within the groups (Research Question 1)

Univariate two-way repeated measurement ANOVA was used to verify to what extent there was an interaction between the group membership and the change in ability over time for each facet of the modelling-specific pedagogical content knowledge. In verifying the model assumptions, it was found that the facets of knowledge about modelling processes and about conceptions/dimensions/aims did not have equal variance at the second measurement point T2; indeed, the Levene test revealed a significant result. However, both cases could be determined using the F_max criterion (F_max = 3.23 and F_max = 2.59) allowing the results to be interpreted.

In each facet, there was a significant interaction effect between group membership and change over time (see Table 6 in the Appendix). For knowledge about conceptions/dimensions/aims, the effect size was small to medium (part. η² = .042)²; for the other facets, the effect size was medium to large (knowledge about interventions: part. η² = .079; knowledge about modelling process: part. η² = .283; knowledge about modelling tasks: part. η² = .204) (Cohen, 1988, p. 368). For the facet of knowledge about conceptions/dimensions/aims, the statistical power was not fully sufficient to prove the small to medium effect. However, the effects of the other facets were demonstrated with high statistical power.

To investigate the effects of the time factor within the groups, univariate one-way repeated measurement ANOVA was performed (see Table 7 in the Appendix). The results show significant effects in the experimental groups in all facets after attending each seminar. The effects in the task group were consistently larger than in the intervention group. No significant effect was found in the baseline group for any facet.

Knowledge about interventions increased in the task group (part. η² = .311) with a large effect, whereas a small to medium effect was observed in the intervention group (part. η² = .112). Knowledge about modelling processes increased in both experimental groups with a large effect (task group: part. η² = .509; intervention group: part. η² = .345). Knowledge about modelling tasks also increased in the task group with a large effect (part. η² = .479) and in the intervention group with a medium to large effect (part. η² = .147). The smallest effects were observed in the facet of knowledge about conceptions/dimensions/aims. In both experimental groups, a medium to large effect (task group: part. η² = .144; intervention group: part. η² = .142) was observed (Cohen, 1988, p. 368). In the task group, the statistical power of the test was excellent. In the intervention group, the statistical power for the knowledge about interventions was too small to prove the effect conclusively. Due to the very small, non-significant effects in the baseline group, the statistical power was again very small in these cases.

Differences between the groups (Research Question 2)

After we report the changes within each experimental group, the differences in development of the facets of modelling-specific pedagogical content knowledge between groups are discussed below. Using post-hoc tests, multivariate one-way ANOVA yielded the differences between the groups. When homogeneity of variance was satisfied between the groups, the Bonferroni correction was used to adjust the α-error level. When homogeneity of variance was not satisfied, the Games-Howell correction was performed.

In the facets of knowledge about interventions and knowledge about modelling processes, both experimental groups showed significant differences relative to the baseline group in the post-test (see Table 8 in the Appendix). However, there were no significant differences between the experimental groups for either of these facets. For the facet of knowledge about modelling tasks, there were significant differences between all test groups. For the facet of knowledge about conceptions/dimensions/aims, there was only a significant difference between the task group and the baseline group.

Discussion

Changes within the groups (Research Question 1)

The results show that, in both experimental groups, suitably designed modelling seminars successfully promoted the pedagogical content knowledge of pre-service teachers for teaching mathematical modelling in all four facets, with medium to large effect sizes (Research Question 1). There was only a small to medium effect for the knowledge about conceptions/dimensions/aims. The measured increases reflect the content of the modelling seminars, which focused on the theoretical foundations of mathematical modelling tasks and pedagogical diagnostics, as well as intervention concepts and their applications. Due to the training that incorporated practical elements, the knowledge of modelling aims and perspectives was emphasized less. In the baseline group, a significant effect was not found for any facet, meaning that there was no evidence of a test-repetition effect. This is also important information, as the test used unchanged as both a pre- and post-test. The significant increase in the modelling-specific pedagogical content knowledge of pre-service teachers qualified to teach at a grammar school (Gymnasium) is consistent with the results of previous studies (Blömeke & Kaiser, 2014; Krauss, Brunner, et al., 2008) on the professional competence of mathematics teachers in general, but it was not clear that it would apply to modelling in particular.

Differences between the groups (Research Question 2)

When comparing the two groups (Research Question 2), both experimental groups (task and intervention) showed significant differences relative to the baseline group at the second measurement point T2. Only the knowledge about conceptions/dimensions/aims in the intervention group was not significantly different from the baseline group. This can be attributed to the reduced emphasis on this facet, as mentioned above. The differences observed between the two experimental groups can be explained primarily by the different emphases of the corresponding modelling seminars at each university.

The treatments of the task group and the intervention group presented both similarities and differences; whereas both modelling seminars covered similar theoretical foundations for mathematical modelling and pedagogical diagnostics in the preparation phase, the emphasis varied according to whether the pre-service teachers were asked to design their own modelling tasks (task group) or to intensively study and test adaptive intervention concepts for mathematical modelling (intervention group). This explains the clear differences in knowledge about modelling tasks at the second measurement point T2. Such knowledge appears to be acquired more intensively by working on self-designed modelling tasks. The significant differences between task group and intervention group, and between each experimental group and the baseline group, can therefore be interpreted as consistent.

A change in knowledge about interventions was observed in the task group relative to the intervention group, with a surprisingly large effect size, even though this facet was only promoted with a special emphasis in the intervention group. Nevertheless, the ability values at the second measurement point T2 were not significantly different, given roughly the same starting conditions. Thus, a differentiated level of promotion was not observed for this facet.

The globally slightly more favourable development in the task group relative to the intervention group is possibly attributable to differences in the pre-service teacher population. To be able to assess this, various data were collected (sex, age, number of semesters, Abitur grade), of which sex and age show no particularities. However, the Abitur grades and the number of semesters studied could possibly indicate disadvantages affecting the intervention group, which could explain the lower increases. On the basis of the pre-tests and the similar structure of the degree programmes, a more comparable situation can be assumed, especially since the distribution of sex and age is comparable. Differences in content knowledge—which was not recorded—are also conceivable, since there are demonstrable connections between content knowledge and pedagogical content knowledge among Gymnasium teachers (Krauss, Brunner, et al., 2008). It could also be considered here that the two facets (knowledge about modelling tasks and knowledge about interventions, see Fig. 1) cannot always be taught in a way clearly separated from one another. Thus, the task group may also learn about possible support for students through intensive involvement with the solution of modelling tasks. In terms of content, it could also be useful to consider whether the intensive study of knowledge about modelling tasks through the construction of the students’ own modelling tasks is not more effective than the use of already completed modelling tasks.

Overall, the differences between the groups, especially in terms of knowledge about modelling tasks, and their clear demarcation from the baseline group indicate the effectiveness of the treatments. Thus, the key goal of acquiring pedagogical content knowledge (Peressini et al., 2004) was achieved with respect to teaching mathematical modelling in both experimental groups. What was interesting were the differences between the experimental groups, which give important indications for the design of modelling seminars.

Limitations

As the description of modelling-specific pedagogical content knowledge (Fig. 1) shows, it is a complex construct that is not easily learned in a course and cannot easily be assessed by means of a test. However, the scales can only be used to make statements about cognitive abilities relating to the evaluation and selection of predefined response options. To avoid an excessively long test, closed and dichotomous items were chosen to record the modelling-specific pedagogical content knowledge, some of which evaluated the relevant abilities using text vignettes. This enables a reliable measurement of the ability dimensions but may lead to lower validity. Nevertheless, it is a common practice in large-scale studies to measure the pedagogical content knowledge using paper-and-pencil tests (Depaepe et al., 2013).

Furthermore, our model focused on knowledge aspects related to teaching and omitted, for example, in the category “teachers’ knowledge about aims and perspectives”, any knowledge about the “modelling work of professional mathematical model constructors” (Frejd & Bergsten, 2016, p. 20). Other knowledge areas such as content knowledge and pedagogical/psychological knowledge, as well as various affective/value-oriented aspects (e.g., Eames, Brady, Jung, Glancy, & Lesh, 2018), were not considered due to test-time restrictions. Incorporating these aspects might provide further insight, since self-efficacy expectations can have a strong impact on teaching practices (Depaepe & König, 2018). Overall, we focused on pedagogical content knowledge as a key factor for determining the cognitive activation potential in lessons (Baumert & Kunter, 2013). This can only be a part of a comprehensive understanding of competence (Blomhøj & Jensen, 2007).

On the one hand, we have used a particularly atomistic approach (Blomhøj & Jensen, 2003) to competences by looking at individual facets of the modelling-specific pedagogical content knowledge and on the other hand by focusing on sub-competencies of modelling in the facet “knowledge about modelling processes”. However, we have not completely lost sight of the holistic approach of mathematical modelling by considering modelling cycles in the facet “knowledge about conceptions/dimensions/aims”. But of course the question remains open as to what contribution the Pedagogical Content Knowledge for the Teaching of Mathematical Modelling makes to the corresponding Professional Competence and how this actually influences teaching.

The study design enabled a comparison of two modelling seminars on the teaching of mathematical modelling, both featuring a practical phase. The advantages of repeating the implementation at both universities included consistency in the lecturer at each campus and an increased number of participants. However, differences in the preferences of the lecturers at each campus and differences between repetitions of the implementation cannot be fully ruled out, despite close coordination during the study. Even if the pre-service teachers had comparable previous experience in their studies, the number of semesters could indicate minor differences between the groups. The inclusion of the baseline group made it possible to rule out any test-repetition effects, as well as confirming that the participants did indeed acquire specific professional knowledge. The results also appear plausible against the background of the emphasis chosen in each case.

Overall, the strong and significant increases in the various facets of the modelling-specific professional knowledge demonstrate a successful acquisition of pedagogical content knowledge, including the selected context of practical integration and reflection.

Conclusion

Implications

For the teaching of mathematical modelling, two beneficial learning environments for the professionalization of pre-service teachers were created to allow relevant area-specific pedagogical content knowledge to be acquired. Teachers with high pedagogical content knowledge use tasks with high potential for cognitive activation and offer good support for individual student learning (Kunter & Baumert, 2013). Accordingly, it seems likely that successfully imparting subject-specific pedagogical content knowledge is of great importance.

The results also indicate that practical elements may have considerable potential in the university education of teachers, which should encourage further research in this area. Especially with regard to the acquisition of professional competence for teaching mathematical modelling, practical phases can be highly significant (Kaiser et al., 2010). The concrete design in the modelling seminars studied provides important indications for the design of practical elements for future teachers. With regard to quality development in teacher education, the integration of practical elements seems particularly worthwhile from a pedagogical content perspective.

Such references to the benefits of integrating practical phases into modelling seminars, which were further strengthened by our study, should be taken up and used specifically for further research.

The model fit and reliabilities of the scales of the test instrument show that the test instrument, which focused on the teaching of mathematical modelling, was successfully used to evaluate modelling seminars of pre-service teachers. This justifies using the model for the pedagogical content knowledge of mathematical modelling, which was developed on the basis of Borromeo Ferri and Blum (2010), although these dimensions were modified in the light of the various models of professional knowledge of pre-service mathematics teachers.

Perspectives

Using a newly developed test instrument, we successfully operationalized the modelling-specific pedagogical content knowledge. Although the test instrument cannot represent every facet of this specific pedagogical content knowledge, due to its use of text vignettes and closed, dichotomous items, the various item formats cover a wide range of professional knowledge. This new and proven test instrument can now be used for further research, especially the development of modelling seminars.

It remains important to examine the extent to which specific optimizations of the seminars lead to further increases in the pedagogical content knowledge of the pre-service teachers. If the modelling seminar were extended with regard to the use of digital media, pedagogical content knowledge about teaching mathematical modelling with digital media would also have to be included in the model. The extension of the test to include more modelling-specific competences (beliefs, etc.) should also be kept in mind. This also opens up the possibility for comparative studies on professional competencies for teaching mathematical modelling in different countries.

Overall, the study provides in-depth insight into the development of professional knowledge among pre-service teachers in a specialized sub-area of mathematics education. This unfolded within the framework of an approach that interweaves theoretical and practical phases, so as to facilitate the promotion of pedagogical content knowledge for teaching mathematical modelling.

Appendix

Table 6 Interaction effects in univariate two-way repeated measurement ANOVA

Table 7 Effects of the time factor in univariate one-way repeated measurement ANOVA

Table 8 Differences between groups for each facet at the second measurement point T2