Educational Research on Learning and Teaching Mathematics

Abstract

One of the main goals of research in (mathematics) education is the generation of knowledge on processes of teaching and learning. The approaches of many research projects in German-speaking countries that contributed to achieving this goal during recent decades are diverse. Many of these projects are characterized by narrowly focusing on distinctive phenomena within learning and teaching mathematics, by taking a multi-step approach that develops theory in a series of consecutive studies (often one area of interest is pursued over many years) and by a mixed-method research strategy that integrates different methodological practices. This chapter provides exemplary insight into these kinds of research in mathematics education in German-speaking countries over the last few decades. After a brief glimpse into the beginnings, we deliver four examples that illustrate the features of these kinds of research and also describe the perspectives of researchers by way of short interview excerpts.

Keywords

9.1 Introduction

As early as the 1980s, mathematics education in Germany was already actively using and developing research strategies focusing on distinctive phenomena of mathematics learning and teaching and combining and developing methods that were expected to support these research goals. Ursula Viet’s investigation of the cognitive development of fifth- and sixth-grade students in arithmetic and geometry (Viet et al. 1982) can serve as an example of such research: Viet and colleagues (1982) emphasized that, while in empirical research on mathematics subject case studies and interventions with pre-post designs dominated, their project aimed to study long-term learning processes. Furthermore, their focus was not on mathematics achievement in general, but on the development of knowledge in focused areas such as “counting and calculating in (non-decimal) place-value-systems” or “geometric mappings in situations with axial symmetry”. In a multi-step design, the project investigated the learning trajectories and learning obstacles of students while working within a “programmed instruction”-environment, i.e., standardized teaching material that excluded teachers and social interaction. Their analysis of clinical interviews and of errors in students’ solutions produced knowledge that is specific for the areas under investigation, such as the prevalence and effectiveness of different strategies in calculating in place-value systems.

Their early study already illustrates core features of the type of empirical research that eventually became a specific feature of mathematics education research: The interest in content-specific learning processes, the development of adequate research strategies, a multi-step long-term research strategy, and the use and adaptation of theories from psychology (cf. “borrowed theories” in Bikner-Ahsbahs and Vohns in this volume).

Many more studies working in a similar vein were launched during the following years, and several thereof were supported by grants from the German Research Foundation (Deutsche Forschungsgemeinschaft—DFG), such as in the 1990s:

• E. Cohors-Fresenborg: Individual Differences in the Mental Representation of Term Rewriting (Cohors-Fresenborg and Striethorst 2003).

• M. Franke: Solution strategies of students in primary school during work on picture-text-integrating tasks. Qualitative empirical investigations on word problems (Franke 1998).

• K. Hasemann: Categories of students’ mathematical thinking processes (Hasemann 1997).

• G. Krummheuer: Reconstruction of “formats of collective argumentation” in primary school mathematics lessons (Krummheuer 1998).

• K. Reiss: Problem solving strategies in spatial-geometric tasks with concrete and computer-simulated material (Reiss 1999; Burchartz and Stein 1998).

• M. Stein: Pupils work on problems with a goal that cannot be reached.

• H. Steinbring: Epistemological and social-interactive conditions for the construction of mathematical knowledge structures (Steinbring 1997).

• B. Wollring: Qualitative empirical investigations on the understanding of probability of children in kindergarten and primary school (Wollring 1994).

Since it is hardly possible to report comprehensively on this strand of research found throughout mathematics education research in German-speaking countries, we present four more recent examples of projects that illustrate how these features were expanded upon during subsequent decades—and omit many others that would also be worth mentioning. The reader who is interested in the breadth of research in German-speaking countries during recent decades can get a fairly good impression by browsing the content of Journal für Mathematik-Didaktik (JMD): all articles since the first issue in 1980 have been retro-digitalized and offer abstracts in English. For specific types of empirical research that are quite different (e.g., design science, classroom studies or large-scale studies), we refer to the respective chapters in this book.

Within the following four research projects in the next two sections, we discuss key aspects of research strategies in mathematics education from a general point of view. Furthermore, we insert excerpts from video interviews with the researchers in which they explain their research goals and strategies when designing and conducting such research in the field of mathematics education. These interviews were presented in a session at ICME-13 in Hamburg. They were discussed by international participants and contain the spontaneous reactions from the interviewed researchers. Their statements have been slightly revised and abridged but are intended to maintain their spontaneous and subjective character.

9.2 Interdisciplinary Research

In 2000–2006, the German Research Foundation set up a six-year priority program for empirical educational research „The Educational Quality of Schools“ (Bildungsqualität von Schulen—BiQua), with a multiple-strategy focus (Prenzel and Schöps 2007) and with the following goals:

• to investigate mathematics and science teaching, since this was the area of concern after the TIMS-Study’s findings—aiming to generate explanations for the mediocre results

• to motivate analysis of domain-specific and cross-curricula learning within and outside school—a rather broad scope

• to initiate interdisciplinary cooperation and methodological development in empirical educational research.

The three-step program was designed to initiate projects ranging from foundational research to implementation. Quantitatively speaking, this resulted in 32 projects, and 80 doctoral studies from all areas of educational research, including mathematics education. Qualitatively speaking, many researches profited from collaborations that led to new approaches, interesting findings, the advancement of empirical research in mathematics and science education, and the development and fruitful integration of theories that had previously not been considered to be interconnected. Some of the projects that inspired mathematics education research were:

• R. Pekrun, R. vom Hofe, W. Blum, S. Wartha, and others: PALMA—development of mathematical competence and motivation over 6 school years

• M. Kunter, J. Baumert, W. Blum, A. Jordan, S. Krauss, M. Neubrand, and others: COACTIV: Linking teacher knowledge and student achievement

• E. Klieme, B. Drollinger-Vetter, F. Lipowsky, C. Pauli, K. Reusser and K. Rakoczy: PYTHAGORAS: Instructional quality in classroom (also: subject-specific) and learning outcome

• R. Bruder, B. Schmitz, F, Perels, E. Komorek, C. Collet and others: Fostering problem solving and self-regulation, and a subsequent teacher-training program

• A. Renkl, S. Schworm, K. Reiss, A. Heinze and others: Learning to prove—The idea of heuristic examples.

9.2.1 Example 1: Problem Solving and Self-regulation

Should general learning skills and mathematical skills be taught in a separate or an integrated way? This quite general question was investigated in a concrete manner within the project of Bruder, Schmitz and colleagues (Bruder et al. 2007; Perels et al. 2007). Educational psychology considers content-related goals as helpful within training programs that aim to promote students’ subject-independent self-regulatory skills (e.g., Hasselhorn and Hager 2001; de Corte et al. 2000). Furthermore, when discussing theories that describe self-regulatory behavior, such as the model by Schmitz and Wiese (2006), mathematics educators recognize many core elements of problem-solving behavior, as already described for example by Polya (1949) (see Fig. 9.1).

Fig. 9.1

This “process model of self-regulation” (according to Schmitz and Wiese 2006) bears much resemblance to the phases of mathematical problem-solving

It thus would appear plausible to devise training programs that combine instructional elements with respect to problem-solving strategies and those that foster self-regulation strategies in different phases.

The program as shown in Fig. 9.2 was implemented in various ways with students of different ages and including elements that trained teachers and parents to support the learning process. Experimentally-controlled variation yielded many interesting findings with respect to the effect of the various parameters. One interesting finding was that the training of self-regulation skills also boosts problem-solving competence, but not vice versa. When adding support by teachers and/or parents, another of the interesting findings was that teacher support enhances the training’s effect while teacher training alone has no effect.

Fig. 9.2

A training program for problem solving and self-regulation (from Perels et al. 2007)

This of course cannot be generalized, and there may be more efficient ways to train teachers. Bruder et al. (2007) developed such a training program that started by transforming the extracurricular program into a teaching concept that could be “hard-wired” into the everyday mathematics classroom. It was implemented and tested in field studies in three phases with pre-service teachers, with teachers in the induction phase, and with regular teachers. Again, problem-solving elements were combined with self-regulation elements.

Unfortunately, no control group design was possible, meaning that the considerable increase in heuristic use and mathematics achievement cannot be identified as an effect of the intervention. However, the researchers demonstrated the impact of the training on many facets on the teacher and classroom level, such as the ability of teachers to analyze problem-solving situations.

Interview with Regina Bruder about her experiences in the collaboration:

Bernhard Schmitz and I met at a psychologist meeting in Darmstadt. He was seeking a domain for his current research on self-regulation. I saw an opportunity to profit from the psychological expertise. My goal was an empirical study with respect to my theoretical work on learning and problem solving. I learned from problem-solving training in mathematics competitions that mental flexibility can be increased by making pupils aware of heuristic principles and strategies. The question was whether this impressive effect was also achievable in everyday lessons.

One target of the six-year research project funded by the DFG was to prove empirically the validity of theoretically-postulated connections between problem solving and self-regulation. It was also clear from the beginning that a sustainable effect was intended to help us develop lessons with a carefully developed intervention. Three stages were needed for this. First, I had to be sure that problem solving and self-regulation abilities are achievable by most students. In a pilot-study during special courses outside the regular classroom, we ensured that combining training in problem solving and self-regulation produces benefits when compared to pure problem-solving training sessions or pure self-regulation training sessions. In the next step, a teaching concept was developed together with dedicated teachers for how to integrate the strategy training sessions from the pilot study within normal lessons. The effects of this implementation were evaluated again, confirming our pilot study’s results. The third step was to transport this lesson concept to teacher teams from various schools. This concept’s implementation by the more than fifty teachers involved was monitored via standardized lesson reports. The pupils’ problem-solving and self-regulation results were assessed via tests and interviews. In the follow-up study we demonstrated that the classes taught with the combination still possessed stable knowledge on heuristics one year later.

What have I learned for further research? In this project I realized that it is not only helpful but necessary to have a strong theoretical background when hypotheses should be reviewed quantitatively. The attempt to generate new theories from quantitative empirical data will not succeed – you cannot get out more than you have put into it. Furthermore, there were differences in the researchers’ view on the project, especially regarding what is still part of the research and what is not. I believe that subject-specific didactic research must not be limited to the generation of general theories: it must develop concepts for transferring the scientific insights into practice as well. We continued to develop further training courses even after the funding ended. The question that remains unanswered is: to what extent are we as researchers also responsible for the results we achieve being put into practice?

9.2.2 Example 2: Proof of Competence and Worked Examples

Mathematical proof is a central but difficult topic in school mathematics. It has been intensively addressed in mathematics education (e.g., Hanna and Jahnke 1996; Healy and Hoyles 1998; Reid and Knipping 2010). Many different approaches to teaching proof have been suggested. Since mathematics educators tend to take their own discipline as a starting point, such suggestions may ask for simplifications of the proof concept that are ‚intellectually honest’ but appropriate for the students’ cognitive level. Going back to Branford (1913) and Freudenthal (1978), this has been done in German Stoffdidaktik, for example by Wittmann and Müller (1988) by Inhaltlich-anschauliche Beweise (visual proof) or by Blum and Kirsch (1989, 1991) by „preformal proof“. However—although these concepts had a substantial impact on the classroom, mostly via teaching materials—they have not been empirically investigated with respect to their effectiveness.

Cooperation between mathematics education and educational psychology, however, yielded a new approach—one that interconnected a substantial mathematical viewpoint with state-of-the-art learning theory: in the aforementioned priority program, Kristina Reiss (München) and Alexander Renkl (Freiburg) came up with the following idea: Among many theories that describe the process of proof generation, Boero’s (1999) six steps—conjecturing, formulating, exploring, deductive organization, formal presentation and discussion within the community—offered a framework that could be translated, with enough adaptations, to the classroom, yielding a heuristic for generating proofs. For learning new concepts, especially in well-structured domains, many experiments revealed the benefit of learning with worked-out examples, thus reducing the cognitive load of executing problem-solving processes in unfamiliar areas.

Both researchers had intensively investigated their respective areas of interest: Reiss and coworkers developed a test to capture levels of proof competence, while Renkl and his team investigated the processes and conditions relevant for learning with worked-out examples.

However, these approaches had to be adapted to the case of proving, since the learning goal was not a well-structured concept but rather a heuristic process (with many vaguely-defined decisions on the way). Several features were implemented to enhance cognitive activity and reduce unwanted cognitive load: (1) blanks within the examples, (2) prompted self-explanations (why certain steps are taken in a proof) or (3) discussion of necessary knowledge (in geometry). Figure 9.3 illustrates the beginning section of a worked example for learning proof.

Fig. 9.3

Worked example for teaching proof (only beginning section, illustrative pictures removed, source: teaching material supplied by A. Renkl)

This approach proved successful in several ways: Students exhibited a moderate increase in proof competence, and less fear of making mistakes. However, it was predominantly the lower-achieving students that profited, while students with solid understanding of the domain may have been bothered by the intervention—a typical phenomenon when using worked examples. A specific teacher-training session could even enhance the effects.

Interview Alexander Renkl about his experiences in the collaboration:

How did the research question emerge?

AR: Kristina Reiss and I were both involved in the same research priority program. And of course I was always interested in Kristina’s project because my interests were always in how to foster students understanding in mathematics and Kristina was also always interested in psychological theories that can be applied to mathematics. Mathematics educators often told me that a drawback of worked examples is that you cannot teach heuristics or creative thinking skills to students. Kristina was working on the project to foster proving skills, so it was quite natural that we would cooperate.

What was the relevance of mathematical educational theories and educational theories for your project?

AR: Kristina had this Boero model of proof generation in mind, namely, a background for how to teach proving or argumentation to the students. It was also important from the math-education perspective to feel good about the mathematical materials you can use. We had to choose some geometry problems to solve. I couldn’t have done that as a psychologist. On the other hand, at that time there were, I think, three important theoretical models related to worked-out-example learning, namely the cognitive load theory approach, Chi’s self-explanation model - because we always emphasize that students should not just breeze through worked-out examples but rather also explain the rationale of the worked-out examples to themselves - and finally the skill acquisition model of VanLehn. And so these three psychological models were important as guidelines to design the student environment as an example-based environment.

Did you experience problems with, or any limitations due to the close connection between mathematics and psychology?

AR, I had carried out many interdisciplinary projects in the meantime, and I had also experienced the usual problems in projects like this, but in this specific case there were no real barriers. Kristina is a math educator who is very familiar with psychology, and I’m not too distant from mathematics education. I was always somewhat interested in mathematics education even before then. And as our perspectives were not that divergent, there was a lot of productive cooperation; it’s also helps when there’s mutual affinity, as things then go relatively easily.

What consequences did this close cooperation have for the results or for the way you tackle the problems?

AR: We conducted some shorter controlled studies managed primarily by us and by the psychologists, and that were inspired by the math-education perspective. On the other hand, the more complex classroom studies were done by math educators. Thus everyone’s contribution was in his or her field of expertise. This went quite well, and we also benefited from the advantage of obtaining findings from the lab, from more controlled lab-type studies, as well as from the classroom - that’s ideal if you want robust findings.

In the second step, I also always need to know what works in the classroom setting, so this cooperation also had the advantage of satisfying this desire of an educational psychologist, namely what I find in the lab can be applied in the classroom.

Could you also claim there was an influence on math education in general or educational psychology coming from your project?

AR: I would say yes, at least in the German math-education community. I got the feeling that worked-out examples didn’t play a genuine role initially. Most of the math educators have absorbed a bit of old-fashioned pedagogy. And the idea developed in psychology of using worked examples as specific means of fostering understanding was sometimes hard to get across to math educators. At least I found it hard to explain - to convince those colleagues; however, this project and our results made the math educators more curious about the potential of worked examples for fostering understanding and so it was a kind of door-opener for this idea to those educator communities.

Would you say that there has even been an impact on the classroom, in the German mathematics classroom?

AR: Perhaps, some ideas from our research are included in some mathematics textbooks. So there may be a minor influence, but I would guess that it is primarily evident in the design of textbooks. I would not say that the typical German math teacher knows much about the theory of example-based learning.

How would you say that this specific project influenced your own further studies, your own research later?

AR: I think it had a huge influence, because we grew to realize that worked examples can be applied for algorithmic content as well as other fields. And so we used structurally similar worked examples for non-mathematical areas, and they worked well in those areas, which opened up a totally new field of application possibilities for worked examples. That was a strong influence.

The second benefit was connected to mathematics. I was then also interested in whether I could set aside other restrictions of typical worked-out examples that math educators often pointed out, for example multiple solutions. We then devised worked examples with multiple solutions. And they work if you use them in a good way. We aso devised worked-out examples containing typical errors and guided students to process, to listen in a beneficial way.

Finally over the last several years I have written theoretical papers about what I call focused information processing. Many of these basic ideas originated from this project on proof finding.

The two examples from research demonstrate how empirical research profits from combining theoretical and methodological perspectives of general education research (educational psychology) and of research in mathematics education. It is important to note that, in both examples the “binding force” in the collaboration was not a simple sort of work sharing, such as. bringing content expertise and methodological expertise into a project. It was rather the complementary theoretical perspective of a relevant situation in teaching and learning: The commonalities of (subject-independent) self-regulation and (subject-specific) problem solving made it fruitful to combine the two aspects and seek parallels or synergies in their combined instruction. Furthermore, it was the partners’ goal in both examples to extend the knowledge in their respective area of interest, e.g., taking the step from worked examples to heuristic worked examples. This step was generated in a productive struggle between the researchers’ field of general learning and mathematical proving with respect to the meaning of each of their theories for their partner’s field.

Furthermore, one also notes in both projects a certain degree of circumspection in developing and implementing an intervention: The researchers carefully plan the interventions not only considering their power to (dis)prove a hypothesis empirically but also to accommodate an everyday classroom environment, e.g., via a multi-step approach—the research presented here builds upon many studies that the researchers conducted already (on their own) to develop and validate their theories.

The aforementioned projects are inspiring examples of research in (mathematics) education in Germany, and many more researchers have had similar experiences. However, while multi-step research focusing on central phenomena of mathematics learning and teaching are quite prevalent in mathematics education research in German-speaking countries, interdisciplinary research is still the exception, since researchers in subject-matter education and general education often tend to have different backgrounds and belong to different university faculties.

9.3 Mixed-Methods Research

The second section introduces two research projects that draw equally from qualitative and quantitative methods in a mixed-method design. As an introduction, a condensed theoretical framework is presented that we refer to again in the discussion of the two research projects.

In recent decades, researchers in mathematics education have expanded their repertoire with research designs that suffice for and can tackle especially multifaceted research problems by integrating different methodological practices, eventually with a mixture of qualitative and quantitative research questions suitable for a complex empirical and theoretical research topic (Johnson and Onwuegbuzie 2004; Hart et al. 2009). However, a review of the latest literature reveals many different theoretical justifications for and definitions of mixed-methods research (MMR), some of which are contradictory (cf. Kelle 2008; Teddlie and Tashakkori 2009).

Although Denzin (1970) originally referred to a combination of multiple forms of qualitative research methods only (Denzin 2012), especially his concept of triangulation has often been referred to in German-speaking countries as a theoretical research background for integrating and employing several (i) data sources, (ii) investigators, (iii) theories, and (iv) methodological practices—both qualitative and quantitative—within a single research design. Remote from its original understanding, the term triangulation is frequently referred to with two different meanings (Schulz 2010; Kelle and Buchholtz 2015, p. 331): (a) “triangulation as a mutual validation of research results” and (b) “triangulation as an integration of complementary perspectives on the subject being investigated to create a more complete image of the research domain.”

Being suitable for both of these meanings, the importance of MMR can be justified by taking a closer look at the specific strengths and weaknesses of qualitative and quantitative methods, and how a balance can be struck by different types of MMR designs (Kelle 2008; cf. Schulz 2011): Qualitative research is restricted to the careful choice of a few cases. Moreover, the analysis of unstandardized data is highly dependent on the individual researcher. This “immediately raises questions about the generalizability of findings and about the intersubjectivity of interpretations” (Kelle and Buchholtz 2015, p. 336). On the other hand, qualitative methods are often useful for exploration, detection, and discovery. They help us “construct new theoretical concepts, categories, and sometimes even whole theories about the domain under study” (ibid.). Quantitative research relies on standardization and requires reliable and objective (observer-neutral) data. This enables the investigation of many cases, which contributes to a theoretically grounded basis for generalizations. Quantitative methods are ideally suited for a theory-driven approach, “whereby precise hypotheses are formulated at the onset, then operational definitions are formulated for the concepts these hypotheses comprise, measurement instruments are constructed, and data are collected and analyzed subsequently” (ibid., p. 334). They are therefore appropriate tools to examine clear-cut causal statements. However, the researchers’ theoretical knowledge or that of the corresponding discipline may be limited. Furthermore, context-bound patterns, structures and rules may form an integral part of particular life worlds under investigation that may be unknown to the researchers, especially if the socio-cultural background of researchers and respondents differs. Such a lack of knowledge may cause severe problems when formulating hypotheses, defining variables, and creating research instruments. Specifically, when examining causal statements, important variables may be “omitted with low levels of explained variance as a consequence”, intervening variables may be overlooked, or functional relations between certain variables may be incorrectly specified “so that causal processes underlying the investigated phenomena are not adequately understood” (ibid., p. 335; cf. Kelle 2008; Schulz and Wirtz 2012).

Being driven by awareness of the specific strengths and shortcomings of mono-methodological research designs, MMR designs may be used as strategies “for overcoming each method’s weaknesses and limitations by deliberately combining different types of methods” (Brewer and Hunter 1989, p. 11). Meanwhile, various designs and manifold categorizations for MMR designs exist and are discussed in German-speaking countries (e.g., Kelle 2008; Kuckartz 2014; Kelle and Buchholtz 2015; Burzan 2016). To discuss many of them would clearly go beyond the scope of this chapter. Instead, we present a concise view into the world of MMR designs that suffices as a framework for discussing the MMR design examples discussed later in this chapter. We concentrate on four prototypical MMR designs:

• (a) In a sequential qualitative-quantitative design, a qualitative study is first used to develop theoretical concepts and hypotheses. Their generalizability is then examined in a quantitative study. A widely used example for this kind of MMR design is when measurement instruments or interventions are developed on the foundation of existing qualitative findings, which enable their operationalization in a quantitative study.

• (b) In a sequential quantitative-qualitative design, for example, incomprehensible statistics may lead to their further investigation with the help of qualitative data and methods. Another example is that a quantitative pre-study may support the selection of cases for a qualitative study.

• (c) In a parallel qualitative-quantitative design, for example, researchers may interview or observe the same individuals simultaneously while applying different methods. In an experimental setting, a learning progress might be examined in a summative assessment via quantitative methods while the underlying learning processes could be validated or identified via qualitative methods.

• (d) A complex qualitative-quantitative design integrates the partial qualitative and quantitative studies systematically, eventually by performing several research cycles altogether. This allows researchers to benefit from the many advantages of both sequential and parallel designs.

Complex designs require extraordinary effort that often can be coped with by well-coordinated teams only. If individual researchers hope to create a MMR design, they will require expertise in both qualitative and quantitative research methods. Furthermore, they should be aware of the specific strengths and weaknesses of qualitative and quantitative research methods so as to combine and integrate them productively and well targeted.

In the following, we present and discuss two examples of MMR designs from Germany: Kathleen Philipp made use of a sequential mixed-methods design (Philipp 2012; Philipp and Leuders 2012). She analyzed students’ strategies while solving several mathematical problems and developed a competence model about experimental thinking in mathematics. This laid the groundwork for an intervention study that confirmed that experimental skills in mathematics can be fostered effectively.

Prediger and Wessel (2013) and Wessel (2014) implemented an integrated, parallel mixed-methods design. They fostered students’ understanding of fractions, and scaffolded the learning processes by facilitating students’ abilities to talk about fractions and their meaning. Their intervention’s efficacy was investigated by a randomized controlled trial and qualitative analyses of the videotaped teaching-learning processes.

9.3.1 Example 3: Understanding and Teaching Experimental Thinking in Mathematics

Knowledge acquisition in the natural sciences is based on experiments. Experimental thinking includes the generation and confirmation of hypotheses. There are similar processes of knowledge generation in mathematics: relationships between different kinds of objects are explored, hypotheses are generated and examined by producing and analyzing examples—as described by e.g. Euler or Pólya. This is the starting point of the project by Philipp et al. (2012) described below.

Despite the analogy, knowledge acquisition in mathematics is quite seldom associated with experimentation (e.g., Bartolini Bussi 2009). Proof and the axiomatic method dominate when establishing mathematical knowledge is discussed. However, that neglects the essential role that experimental thinking in mathematics plays in (i) generating and confirming hypotheses by examining examples before coming to any final proposition, and (ii) in arriving at a mathematical proof.

Concerning the natural sciences, scientific discovery can be described as a process of Dual Search (SDDS, Klahr and Dunbar 1988). The search in two problem spaces (one hypothetical, the other experimental) shapes hypothesis generation, experimental design, and the evaluation of hypotheses.

In their first study, Philipp et al. analyzed the experimental activities of pupils in mathematical problem situations (Leuders et al. 2011). A theory about experimental thinking in mathematics was developed relying on those findings while keeping the theoretical background of experimental thinking in mind. In the following study, experimental thinking skills in mathematics were identified and operationalized. This measurement was used in a pre-post test design to assess the learning effect of an intervention that fostered students’ skills in experimental thinking.

The first study’s cohort consisted of nine students (prospective teachers) and twelve pupils (grades 3 and 4). They were videotaped while solving mathematical problems. The problems had been chosen to be open-ended and to trigger many hypotheses and examples to solve the problem. One of those problems was the “sums of consecutive numbers” (see Fig. 9.4: “stair numbers”).

Fig. 9.4

Sums of consecutive numbers—example problem (cf. Philipp 2012, Abb. 16.1.)

The analysis made use of Grounded Theory. Procedures were identified and coded. The system below of four core procedural categories became apparent via axial coding: (1) generating examples, (2) structuring examples, (3) generating hypotheses, and (4) testing hypotheses. These four core categories are interrelated in the experimental-thinking process while trying to solve a problem (Fig. 9.5).

Fig. 9.5

Model of experimental thinking in mathematics (cf. Philipp 2012, p. 289)

In the next quantitative study, the intervention was developed according to approved design principles for fostering problem-solving strategies (Bruder 2003) and included four phases: the introduction of problems, explaining problem-solving procedures, reflecting on procedures and applying them to other situations. Participants were sixth graders (middle school): 126 students were assigned to the intervention group, and 101 students to the control group. The intervention lasted about three weeks. A pre- and post-test with a control group were administered immediately before and after the intervention, a follow-up test about six weeks later. The pre-, post- and follow-up-tests included items for measuring “structuring examples” and “testing hypotheses”. The analysis revealed a significant increase in large effects in both areas, which remained stable over 6 weeks. The large effect sizes support the authors’ claim that such procedures can be fostered effectively in regular mathematics classrooms.

Interview with Kathleen Philipp (KP), about her experiences within the MMR design described above:

How did you come up with your research question?

KP: “When I was a teacher instructing my pupils in class, I often observed curiosity in children’s mathematical thinking. I was given the chance to take part in a large research project as a PhD student on the topic of “experimentation” in mathematics or natural sciences. Being embedded in that research context, our research goal was to analyze the significance of experimentation in mathematics, and I was pleased to be able to take a closer look at the experimental thinking of students in the mathematics classroom.”

How did your theoretical background influence the research design?

KP: “The theoretical background concerning processes that are characterizable as experimental approaches was very helpful for me to understand the importance of such processes. When Euler, and later Polya, described experimental processes, I realized it was worth trying to see whether children can do mathematics in a similar manner. One of the challenges in the beginning was to find tasks for students that allowed them to find their own ways to solve a problem, even when they were unfamiliar with such tasks. It was also clear that the interesting processes are internal ones and that it would be hard to observe and analyze them, in particular when young children are involved. To overcome these challenges, I used different methods to collect data.

The theoretical background of the two problem spaces from Klahr and Dunbar helped me understand the complex processes many students demonstrated in these settings. It was very obvious that examples and hypotheses play an important role in mathematical experimentation. But there was something else we needed to understand. I refined the existing theory about two problem spaces and identified a third space that characterizes the kind of strategies used to switch between the two spaces, i.e., between examples and hypotheses.”

How did you choose and develop your research strategies?

KP: “First I identified the experimental strategies students use when exploring mathematical problems. When I noticed that students do mathematics in a manner resembling that of mathematicians, I wondered if it would be possible to teach such experimental strategies in class. To investigate this, I had to think about a means of measuring such strategies. Then I developed tasks that would reveal whether experimental strategies were being applied to find a solution. The third step was to develop a teaching unit in the standard curriculum in which I could integrate the training of experimental strategies. Finally, I had the teaching unit taught in several classes. All classes were taught by their regular mathematics teachers. At the end, I compared how frequently the respective students used experimental strategies successfully with classes that had not participated in the training session but who had been taught the same contents by relying on the school textbooks.”

Which kinds of triangulation(s) did you implement in your mixed-method design?

KP: “I triangulated several theories and data sources. To generate and analyze my data, I applied both qualitative and quantitative methods.”

Why did you implement those kinds of triangulation? Can you describe the interplay of or roles of ‘theory development’ and ‘theory confirmation’ in your study?

KP: “The development of my research project required different methods. To identify experimental strategies, I had to take a qualitative approach, because the aim was to investigate and understand students’ approaches in depth, and to create a foundation for developing a theory of experimentation while learning mathematics. I therefore applied the Grounded Theory to analyze the students’ approaches, which has the advantage of enabling you to develop a theory from your data. Thus, as is typical when using Grounded Theory, it was feasible to identify core strategies that students apply in experimentation.

Quantitative methods were necessary to measure the use of such experimental strategies, which made it possible to investigate a larger group of students (about 230) and make comparisons with a control group.

On the one hand, different methods were necessary to find answers to the specific research questions. On the other hand, the measuring and fostering of experimental strategies validated the theory. I relied on different data sources to observe and assess experimental strategies.”

Was implementation in the classroom important from the start?

KP: “Yes, implementation in the classroom was important for me personally. Working as a teacher for many years, it was my research concern to link my research findings to actual practice. In my opinion, it is crucial that research results in our field be relevant in mathematics education.”

How did the project influence your later research projects personally?

KP: “For me, this research project was fundamental to the evolution of my research interests. It is still important to me that research results be closely associated with practice. Another aspect is that I have a deep interest in understanding people’s thinking and their thought processes.”

9.3.2 Understanding and Teaching Fractions

Our description of the fourth project example follows Prediger and Wessel (2013) and Wessel (2014) who work on language challenges and conceptual mathematical challenges for students with low language proficiency. Language proficiency is the key determinant of mathematics achievement—not immigrant status or being multilingual (Heinze et al. 2009; Prediger et al. 2018). Students with low language proficiency encounter reading obstacles as well as difficulty in developing conceptual understanding (Prediger et al. 2018).

To support students’ constructions of meanings for new mental objects and relationships (Steinbring 2005), the authors draw upon the principle of relating verbal, symbolic, graphic, and concrete representations (Duval 2006; Lesh 1979; Bruner 1967). Conceptual learning processes involve the acquisition of new linguistic, graphic, and symbolic means for expressing mathematical concepts and their meanings (Schleppegrell 2010). This illustrates how deeply interwoven mathematics learning and language learning are (Pimm 1987).

The principles of relating representations has been extended to relating different linguistic registers also, such as establishing first- and second-language registers, mathematical technical registers, a school language register, and the students’ everyday registers (see Fig. 9.6: Prediger and Wessel 2013). “A register can be defined as the configuration of semantic resources that a member of a culture typically associates with the situation type… in a given social context” (Halliday 1978, p. 111).

Fig. 9.6

Registers and representations (Prediger and Wessel 2013, p. 438)

The school register is characterized by context-reduced and complex linguistic means. It differs from the everyday register by appearing conceptually written even if medially oral. Unlike the technical register, the school register is seldom taught explicitly in school. In particular, this is problematic for underprivileged language learners who acquire only everyday language and vocabulary at home but nevertheless require the school register for higher thinking skills (Clarkson 2009), while privileged (first- or second-) language learners are exposed to the school register already at home (Schleppegrell 2004; Cummins 2000).

An intervention was designed that included translations between all registers and related them to each other. The choice of suitable activities for each moment in the learning process was guided by the design strategies of “pushed output” (Swain 1985) and scaffolding (Hammond and Gibbons 2005). The language- and mathematics-integrated intervention aimed to initiate the construction of mathematical meaning in combination with opportunities to establish the required language means (Prediger and Wessel 2013: Fig. 9.7):

Fig. 9.7

Macro scaffolding (Prediger and Wessel 2013, p. 444); “Anteil” in (e) is used deliberately by the authors in their translation of the task. It signifies the mental image connected to a fraction as a part of a whole

The intervention study took the mixed-methods approach to investigate effects from two research questions in a triangulating manner (Prediger and Wessel 2013, p. 443): “(1) To what extent do students who participate in the language- and mathematics-integrated intervention improve their achievement in the fraction test? (2) What is the situational potential of the intervention activities to initiate students’ constructing of meanings and activating of linguistic means in the school and technical register?”

The study sample consisted of 72 seventh-grade second-language learners with below average math performance and limited German language proficiency. The intervention consisted of six lessons lasting 90 min in 2-to-1 sessions (two students with one teacher). A pre- and post-test randomized controlled trial design with a control group was realized (Fig. 9.8).

Fig. 9.8

Mixed-methods design (Prediger and Wessel 2013, p. 445)

The control group was taught by their regular teacher with the usual fraction textbook repetition program. The 18 × 540 min were video-taped, and relevant episodes were selected according to research question 2. The data corpus for the qualitative analysis comprised selected videos, all written materials from the interventions, and the teachers’ lesson plans.

The quantitative analysis of the learning gains revealed a low-to-medium effect (intra-group effect size measured by d = 0.42) in the control group, but a very strong effect (d = 1.22) in the intervention group. Those differences became significant in a later study with a larger cohort (Prediger and Wessel 2018).

Quantitative analysis of learning effects was complemented by a qualitative analysis to exploratively reconstruct the situational potential of the design strategies and instructional activities in the learning processes (Prediger and Wessel 2013: Fig. 9.8). The qualitative analysis included in-depth analysis of the videotaped teaching–learning processes: (1) the aim of sequential analysis was to reconstruct individual and interactional processes of meaning construction for fractions, (2) several qualitative coding procedures for students’ utterances were conducted to capture different linguistic aspects, and (3) students’ development of mathematical meaning and linguistic means were reconstructed in their interaction with concrete elements of the instructional design.

This qualitative analysis identified the specific potential of macro- and micro-scaffolding in combination with the design strategy of pushed output.

The MMR design was reflected in the study findings (Prediger and Wessel 2013, p. 453): “Whereas these quantitative results alone might have been explained on the basis of the intensity of individual teaching, the case studies gave some first insights into how the design elements offered fruitful learning opportunities, since they showed the deep interconnection of linguistic and mathematical learning processes: …With the help of macro and micro scaffolding, low-achieving students successfully proceeded from their everyday language to a language for thinking and talking about structural relationships.”

Interview with Susanne Prediger (SP), about her experiences within the above - described MMR design:

How did the research question emerge? Was it a rational decision?

SP: “A research question is also a matter of personal choices, but it is of course based on a rational decision. Regarding the challenges of language learners in mathematics: by 2011, we had a situation where we had a lot of statistical evidence of connections between language proficiency and mathematics learning. But we did not know much about how to foster language learning. We thus chose to continue to dig deeper and beyond the current descriptive knowledge, to find answers about how to foster language learners and what could work for them.”

How did the theoretical background influence the research design?

SP: “That it definitely does. In our case it’s how you conceive of the role of language in the learning process. When we consider language only as relevant in its communicative function, it might suffice to gather knowledge about how to approach word problems and alleviate language difficulties in word problems. But we start from the assumption that language has a deep cognitive function within mathematical learning processes. If we want to delve deeper into that, we need to study the learning processes, and see how the interplay between language and mathematics really works. That is why we had to initiate mathematics- and language-integrated processes. In everyday classrooms, we observed hardly any conceptual learning opportunities for students with low language proficiency, which created the need to establish an instructional design specifically for those students as a research context for our research questions.”

What kinds of triangulations did you implement in your mixed-methods design?

SP: “First of all it is a triangulation of data, because we wanted on the one hand to gather data about the intervention’s function to see if it works. To test what works, you need a pre- and a post-test, you need to conduct a randomized controlled trial. But if you want to learn how it works, you really need data on the actual processes, meaning video data and qualitative analysis.

On the theoretical level, we preferred working with networking theories. It is a similar idea, but it is not just about putting the two next to another. You really have to integrate the different theories. Regarding this topic of language and mathematics education, of course you need theories about mathematics learning and you need theories about language and its role, and they have to be combined.”

Can you describe the interplay between ‘theory development’ and ‘theory confirmation’ in your study?

SP: “In the classical sense we only discuss confirming theories, which is all about hypotheses and validating them. The randomized controlled trial is the research approach of choice for validating hypotheses. In our case, we could validate the hypothesis that students with low language proficiency learn better when they are given language- and mathematics-integrated learning opportunities.

However, before you can validate a hypothesis, you must develop categories to characterize phenomena and to generate hypotheses to be validated. This is where qualitative, explorative research approaches are most suitable.

In our research, we really combined these two approaches: we validated a hypothesis that had been quite general and invested a lot of time and energy to generate a theory on a more fine-grained level. For this, we had to develop the categories in which we could formulate phenomena and refined hypotheses. The importance of generating categories is often underrated.

In the present project the validation of the hypothesis that a language- and content-integrated intervention might be more effective was only a starting point. To elaborate upon our theory, we had to investigate the processes in depth to understand what exactly happened in the learning processes. And this qualitative research enabled us to generate categories that we could use as a starting point and for initial hypotheses for the next project. Within the next project, which is now being funded by the German Research Foundation, we again have some hypotheses that we can validate or reject, and we will generate new questions and new aspects to be studied in greater depth. (Prediger and Wessel 2018)

Thus, there’s always interplay between hypothesis generation and its validation. Indeed, the categories are really the most important element in terms of theory generation. There is a worthwhile article by Cobb and diSessa, who talk about ontological innovations. Their article puts emphasis on the generation of categories as the major output of research.”

What do you personally prefer: to start with a qualitative study, or take a quantitative approach?

SP: “(Laughs) Neither nor - it’s more like going backward and forward. Sometimes, like in this project, there was quantitative evidence of a problem with language proficiency and mathematics achievement. Thus, we started to understand what the problem was qualitatively, and conducted design research cycles to devise an intervention that we could put to the test in a randomized controlled trial. Yet I would never want to conclude with a randomized controlled trial, because we need to understand how things work and what really happens in these processes. So, it really is a backward-forward movement. I’m really pleased that we have funding for two successive projects, including the one described here. In both we dug deeper and deeper into the details about what students really need, into what kinds of fostering in language terms they require, and into the type of fostering that is genuinely effective.

It’s like the question about the hen and the egg - you can’t tell where to start, you have to take a combined, cyclical approach.”

• Summarizing comments on the two MMR projects and the interviews

Many researchers in mathematics education consider themselves to be engaged in an applied discipline, where research should contribute to improving education. Learning mathematics with understanding is a complex process, and (Hart et al. 2009, p. 39) “mixed methods research may be an appropriate response to calls for greater generalizability of results while maintaining enough detail about the processes of teaching and learning to be valid and useful.” This idea on the purpose and use of MMR in mathematics education was expressed clearly by the two researchers we interviewed, Kathleen Philipp and Susanne Prediger. It goes together with the concern expressed by Hart et al. (ibid.), that research in mathematics education does not only need to know “if particular educational experiments improve learning with understanding but also how those results are achieved and why we can expect them to be replicated elsewhere.”

The two examples above and the interviews with the researchers illustrate how MMR in particular can contribute a great deal of both scientific and practical knowledge to the evidence on “what works” (cf. Chatterji 2005): in particular, the MMR project discussed by Kathleen Philipp (cf. Philipp 2012; Philipp and Leuders 2012) exemplified how the use of systemic, contextually-grounded studies in early phases may be followed by more sharpened, analytic experimental studies in later research phases. The MMR project discussed by Prediger and Wessel (2013) and Wessel (2014) underscored that the combined use of more than one research method may help us reveal patterns, develop new constructs and deepen the understanding of relationships between constructs, especially causal relationships. However, as Prediger expressed it in her interview, it is not necessary “to end with a randomized controlled trial, because we want to understand how the things work and what really happens in the processes.”

9.4 Conclusion

The four examples of research presented here can by no means claim to be representative for educational research in mathematics in German-speaking countries. However, they do illustrate several of the characteristics and strategies of research often encountered and explicitly discussed within the research community—in many other countries beyond German-speaking Europe.