In everyday life, people face problems that do not include all the essential information, requiring assumptions to be made and allowing for multiple solutions. For example, if you must arrive at work at 8 a.m. and you know that the bus ride takes about 20 min after the 5-min walk from your home to the bus stop, you might consider factors, such as how much time you need to shower and have breakfast or how much traffic you expect on the bus route, before setting your alarm the night before. On the basis of research on open(-ended) problems (Nieminen et al., 2022; Silver, 1995; Yeo, 2017) and modelling (Cevikbas et al., 2022; Niss et al., 2007; Schukajlow et al., 2023), we refer to these types of problems as open modelling problems. In the past, many researchers have underlined the importance of open problems in contrast to closed problems (i.e., problems where all the necessary information is given, and only one correct solution is possible; Becker & Shimada, 1997; Silver, 1995; Stacey, 1995). Dealing with openness involves making assumptions, which is a fundamental mathematical activity for proofs, modelling, and the acquisition of mathematical knowledge (Stylianides & Stylianides, 2023). Prior research has demonstrated that students have trouble solving open problems (Cai, 1995) and more specifically open modelling problems (Brown & Stillman, 2017; Ng, 2018). However, not much is known about cognitive barriers while solving open modelling problems or about how to help students overcome these barriers.

Our first aim was to analyze students’ cognitive barriers as they solved open modelling problems. To do so, we used prior research to develop instructional prompts that focused on the potential barriers. The effects of these prompts on students’ performances should indicate the significance of the potential barriers involved in solving open modelling problems. The second aim was to analyze whether these instructional prompts support students’ solution processes. To address these aims, we conducted a mixed-method sequential study that consisted of a quantitative experimental study and a qualitative study, which helped us describe and differentiate between the barriers, thereby providing context for interpreting the results of the quantitative study.

1 Theoretical models and empirical results on instruction in solving open modelling problems

In this section, we present a literature review on open problems, open modelling problems, and instructional prompts.

1.1 Open modelling problems

Open problems are central to the discipline of mathematics and mathematical thinking. According to Silver (1995), in mathematics and mathematics education, the term “open(-ended) problem” is used for problems that do not yet have a solution (e.g., the Riemann hypothesis), for problems that have multiple solution methods (e.g., graphical and numerical) or multiple results (e.g., “How much toothpaste do you use in a month?”), or for problems whose solution calls for new problems to be posed or generalizations to be made (e.g., “Find three consecutive integers whose sum is divisible by three” generalized to “The sum of three consecutive integers is always divisible by three”). Researchers have characterized open and closed problems by referring to their structure (Jonassen, 1997; Simon, 1973). In closed (well-structured) problems, the initial state, goal state, and intermediate states are all clear. This type of problem includes all the information needed to solve the problem. Solving closed problems requires problem solvers to apply well-known procedures that will lead them to one correct result at the end of the solution process (e.g., 4 + x = 5). By contrast, open (ill-structured) problems have a vague initial state, a vague goal state, or vague intermediate states (i.e., ill-structured problems lack a clear definition of their solution spaces (Simon, 1973)). Open problems can be solved by applying multiple solution methods, and they can have multiple results (Klavir & Hershkovitz, 2008). In our study, we analyzed problems with a clear goal state but a vague initial state and a vague intermediate state, so that solving them required problem solvers to make assumptions, which led to different correct solutions. Open problems can be used to prompt students to learn mathematical modelling.

Processes by which information is transferred between reality and mathematics are the core of mathematical modelling (Niss et al., 2007; Schukajlow et al., 2023). These transfer processes are usually described by so-called modelling cycles. Many modelling cycles (e.g., Blum & Leiss, 2007; Galbraith & Stillman, 2006; Verschaffel et al., 2000) emphasize the importance of understanding the real-world situation; idealizing, structuring, and simplifying it; constructing a mathematical model; applying mathematical procedures; interpreting the mathematical results; and validating the final result.

Openness in modelling problems can refer to the following three characteristics:

  • Openness of the initial state of the problem: Numerical or non-numerical data that are essential for solving modelling problems are missing or vague (see the example in Fig. 1).

  • Openness of the intermediate states: The models and mathematical procedures are not given in the problem.

  • Openness of the goal state: The question is vague, and it must be specified by the problem solver. For example, in the Speaker problem (Fig. 1), the question can be: “Is it worthwhile to buy the box?” The expression “Is it worthwhile” is vague and can refer, for example, to the financial cost or to an unnecessarily large amount of paper needed to package the speaker.

Fig. 1
figure 1

Open modelling problem: Speaker

Full size image

One example of a modelling problem with a vague initial state and a clear goal state is the Speaker problem (see Fig. 1). This problem is missing information about the position of the speaker in the box and information about the diameter of the speaker.

Any of the activities described in modelling cycles can represent cognitive barriers when solving modelling problems (Blum, 2015; Galbraith & Stillman, 2006; Schukajlow et al., 2023). In the case of modelling problems with an open initial state, it can be very demanding to structure, simplify, and idealize the given situation. As these activities occur at the beginning of the modelling process, it is crucial to overcome barriers in these activities in order to develop a meaningful solution. On the basis of prior research (Dewolf et al., 2017; Galbraith & Stillman, 2001; Ge & Land, 2004), we distinguish between three barriers in dealing with the openness of a problem while solving open modelling problems (see Fig. 2).

Fig. 2
figure 2

Barriers in the processing of modelling problems with an open initial state (adapted from Krawitz et al., 2018)

Full size image

From primary school to university, students have been found to overlook or ignore the openness of problems while solving them (Verschaffel et al., 2000). Analyses of students’ solution processes (Chang et al., 2020; Galbraith & Stillman, 2001; Kanefke & Schukajlow, 2022) have indicated that students have trouble noticing that information is missing when solving modelling problems. They often ignore the openness of the problem, link numerical data from the problem by using a familiar mathematical procedure, and thus, process open problems in a similar manner as closed problems. In the Speaker problem, for instance, students tend to ignore the fact that some important information is missing and calculate the diagonal of the cuboid with the lengths, width, and height of the box (\(\sqrt{{14}^{2}+{14}^{2}+{10}^{2}}\approx 22.2\)), compare it with the height of the speaker (18.4), and conclude that the speaker fits in the box. Students’ beliefs about problems were found to be the primary reason for this behavior (Djepaxhija et al., 2015). One of these beliefs is that all numerical data must be given in the problem. Obstacles in noticing the openness of problems turned out to be greater for problems that included numerical information than for problems that did not include any numerical information for which the openness of the problem was obvious (Krawitz et al., 2018).

A second barrier is that it is challenging to identify the unknown quantities that are needed to find the solution. To solve the Speaker problem, students must identify the diameter of the speaker as an unknown quantity. Hankeln (2020) found that some students encountered significant barriers when they were missing information needed to solve the problem and put a lot of time and effort toward finding out what information they needed to solve the problem. The barriers occurred not only at the beginning but also later in the solution process when students were setting up a mathematical model or while they were validating the model and results (Anhalt et al., 2018; Czocher, 2018). These difficulties are typical of novices, who tend to apply the mathematical procedures immediately. By contrast, experts usually take time at the beginning of the process to identify what quantities are needed and to make assumptions (Ge & Land, 2004; Voss & Post, 1988).

A third barrier is that it is challenging to make realistic numerical assumptions. In the Speaker problem, after identifying the importance of the diameter, students have to make a realistic numerical assumption about it, for instance, that it might be about 6 cm (e.g., estimating that the diameter of the speaker is about one third of its height). Some students identify a quantity that requires them to make an assumption but do not proceed further in the solution process, or they make inappropriate assumptions (Kanefke & Schukajlow, 2022). For example, while solving a problem that required the growth of tree leaves to be modeled, preservice teachers used inadequate assumptions that led to unrealistic solutions by assuming that the leaves fall off a tree after exactly 87 days (Anhalt et al., 2018). One important reason for inadequate numerical assumptions is a lack of knowledge about the real-world phenomenon described in the task (Krawitz, 2020). Making assumptions requires students to activate their knowledge about the real-world situation and to apply various strategies to master these requirements (Ärlebäck, 2009; Ferrando & Albarracín, 2021).

1.2 Instructional prompts for solving open modelling problems

Teaching mathematical modelling is demanding for both teachers and students (Niss & Blum, 2020, p. 2ff). Research on setting up challenging tasks indicates that maintaining the cognitive demands while introducing the task is very important for students’ opportunities to learn (Jackson et al., 2013). For modelling problems, Geiger et al. (2022) observed that many teachers either ask students to work without enough of an introduction or give them too many hints before or while they are solving the problems. Despite the importance of research on interventions in the area of modelling, there is a clear gap in this field (Schukajlow et al., 2023). It is important to know what kinds of instructional prompts help students overcome cognitive barriers while solving open modelling problems. Instructional prompts are brief instructions that can be used to scaffold students’ learning by including them in static scaffolds, such as solution plans (Schukajlow et al., 2015), or by using them dynamically in teacher interventions. Teachers can use instructional prompts while setting up a task, during the solution process, or while students’ solutions are discussed in the classroom.

Two types of instructional prompts that differ in the specificity of instructions are general and task-specific prompts. A general instructional prompt comprises a warning to be careful while solving the problems on a test. These prompts did not affect students’ abilities to notice that the problems did not include all the information needed for their solution (Reusser & Stebler, 1997, p. 319). Effects of task-specific prompts that are tight to the specific task were mixed. The percentages of realistic solutions were higher if each task included information that warned students to be careful and think about the solution before answering (Reusser & Stebler, 1997). However, as realistic solutions in this study included students’ answers that the problem could not be solved, we do not know how students’ performances were affected by this instructional prompt. In studies with word problems, task-specific instructions that were aimed at drawing students’ attention to missing information did not affect the number of realistic solutions. An example of a word problem that requires assumptions to be made about missing information is the Rope problem: “Mr. Meier wants to have a rope long enough to stretch between two poles that are spaced 12 m apart, but the pieces of rope he has are only 2 m long. How many of these pieces would he need to tie together to stretch between the poles?” (Krawitz et al., 2018). While solving the Rope problem, highlighting the knots in the picture showing the process of making a larger rope from smaller pieces of rope did not increase the number of students who took the additional length of the rope into account (Dewolf et al., 2017). By contrast, asking students to provide two solutions and practicing a similar problem was found to improve students’ performances on the Rope problem (Krawitz et al., 2018). One explanation for this finding is that requiring students to develop two solutions implies that this problem can be solved in different ways. Some evidence of the effects of instructional prompts has also come from research on modelling problems. In the project on the development of a framework for fostering mathematical modelling, researchers discussed with teachers the possibility of making different assumptions while solving a problem (Geiger et al., 2022). Teachers who implemented these instructions in the classroom reported positive effects of these instructions on students’ progress in modelling. Positive effects of instructions to make assumptions (“Look for the data you need and, if necessary, make assumptions!”) were found in a study that analyzed the effects of a solution plan on modelling (Schukajlow et al., 2015). We conclude that instructional prompts affect solution processes if the instructions are specific enough to help students overcome their inadequate beliefs about the problems and if students have sufficient knowledge about how to integrate the information that is included in the prompts into their solution process. Furthermore, students who receive instructional prompts might learn that some information in open modelling problems can be missing and may thus transfer this knowledge to open modelling problems that do not include task-specific instructions. However, the differential effects of instructional prompts on the barriers in the solution process of open modelling problems are still unknown.

2 Aims, research questions, and hypotheses

The present studies were conducted within the framework of the Offene Modellierungsaufgaben in einem selbständigkeitsorientierten Unterricht (OModA, in English: Open Modelling Problems in Self-Regulated Teaching) project. The OModA project is aiming at investigating cognitive, strategic, and affective conditions for the teaching and learning of open modelling problems. In a prior study, we analyzed students’ interest in solving open modelling problems and its relationship to students’ performance (Schukajlow et al., 2022). In the present study, we aimed to identify the optimal cognitive conditions for the teaching and learning of open modelling problems. In the quantitative study (RQ1 and RQ2), we compared the performances of two groups that received different types of instructional prompts with the performance of a control group (CG) without prompts. One intervention group received instructional prompts about the openness of the problem and about what quantity was unknown (unknown quantity group (UQG)). The other intervention group received instructional prompts about what quantity was unknown and information about the assumptions to be made (assumption group (AG)). In the qualitative study (RQ3 and RQ4), we aimed to achieve in-depth insights into students’ mathematical thinking, which should help explain the results of the quantitative study. The studies were conducted in 2021 and 2022.

  • RQ1: How do task-specific instructional prompts for solving open modelling problems affect students’ performances in making realistic numerical assumptions and in solving the modelling problems?

  • Hypothesis 1.1: Students in the UQG outperform students in the CG.

  • Hypothesis 1.2: Students in the AG outperform both students in the UQG and students in the CG.

  • RQ2: How do task-specific instructional prompts for solving open modelling problems affect students’ performances in making realistic numerical assumptions and in solving modelling problems on an additional problem that does not include instructional prompts?

  • Hypothesis 2.1: Students in the UQG outperform students in the CG in making realistic numerical assumptions and in modelling.

  • Hypothesis 2.2: Students in the AG outperform students in both the UQG and the CG in making realistic numerical assumptions and in modelling.

  • RQ3: What cognitive barriers do students experience regarding the openness of modelling problems?

  • RQ4: How do task-specific instructional prompts about unknown quantities and suggestions for numerical assumptions for these quantities help students overcome these barriers, and how do students process the subsequent problem that does not include an instructional prompt?

3 Quantitative study

3.1 Methodology

A total of 363 German ninth, 10th, and 11th graders (49% female; mean age 15 years) from six comprehensive schools (German Gesamtschule) and three grammar schools (German Gymnasium) volunteered to be in this study. Students from the current sample had not participated in any specific educational program and did not have any previous experience with open modelling problems.

Students within each class were randomly assigned to the CG, UQG, or AG. Before the treatment, they filled out questionnaires (10 min), and after the treatment, they took a test on intramathematical problems (i.e., problems without a connection to reality; 15 min). The procedure consisted of a treatment phase and a transfer phase (55 min). In the treatment phase, students in the UQG and AG worked on open modelling problems with instructional prompts, and in the transfer phase, on an open problem without instructional prompts. Students in the CG did not receive any instructional prompts. All paper–pencil tests and materials were distributed by trained master students, who were advised to follow a detailed description of the procedure of the study from a manual.

3.1.1 Material

We developed or adapted seven modelling problems to test students’ performances in making realistic assumptions and in solving modelling problems (see examples in Figs. 1 and 3). The material was originally in German. All problems could be solved by using the Pythagorean theorem. Students were familiar with this mathematical topic. In a first pilot study in spring 2021 with 143 students, we tested how students solved these problems, and we developed a coding manual (Schukajlow et al., 2022). In a second pilot study in autumn 2021 with 54 students, we examined the comprehensibility of the instructional prompts.

Fig. 3
figure 3

Open modelling problems: Shortcut Route, Tree, TV, and Fence

Full size image

The Shortcut Route problem is missing information about the speed limits in the residential area (usually up to 30 km/h in Germany) and on the federal road (usually up to 100 km/h in Germany). For the solution to the Tree problem, an assumption must be made about the part of the pole that is hammered into the ground. In the TV problem, information about the width of the frame is missing. In the Fence problem, students need to make assumptions about the lengths of the ends of the crossbeam.

3.1.2 Instructional prompts

Students in the UQG and AG received instructional prompts for six of the problems. The instructional prompts in the UQG included information about what quantity was unknown and had to be estimated in the problem (see the examples in Table 1). In the AG, students received instructional prompts about the assumptions that must be made for the unknown quantities (see examples in Table 1).

Table 1 Examples of instructional prompts for the UQG and AG
Full size table

3.1.3 Performance in making realistic numerical assumptions and in solving modelling problems

Two trained master students independently coded students’ solutions to the seven problems on realistic numerical assumptions and correctness. Students’ solutions were scored 1 if they included a realistic assumption and 0 if they did not. For example, in the Speaker problem (Fig. 1), an assumption would refer to the speaker’s diameter. An unrealistic assumption for a diameter of the speaker was, e.g., 13 cm, whereas according to the picture, the diameter is obviously less than half of its height which is 18.4 cm. Intercoder reliability (Cohen’s kappa) ranged from 0.74 to 1.00 for the seven problems. The internal consistency (Cronbach’s a) for the six problems used in the treatment phase was 0.78. Students’ solutions to the problems were scored between 0 and 4 for correctness. Each solution could receive 1 point for each of the following: identifying a mathematical model (i.e., identifying the geometrical representation of the real situation), setting up a mathematical model (i.e., setting up a correct equation for the Pythagorean theorem), calculations, and interpretation. Cohen’s kappa ranged from 0.88 to 0.98 for the seven modelling problems used in the treatment phase and Cronbach’s a for the six problems was 0.78.

3.1.4 Intramathematical performance

The intramathematical performance test included six Pythagorean theorem problems, all of which had been used in prior studies (e.g., Krawitz et al., 2022). The internal consistency of the test was 0.80. An ANOVA demonstrated that students in the CG, UQG, and AG did not differ in their intramathematical performance, F(2, 363) = 0.460, p = .632, indicating that random assignment went as intended.

3.1.5 Statistical tests

To answer RQ1, we used a univariate ANOVA with the independent factor treatment condition (CG, UQG, and AG) and dependent variables performance in making realistic assumptions or performance in solving problems. As the assumption of homogeneity of variance was violated according to Levene’s test, we used Welch’s t test to calculate p- and t-values for post hoc tests. To answer RQ2 regarding the solution to the Fence problem, we used a nonparametric Chi-square test. However, the results of the Chi-square test should be interpreted with caution, as some of the observed numbers in each cell were less than 5.

3.2 Results

Descriptive statistics for students’ scores on the performance test are presented in Table 2. The low scores in the CG show that it was challenging for students to make realistic assumptions and solve the problems.

Table 2 Means and standard deviations in the CG (N = 119), UQG (N = 126), and AG (N = 118)
Full size table

RQ1 asked how instructional prompts affect students’ performances in making assumptions and solving problems. As expected, the students in the three groups differed in their performance in making assumptions, F(2, 361) = 50.41, p < .001, η2 = 0.219. Post hoc t tests revealed that students in the UQG outperformed students in the CG, t(129) = 7.15, p < .001, Cohen’s d = 0.88 and students in the AG outperformed students in the UQG, t(226) = 3.83, p < .001, Cohen’s d = 0.51.

Similar results were also found for the comparison of students’ performances in solving problems. An ANOVA demonstrated differences between the three groups, F(2, 361) = 46.06, p < .001, η2 = 0.204. Post hoc t tests indicated that students in the UQG again outperformed students in the CG, t(130) = 6.87, p < .001, Cohen’s d = 0.87 and students in the AG outperformed students in the UQG, t(216) = 3.873, p < .001, Cohen’s d = 0.50.

In line with Hypothesis 1.1, instructional prompts about the unknown quantities that had to be estimated in the problem had positive effects on students’ performances in making assumptions and in solving problems. In line with Hypothesis 1.2, in addition to prompts about the unknown quantities, instructional prompts that included information about the assumptions to be made had stronger effects on students’ performances than just instructional prompts about the unknown quantities, which in turn had a stronger positive effect when compared with the group that was not given any instructional prompts.

RQ2 focused on students’ performances in solving a problem that did not include instructional prompts. Therefore, it addressed the question of whether students are able to transfer the effects of instructional prompts dealing with openness to other open modelling problems. A total of 13% of the UQG, 8% of the AG, and 3% of the CG made realistic assumptions while solving the Fence problem (see Table 3). Chi-square tests indicated differences between the distributions of the scores in the three groups for students’ performances in making realistic assumptions, χ2(2) = 7.048, p = .015. Results of post hoc tests supported Hypothesis 2.1 and partly supported Hypothesis 2.2. Students in the UQG and AG made realistic assumptions more often than students in the CG, χ2(1) = 7.12, p = .004; χ2(1) = 2.79, p = .04, respectively. However, no differences in making realistic assumptions were found between the students in the UQG and AG, χ2(1) = 1.14, p = .14.

Table 3 Solution score distributions (percentages and numbers) for the Fence problem (transfer phase) in the CG, UQG, and AG
Full size table

A total of 8% of the UQG, 6% of the AG, and 3% of the CG solved the problem completely correctly (see Table 3). Chi-square tests did not indicate differences between the distributions of the scores in the three groups for students’ performances in solving the problem, χ2(4) = 5.328, p = .13. However, the UQG and CG showed differences in their performances, χ2(2) = 5.413, p = .033, but there were no differences between the UQG and AG or between the AG and CG (both ps > .10).

These results supported Hypothesis 2.1 and also partly supported Hypothesis 2.2. Instructional prompts about unknown quantities that must be estimated in the problem had positive effects on students’ performances in making assumptions while solving a problem that did not include an instructional prompt. Furthermore, instructional prompts had small but significant effects on students’ performance. In addition to prompts about unknown quantities, instructional prompts that also included information about the assumptions to be made affected students’ performances in making assumptions when solving the problem that did not include an instructional prompt, but they did not affect students’ performances.

3.3 Discussion of the quantitative study

Positive effects of instructional prompts indicate that difficulties in identifying unknown quantities and making realistic numerical assumptions are considerable barriers that students face when solving open modelling problems. These results explain students’ difficulties in making assumptions observed in prior studies, which found out that some students do not notice the openness, some students notice it but conclude that such problems cannot be solved, and other students make unrealistic assumptions (Anhalt et al., 2018; Chang et al., 2020; Dewolf et al., 2017; Galbraith & Stillman, 2001; Hankeln, 2020; Reusser & Stebler, 1997). Providing students with instructional prompts that help them identify unknown quantities and make realistic numerical assumptions about these quantities helped some students overcome the barriers and develop a realistic solution for the respective modelling problems. Therefore, these prompts can be used as a scaffolding method in class if the goal is to improve students’ processing of a specific problem.

An analysis of the transfer of the understanding that some quantities might be unknown and that it might be necessary to make assumptions to solve a modelling problem revealed a mixed picture. It was a promising result that a few students from the UQG and AG included numerical assumptions in their solutions to the problem that did not include an instructional prompt. One explanation for this finding might be that students’ belief that no assumptions are needed to solve mathematical problems had changed. However, the effects were small. Therefore, we conducted a qualitative analysis to shed light on this phenomenon and provide a context for the interpretation of the results of the quantitative study.

4 Qualitative study

4.1 Methodology

Four ninth graders (one female, all about 16 years old) from two grammar schools (German Gymnasium) volunteered to participate in the qualitative study. As the modelling problems were found to be difficult for students in a pilot study (Schukajlow et al., 2022), we selected students with at least average mathematical grades for this study (three students with excellent grades and one with an average grade). One of the participants (Alex; we gave each student a pseudonym) had previously worked on real-world problems that required assumptions to be made.

With each student, we conducted individual sessions that included four phases: instruction on the thinking-aloud method, problem solving, stimulated-recall interview, and semi-structured interview. A session began with a thinking-aloud phase that consisted of a video on how the thinking-aloud method works, time to practice this method, and individual feedback (Schulze Elfringhoff & Schukajlow, 2021; Tropper et al., 2015). In the problem-solving phase, participants solved an open modelling problem without information about the openness of the problem (Shortcut Route problem, Fig. 3), a problem with information about the unknown quantity (“To solve the problem, you must estimate the width of the speaker”; Speaker problem, Fig. 1), a problem with information about the unknown quantity and the assumption that had to been made (“To solve this problem, you must know the width of the frame. Assume that this width is 12 cm”; TV problem, Fig. 3), and finally another problem without any information about openness (Tree problem, Fig. 3). In the stimulated-recall interview, students watched the video of their solution processes, commented on their solution processes, and answered the interviewers’ questions. In the semi-structured interview, the students answered questions about their perceptions of the problems and prior experience with open problems.

4.1.1 Data analysis

The transcripts of the problem-solving videos were sequenced in accordance with the events that occurred during the interview with a focus on the process of dealing with open modelling problems. Sequences of the stimulated recall interviews were assigned to the corresponding problem-solving sequences. The problem-solving sequences were categorized by the second author who applied a qualitative content analysis (Mayring, 2015) and evaluated and discussed the results with the first author. Both authors have applied qualitative data analyses in the area of modelling in the past (e.g., Krawitz, 2020). In the coding process, we used a deductive-inductive approach to data analysis. Deductively, we used (1) noticing the openness and (2) making assumptions as the main categories (see Table 4 for descriptions of these categories). Initially, we began the coding process by identifying all sequences in which students encountered difficulties or mastered noticing the openness or making assumptions. Through this process, we inductively identified two subcategories within the main category of making assumptions. These subcategories were (2a) recognizing that assumptions about unknown quantities can be made and (2b) noticing the need to make assumptions about unknown quantities. Further, we inductively identified a third main category ((3) including assumptions about unknown quantities in the mathematical model). Sequences from this category were found to differ from noticing the openness and making assumptions and were revealed as important in the solution processes regarding the openness of the problems. The coding process was noted in a table with the categories in the rows and the cases in the columns. For example, the sequence “What is the diameter of the speaker? I would say, about as large as my water bottle. […] Okay, it is about 7 cm” was paraphrased as “Estimated the length of the diameter of the speaker (7 cm),” and this was coded as a realistic numerical assumption made by Alex.

Table 4 Barriers with respect to the openness of the problems
Full size table

4.2 Findings

RQ1 addressed students’ barriers with respect to the openness of the problems. In Table 4, we noted the categories that we developed deductively or inductively.

4.2.1 Shortcut Route problem

To answer RQ1, we analyzed students’ solution processes for the Shortcut Route problem. Bella and David did not notice the openness and calculated and compared the distances of the routes in their solutions. David explained in the interview that he briefly considered taking into account speed limits (possibility that assumptions can be made) but decided that it was not important for the solution of the problem. Noticing and dealing with openness was also a challenge for Christopher, who had included only calculations about distances in his solution (Fig. 4). In his answer, he noted that his recommendation was based on distances. In the interview, he emphasized that he was not sure whether the driving time was important (“… time could be also important for the recommendation”) and explained that it would have been better if the problem had explicitly addressed a missing quantity (“I think it would be helpful if the problem included the information that speed limits were not important for the solution”). David’s and Christopher’s solution processes demonstrated that it was not sufficient to notice that it was possible to make an assumption. In addition, it was important to notice that it was necessary to make assumptions in problems.

Fig. 4
figure 4

Christopher’s solution to the Shortcut Route problem

Full size image

Alex directly recognized the importance of making assumptions. He made situational assumptions about the position of two federal roads (“… under the assumption that the road is perpendicular to the junction”) and about driving on the federal road (“Because there are houses next to the road, the driver has to look for pedestrians”). Furthermore, Alex pointed out situational requirements that he did not consider in his solution (“The speed while turning at the junction can be ignored”). Beginning with situational assumptions, he made realistic numerical assumptions about the speed limits (80 km/h on the federal road and 30 km/h in the residential area), calculated the distances and the time needed for the two routes (with some errors in his calculations), and gave a realistic answer to the problem (“It is not worth it because of the speed limits”). Alex’s processing of the Shortcut Route problem demonstrated his deep mathematical thinking about the situation as he retrieved a speed limit (100 km/h on the residential road) from his memory, analyzed the situation (pedestrians near the road and changes in speed because of the turn at the junction), and took the relevant situational considerations into account while making numerical assumptions (adjusting 100 to 80 km/h on the federal road). Alex noticed the openness, recognized the need to make assumptions about unknown quantities, made realistic situational and numerical assumptions, and included the assumptions in his mathematical model. The analysis indicated possible barriers to noticing the openness (Bella) and recognizing the need to make an assumption (Daniel, Christopher).

RQ2 was about students’ thought processes when solving modelling problems that included instructional prompts and about solving a subsequent task with no instructional prompts.

4.2.2 Speaker problem

While solving the Speaker problem, all students considered how they could integrate the unknown quantity (diameter of the speaker) into their solution. Daniel and Bella had trouble making an assumption about the unknown quantity. Daniel decided that the speaker might fit in the box only if it were placed diagonally on the bottom of the box. However, he did not know how to integrate the information about the width of the speaker into his considerations (inability to construct a mathematical model that includes assumptions). Bella placed the speaker on the bottom of the box, calculated the diagonal of the bottom of the box, and compared it with the height of the speaker (Fig. 5). While interpreting the result, she mentioned the unknown quantity (“It depends on the width of the speaker […] the maximum width would be 1.4 cm. I think this is too narrow.”; not recognizing that an assumption can be made). Alex and Christopher made a numerical assumption about the unknown quantity. Christopher estimated the diameter as 6 cm because the calculations were easier, rather than 7 or 7.5 cm, which he considered more realistic assumptions. Alex assumed that the diameter of the bottom of the speaker was similar to the diameter of his water bottle and used the diameter of his water bottle (6 cm) as a proxy for the diameter of the speaker. In the interview, Alex explained that the object at hand helped him make a numerical assumption (“It would be difficult [to make an assumption about diameter of the speaker] if I did not have a water bottle with me”). The analysis of students’ thought processes while solving the Speaker problem demonstrated that students had trouble making numerical assumptions (Daniel and Bella) and integrating an unknown quantity into the mathematical model (Daniel).

Fig. 5
figure 5

Bella’s solution to the Speaker problem

Full size image

4.2.3 TV problem

Solving the TV problem required students to integrate an instructional prompt that included a numerical assumption about the unknown quantity (12 cm for the width of the frame) into the solution. All students were able to integrate this unknown quantity into their solutions (see Christopher’s solution in Fig. 6). An important observation about numerical assumptions was observed in Alex’s thought process. Alex doubted that the width of the frame could be 12 cm and estimated the frame as 6 cm: “Ok, a normal TV does not have a frame of 12 cm… It means probably the whole frame [bottom and upper part of the frame]. It is a Samsung UE46. It looks like a relatively new TV. The frame has to be 6 cm at most.” Even though the information about the unknown quantity was given in the problem, Alex changed it on the basis of his real-world knowledge and made his own numerical assumption. All other students included the numerical assumptions offered in the instructional prompt in their mathematical model.

Fig. 6
figure 6

Christopher’s solution to the TV problem

Full size image

4.2.4 Tree problem

The last problem (i.e., the Tree problem) did not include instructional prompts. None of the four students noticed its openness, and none considered including the part of the pole in the ground or at the place where the pole was bound to the tree in their solutions. All students applied the Pythagorean theorem and calculated the length of the pole as a hypotenuse in a right-angled triangle (see Fig. 7). During the solution process, Alex and Christopher were not sure whether they had overlooked some information (Christopher: “I have a feeling that it was too simple. It could not be the final [solution]. However, I will leave it as it is.”). In the interview, Alex was confident that his solution to the Tree problem was correct (“I liked the Tree problem the most because I think this solution is really accurate. The other solutions included estimations.”). The analysis of solutions to the Tree problem indicates that students cannot easily transfer their experiences in solving problems with instructional prompts to a problem that does not include an instructional prompt.

Fig. 7
figure 7

Alex’s solution to the Tree problem

Full size image

4.3 Discussion of the qualitative study

Qualitative analyses of students’ processing of open modelling problems indicated students’ difficulties in noticing the openness of the problem and making assumptions about unknown quantities as well as the barriers that were proposed theoretically (Krawitz et al., 2018) and confirmed empirically in the quantitative study. Interestingly, the analysis of students’ solution processes indicates that making realistic assumptions about unknown quantities requires them to recognize both the possibility and the necessity of making assumptions. Including assumptions in a mathematical model was found to be a particular challenge for students. These results expand prior findings about barriers to solving open modelling problems (Anhalt et al., 2018; Chang et al., 2020; Galbraith & Stillman, 2001; Hankeln, 2020; Niss & Blum, 2020) by differentiating between barriers in making assumptions and by adding a new barrier to the model of barriers in the processing of modelling problems with an open initial state adapted from Krawitz (2018).

Instructional prompts helped students overcome barriers to solving open modelling problems. A prompt about the unknown quantity encouraged students to consider it in their solution. One interesting finding was how students reacted to this prompt. Some students did not make a numerical assumption explicitly but considered whether a calculated number would correspond to reality. This is an important result that indicates the persistence of students’ beliefs that only information that is given in the problem should be used in their calculations (Djepaxhija et al., 2015). Another implication from this finding is that this prompt enhances the validation of results, which was observed to be a rare but important modelling activity in the past (Czocher, 2018; Schukajlow et al., 2023). A prompt that included an unknown quantity and a numerical assumption was revealed to be effective for dealing with missing information in the specific task. One interpretation of this finding is that this instructional prompt transforms the open problem into a closed problem, a type of problem students are familiar with. Unexpectedly, none of the four students was able to transfer their experiences from solving problems with instructional prompts to a new problem. One explanation might be the difficulty of noticing the openness in the Tree problem. As one of the students was able to recognize the openness of the first problem but did not recognize it in the Tree problem, we hypothesize that noticing the openness of a problem strongly depends on the problem at hand. This result is in line with previous research that revealed the importance of the type of problem for whether students could notice their openness (Krawitz et al., 2018) and emphasizes the challenges involved in designing modelling problems (Geiger et al., 2022).

5 Summary, overall discussion, limitations, and conclusion

In this study, we aimed to uncover cognitive barriers to solving open modelling problems and examining the effects of instructional prompts on students’ performances and solution processes. The quantitative study indicated that students’ inabilities to notice the openness of a problem and identify unknown quantities were important barriers to solving open modelling problems. The failure to make numerical assumptions is another important barrier that can prevent students from solving an open problem. These findings validate important parts of the model that we adapted from a prior study (Krawitz et al., 2018). Our findings add to results from quantitative studies that were conducted with word problems (Djepaxhija et al., 2015; Verschaffel et al., 2000, 2020) and findings from qualitative studies in the area of modelling (Anhalt et al., 2018; Galbraith & Stillman, 2001; Hankeln, 2020; Kanefke & Schukajlow, 2022). The qualitative study helped us understand the results of the quantitative study by illustrating the significance of the hypothesized barriers, by differentiating between barriers to the possibility versus the necessity of making assumptions, and by identifying the new barrier to integrating assumptions into the solution. These novel results contribute to the theory of solving open modelling problems.

Another important finding from our study is about the positive effects of task-specific instructional prompts on students’ solution processes and performances. Prior research demonstrated that teachers should maintain cognitive demands while introducing challenging tasks (Jackson et al., 2013) such as open modelling problems. The advantage of the instructional prompts is that they do not take much time, can easily be integrated into regular lessons, and can be handed over to students who cannot overcome the barriers on their own. Instructional prompts about unknown quantities and about assumptions that had to be made supported individual solution processes. The effects of instructional prompts that included assumptions about the unknown quantities were stronger than the effects of instructional prompts that simply suggested that some quantities were unknown. We expect that the effects of these instructional prompts can be even stronger if students work on the problems in groups and can discuss how to use these prompts to solve the problem. However, our quantitative study indicated that instructional prompts do not improve the correctness of solutions for the majority of students. The qualitative study shed light on these results and suggested that students could not integrate the instructional prompts into their mathematical models. Future studies should analyze how students’ individual characteristics (e.g., mathematical knowledge or reading comprehension) affect the effects of instructional prompts. Furthermore, future research should address the question of which teaching methods help students integrate the instructional prompts into their mathematical models.

Uncovering a transfer effect of instructional prompts to a new open problem is another important new result of our study. Brief instructional prompts about what quantity is unknown in a specific problem can evoke learning processes. However, the magnitude of these transfer effects was small. We expect stronger effects if students receive more comprehensive instructions focused on the barriers to solving open modelling problems and practice how to solve open modelling problems in the classroom. In the past, integrating instructional prompts into scaffolding instruments such as solution plans or encouraging small group discussions was found to be effective ways to improve students’ modelling competence (Ärlebäck, 2009; Durandt et al., 2022; Ferrando & Albarracín, 2021; Parhizgar & Liljedahl, 2019; Schukajlow et al., 2015). Interestingly, to solve the task in the transfer phase, instructional prompts that pointed out an assumption about the unknown quantity were less helpful for finding a solution than prompts about identifying the unknown quantity. This unexpected result indicates that some instructional prompts have stronger effects on the solution process of a particular problem, whereas other instructional prompts foster transfer processes instead. To help students learn to solve open modelling problems, it seems to be more promising to state which quantity is unknown and encourage students to make their own numerical assumptions than to give them a specific number to use as an assumption in a given problem.

Furthermore, as the transfer effects were small, the design and evaluation of teaching methods that can help students learn how to deal with open problems are an important direction for future studies. A qualitative analysis in which students did not notice the openness indicated that the complexity and the context of the problem might be important for noticing the openness. If students are not familiar with the context, they could have problems noticing the openness and making realistic assumptions. Future studies should investigate how making numerical assumptions is related to task design.

The study has several limitations. In our quantitative study, we used a between-subjects design by randomly assigning students in each class to one of three conditions. This design was used to avoid having an interaction between the instructional conditions as would be the case for a within-subjects design. However, our design did not guarantee that the three groups were equal. As we did not find differences in intramathematical performance, we consider the groups to be comparable. Another limitation is that we used a convenience sample. Our results are not representative of Germany, and the results might be different for other populations (e.g., primary school students) or for other cultures (e.g., Eastern cultures). Furthermore, as many students could not solve the Fence problem, it would be interesting to analyze types of errors that occurred in students’ solutions in future studies. In our qualitative study, we used a within-subjects design. Consequently, the instructional prompt offered for the second problem could also have affected students’ solutions to the third problem, which included another type of instructional prompt. The results may have been different if each student had received only one type of instructional prompt. Finally, we used this mixed-method study to test two different effects: one involving cognitive barriers and one the effects of instructional prompts. A limitation of this approach is that these claims are partly dependent on each other, as results on cognitive barriers rely on effects of instructional prompts to some extent. However, the advantage of this approach is that it contributes to two different lines of research: one on cognitive barriers in solving problems and one on instructional prompts.

Our mix-method study concludes that solving open modelling problems is demanding for students. Noticing the openness, identifying unknown quantities, making realistic numerical assumptions about the unknown quantities, and integrating these assumptions into the mathematical model are important cognitive barriers in solving open modelling problems. Instructional prompts about unknown quantities and needed numerical assumptions improve students’ solution rates and in parts also stimulate the performance in solving a transfer problem. Thus, our study contributes both to the theory of mathematical modelling by describing barriers in solving modelling problems and to the teaching practice of mathematical modelling by demonstrating positive effects of instructional prompts on the performance in solving modelling problems.