Heuristic Trees as a Digital Tool to Foster Compression and Decompression in Problem-Solving

Abstract

This design-based study addresses the issue of how to digitally support students’ problem-solving by providing heuristics, in the absence of the teacher. The problem is that, so far, digital tutoring systems lack the ability to diagnose students’ needs in open problem situations. Our approach is based on students’ ability to self-diagnose and find help. To this purpose, we introduce a new type of digital, interactive, help-seeking tool called a heuristic tree. Students’ use of this tool is supported by a help-seeking flowchart. The design of heuristic trees is based on our reinterpretation of the notion of heuristic in terms of terms of compression. Our research question is: How do heuristic trees and the help-seeking flowchart influence students’ problem-solving behaviour? This question was studied in the context of a number theory course for in-service mathematics teachers. During five weeks, fifty students worked on fifty-five problems supported by heuristic trees. Our data consists of video observations of two small groups of students, a teacher log, interviews with these two groups, and a survey filled in by twenty-three students. The main results are that the support by heuristic trees and the help-seeking flowchart allows students to work in the absence of a teacher and to engage strongly with problems, maintaining ownership of the solution methods. Moreover, as intended by the tree structure, students learned to focus not just on the small steps of the solutions, but also on the general heuristic techniques, theorems, and concepts that should be learned in the process of finding those solutions.

In many mathematics courses, students work on problems in the absence of a teacher, either at home—in particular during a pandemic—or unattended in the classroom. Common practice with text book problems is to provide a section with hints. However, a single hint cannot be tailor-made to all students’ needs. A more sophisticated solution is to provide support within an intelligent tutoring system (see, for example, Roll et al., 2014), but there are limits to the diagnostic capacities of such systems in open problem situations. In such situations, a system easily provides more help than needed, spoiling a student’s sense of ownership of the solution approach. In the absence of support, students’ work on problems can be inefficient—being stuck on a problem for a long time—or students may reside to consulting the answer model, reducing engagement and sense of ownership. So, the challenge is to improve the way to support students working on open problems in digital learning environments.

In this study, we propose a form of teacher-independent digital support that invites students to diagnose their own needs and to “navigate” to suitable help, promoting efficient work and ownership of the solution. By “teacher-independent”, we mean that teachers are not available in their role to help students diagnosing why they are stuck or forming a suitable new or adapted approach. The course that forms the context of this study aims both at teaching problem-solving and at teaching through problem-solving. The subject of the course—number theory—is addressed through proper problems not of reproductive nature, and meanwhile, students are stimulated to develop their general problem-solving skills. The proposed tool—a heuristic tree—structures the support in a way that integrates these two goals.

We introduce a way to model support on compression, a central organisational feature of mathematical knowledge. Compression is the process in which a category of objects, a multi-step technique or procedure, or a piece of theoretical reasoning is encapsulated in a single new concept (Tall, 2013).This phenomenon is described as a thought process (Thurston, 1990) and is observed in spoken and written language (Sfard, 2008). Proficient problem-solvers express their solution strategy in compressed, abstract language. For instance, the sentence “By constructing a perpendicular line, I found a right triangle to which I applied the Pythagorean theorem” is easily understood by a more advanced secondary school student, but is too compressed for a novice. For a novice, the process of constructing perpendiculars, the concept of a right triangle, and the statement and application of the Pythagorean theorem are intricate, extensive, and neither easily applied nor understood in one sentence because they are the result of compression. Once a problem approach is found in a compressed discourse, for each step or layer, the details need to be filled in and conditions need to be checked. We term this process decompression. The dynamical process of compression and decompression plays a central role in mathematical thought and complex problem-solving (Arnon et al., 2014; Sfard, 1991, 2008; Tall, 2013). The goal of providing support through heuristic trees is to foster the development of such a compressed discourse in students, as well as an ability to decompress knowledge in concrete problem situations.

The aim of our research is to study students’ use of a tool (a heuristic tree) that supports their problem-solving process while highlighting the compression–decompression dynamic. Heuristic trees were introduced by Bos (2017) and further studied by Lemmink (2019). They allow students to decompress help in the form of heuristic techniques, general concepts, and theoretical statements into their components, by browsing through a tree-like structure (see Fig. 1). The digital heuristic tree is presented as an interactive image in which the text in the nodes becomes visible by clicking. In order to guide students when to use the heuristic tree, a help-seeking flowchart is provided.

Fig. 1

A fragment of a heuristic tree with all help-cards opened

Theoretical Framework

Three theoretical perspectives informed this study: heuristics in problem-solving, compression and decompression, and help-seeking in digital environments. These perspectives lead to design principles for heuristic trees, listed below in the next section.

Heuristics in Problem-Solving

The book How to Solve It by George Pólya (1945) discusses how teachers guide students while solving problems using heuristic support. Problem-solving is understood as the process of overcoming the obstacles that compose the problem: the task is not solved as a routine process, but instead a creative act is involved. The book contains a wealth of heuristics and example problems to which they are applied. In these examples, the teacher plays a guiding role. The role of the teacher is to diagnose what the student needs, to introduce a suitable heuristic and then to explain how it is applied in this context. Moreover, Pólya discerned four phases in the process of problem-solving: (1) understand the problem, (2) make a plan, (3) carry out the plan, and (4) look back.

Schoenfeld (1985) developed Pólya’s ideas into a general framework of what people know and do when they work on mathematical problems. Schoenfeld distinguished four categories: resources, heuristics, control, and belief systems. Resources consist of the basic mathematical knowledge—in our words: concepts, techniques, propositions—available to the student to attack a problem. According to Schoenfeld, heuristics are general techniques, strategies, or rules of thumb that suggest how to understand a problem and progress towards a solution. Later, we shall present the interpretation of what heuristics are that inspires this study. “Control” refers to issues of metacognition, e.g. of making and abandoning plans and of finding or refusing help. We return to this important aspect of problem-solving when we discuss our help-seeking flowchart. Finally, belief systems refer to the perspective with which one approaches mathematical problems. Though we acknowledge the importance of this aspect, it is not addressed in this study.

In response to Pólya’s book, many scholars and teachers tried to teach heuristics in a way that enables students to transfer them to new problem situations, but this turned out to be complex. Schoenfeld (1985) states that:

Many heuristic labels subsume half a dozen strategies or more. Each of these more precisely defined strategies needs to be fully explicated before it can be used reliably by students. (p. 73)

Without the underlying subject domain knowledge, these abstract strategies are hard to transform to concrete approaches. He had success by teaching concrete heuristic techniques for specific types of problems, such as:

Consider a similar problem with fewer variables. If the problem has a large number of variables and is too confusing to deal with comfortably, construct a similar problem with fewer variables. You may then be able to (a) adapt the method of solution to the more complex problem (b) take the result of the simpler problem and build up from there. (p. 195)

instead of teaching general heuristics, like “simplify the problem situation”. These two approaches—using general heuristics and using concrete heuristics—lead to the help-seeking flowchart which we will also present below in the next section.

Schoenfeld concludes that explicit instruction in heuristic techniques can make a difference. He brings an interesting nuance to this statement:

The most probable interpretation of what took place during the practice sessions is that the explicit mention of the heuristic techniques served to bring those skills to the students’ conscious attention and to help them codify and reorganizing their existing knowledge in such a way that those skills could now be accessed more readily. (p. 209)

His formulation (“codify and reorganizing their existing knowledge in such a way that those skills could now be accessed more readily”) alludes to a more modern description of what happens: compression and decompression of a technique. For our study, we take from this that suitable organisation of successful heuristic support explicates general heuristics into more concrete heuristic techniques, and applies general heuristics to a particular problem. This is our first motivation for design principle 1 for heuristic trees. Another design principle, number 3, incorporates Pólya’s structure in problem-solving approaches.

Lester (2013) listed some problems in research on problem-solving instruction, some of which we address in this study. He claims that there is too little attention to the role of the teacher in problem-solving research and acknowledges the important role of the teacher in observing students and listening to them, making sense of what they do, recognising their issues, and responding appropriately. We agree to this important role, but believe it is actually too much to ask for in the setting of a classroom full of students. In such a setting, teachers lack time to address all students’ needs. This is where heuristic trees come in as an additional source of support for students. Moreover, Lester and Cai (2016) state that, “In addition to making problem solving a regular part of everyday instruction, homework can extend learning opportunities and engage students in independent problem solving”(p. 121), and in a homework setting, the teacher will not be around to play these roles. Again, for this situation, heuristic trees could provide a solution. We conclude that heuristic trees are meant as a tool to support students in practical situations where the teacher is not able to do so.

Compression and Decompression

Compression is characterised by a shift of attention from a multitude of phenomena to common properties of those phenomena together with a filing away of details in a reliable way to long-term memory or reference books (Thurston, 1990).Compression is a cognitive process of reorganising mathematical knowledge, observable in the language of interpersonal mathematical discourse (Sfard, 2008). Compression applies to three aspects of mathematical organisation: objects, procedures, and statements. Compression is not unique to mathematics, but unique to mathematics is the way compressed content can be part of many new rounds of compression creating deeply nested abstract structures. The type of compression we discuss should not be confused with the notions of compression in mathematical logic or information theory.

We distinguish two forms of compression: compression on cases—cases of objects, cases where procedures apply, and cases where statements apply—and compression on steps—steps in a technique or details of a proof of a statement. Compression on cases of mathematical objects means that a multitude of objects is treated as instances of an overarching category. Similarly, reorganising knowledge on procedures and statements, such that separate cases are treated uniformly, is a form of compression on cases. This reorganising is characterised by a shift of attention to properties of the category instead of properties of the individual objects. Literature provides various vocabularies to discuss the phases, stages, or levels of abstraction processes that apply to compression on cases (Hershkowitz et al., 2001; Tall, 2013; van Hiele, 1986; White & Mitchelmore, 2010).

Compression on steps has been described within several frameworks: for example, operational-structural by Sfard (1991), APOStheory by Dubinsky, and collaborators (Arnon et al., 2014) and procept theory by Tall (2013). It means knowledge is re-organised such that separate steps of a technique or proof (a line of reasoning) are treated as part of an overarching process. Attention shifts to properties of the technique, and details are filed away in long-term memory.

For example, consider proof by induction—this is one of the heuristics taught in the experiment by Schoenfeld that we discussed above. On the one hand, the goal is compression on cases: students need to familiarise themselves with separate cases where induction applies; recognise the similarity between them (in each case, a statement $S(n)$ needs to be proved for all $n\epsilon {\mathbb{N}}$); next, they acknowledge this as a category of tasks where induction might apply. Students talking about cases in this way (“I think to this case induction might apply, because…”) is a sign that compression on cases has taken place, because their attention has shifted to the properties individual cases might have to decide whether they fall into the category. On the other hand, the goal is compression on the procedure: each proof by induction consists of a set of similar steps and universally applicable techniques.

A proficient problem-solver solves problems by combining, adapting, and elaborating techniques, using concepts and statements. For this purpose, the employed mathematical discourse needs to be formulated in compressed language, to render insurmountable series of steps that form the obstacles that compose the problem surmountable. In this study, we see compression as a central process to foster in students, supporting their ability to solve problems. We make the following reinterpretation:

• A heuristic is a form of help formulated in a compressed language.

A compressed presentation of a technique, like “draw a help line”, we call a heuristic technique. We conclude that help should be organised in a way that students first focus on general properties of the heuristic technique, but next, if necessary, they can decompress the technique into the details applied to the problem. Similarly, concepts and statements should first be offered in a compressed way, and only be decompressed if needed for the student. For example, decompression refers to a technique being expanded into several steps, or a statement or concepts shown to apply to a specific case. This motivates design principle 1 below.

By providing support on the heuristic level first, students have the opportunity to maintain ownership of the details of the technique and how they apply to the problem. This is illustrated in Fig. 1, where the heuristic technique “draw a help line” is applied to the case of a circle and to the case of a right triangle. These two more decompressed forms of help are then further decompressed by applying them to the case of the problem.

Help-Seeking in Digital Environments

Schoenfeld (1985) addresses the importance of self-regulation in mathematical problem-solving (he calls it “control”): the ability of “selecting and pursuing the right approaches, recovering from inappropriate choices, and in general monitoring and overseeing the entire problem-solving process” (p. 98). Another aspect of meta-cognition is the ability to solicit help when needed (Aleven et al., 2003, 2006). This has a completely different dynamic depending on whether the help is offered by a teacher or by impersonal digital means, since in the latter case the social aspect is neutralised—e.g. feelings of shame or the relation to the teacher do not play a role. Moreover, a teacher influences the start of a help-seeking process by inviting and supporting students to formulate a relevant question. A number of studies have suggested that offering heuristic help independent of the teacher in a digital environment can still improve efficient problem-solving (Pol et al., 2008, 2009; Roll et al., 2014), but on the condition that students develop effective suitable help-seeking skills.

Students tend to work less efficient on problems in absence of a teacher. Aleven and colleagues (2006) list help-seeking mistakes that students make when soliciting without the teacher’s support: e.g. spending too little time with a hint, requesting a hint when it is not needed, and trying an unfamiliar (not mastered) technique (when one should ask for help). Two categories stand out: asking a hint too fast (a form of help-abuse)—for all sorts of reasons—and help-avoidance, i.e. avoiding asking a hint. Consequently, they develop a model of desired help-seeking with an eye towards implementing it from their digital, help-seeking tutor (Aleven et al., 2006; Roll et al., 2011).

We would like to point out how balancing the fine line between getting too much or too little help is related to maintaining a sense of ownership of a solution. Not asking for help could be motivated by a desire to maintain maximal ownership, whereas asking for too much help decreases ownership more than necessary. Help-seeking is closely intertwined with engagement in a problem, by which we mean continued solving attempts. As long as a problem itself is interesting and challenging, engagement into the solving process is ensured. As soon as a student is stuck, the engagement is at stake. Providing just-in-time help that allows students to maintain ownership is essential to maintain engagement in the problem and not to reside in consulting the answer model. This motivates the use of a help-seeking flowchart in our design, in addition to the heuristic trees.

Research Questions

Finding help using a digital tool, heuristic tree, completely changes the help-seeking experience, compared to finding help from a teacher. Students need to adapt and develop their problem-solving behaviour accordingly to use the tool for their benefit. In this study, we would like to address the following main research question: how do the digital experiences of finding on-line help in heuristic trees—experiences structured with the help of the help-seeking flowchart—influence students’ problem-solving behaviour? This leads to the following three sub-questions. How do heuristic trees and the help-seeking flowchart contribute to:

• (RQ1) students’ efficient, teacher-independent problem-solving?

• (RQ2) students’ help-seeking and engagement in problem-solving?

• (RQ3) the way students structure the problem-solving process, in particular with respect to the role of compression and decompression?

Based on experience from the preliminary study by Lemmink (2019), we formulated the following hypotheses in response to these questions.

• A. The support by heuristic trees allows students to work on problems without support of a teacher.

• B. Heuristic trees are less flexible in the help they provide than a teacher. This impacts on the help-seeking process, in particular for below-average students.

• C. Heuristic trees allow students to engage with a problem for a longer period. Supporting a good start with the problem and providing help in small steps—leaving following steps to be worked on—allow students to maintain ownership of the solution.

• D. Heuristic trees allow for students to focus on general heuristics and compressed concepts, techniques, and statements, and on how to decompress and apply them in concrete situations.

• E. Students using heuristic trees improve their problem-solving and help-seeking behaviour. They have a more phased approach (in the sense of Pólya) and consider several heuristic techniques before embarking on one.

Method

This design-based study (Bakker, 2018) is a small-scale test of heuristic, tree-supported problem-solving in the context of a number theory course for in-service mathematics teachers.

Design: Heuristic Trees

A heuristic tree is an organisational structure for heuristic support for problem-solving (Bos, 2017; Lemmink, 2019). A heuristic tree consists of an interactive page that provides a collection of hints structured as a tree supporting a specific problem to be solved. Figure 2 shows a snapshot of a heuristic tree. We will focus on its form first, and postpone an analysis of its mathematical content. Notice that the figure actually consists of three “trees” in the proper mathematical sense, related to Pólya’s phases: orientation, making and executing plans, and completion. The branches are drawn from left to right, and at each node, there is a hint, which is only revealed when clicked on. To guide the students to the appropriate hints, the leftmost node in a main branch of the tree carries, when still closed, a short description of its content, usually in the form of a question.

Fig. 2

A heuristic tree in which a student has “opened” three hints (grey boxes)

Each heuristic tree needs to be tailor-made to the problem it supports. The first author introduced the following design principles for heuristic trees (Bos, 2017).

• 1. Compression-decompression ordering: The order along a branch should be from general to more concrete hints, thereby decompressing and specifying the initial information.

• 2. Logical ordering: The structure of the tree (both along and across branches) should represent a logical order of reasoning within a solution model. It should highlight the main structure of the argument, and separate main and side issues.

• 3. Problem-solving stages ordering: The various branches should also be ordered following the various stages of problem-solving. In general: orientation, making and executing plans, and completion.

• 4. Independence: The help offered in different branches should be independent stepping stones, in the sense that, for the help offered in one branch, no information in any of the other branches should be needed.

• 5. Rationing: Each click should not give more help than needed.

• 6. Revelation: The questions that are shown in the unopened heuristic tree should not give away the hint, but instead give an indication of what help can be obtained along that branch. Neither should one hint reveal the content of a next.

Principle 1 is the central one, and it is an implementation of insights on heuristic support and compression (described in the previous section). The challenge for the designer is to identify the general concepts, heuristic techniques, and statements (theorems or propositions) that are needed to solve the problem. These should be presented on the cards at the start of the trunks. The visible side of the card presents a question, a possible heuristic in the sense of Pólya, to guide the help-seeking. Principle 3 is inspired by Pólya’s phased structure for problem-solving. As mentioned, design principle 2 represents the goal to foster insight in the structure of the problem. Principles 4, 5, and 6 help students to avoid making help-seeking mistakes. In particular, these principles ensure students are not given more help than needed, and thus promote maximal ownership. Students have control of their sense of ownership by deciding to click further down the branches and receive more detailed help on the problem or not. Moreover, by consistently applying all principles, students will navigate heuristic trees more easily to find suitable help.

As an example, Fig. 3 shows a fully opened heuristic tree (as used in this study), together with the questions that appear in the unopened tree. The tree is designed for the task of proving a certain equation for the greatest common divisor (gcd). You can see the design principles in action. In the first branch of the orientation phase, one recognises decompression (principle 1). In the first card, a concept is introduced; in the second, the details of the statement of the definition are added; the third can be understood as a decompression over cases, since it is a transition from a statement for many cases to one concrete case. The first branch in the making and executing plans phase is an example of decompression of steps.

Fig. 3

A fully opened heuristic tree (The questions that are visible in the unopened tree are superimposed. The problem to which the heuristic tree applies is: prove that $\mathrm{gcd}(a, b) = \mathrm{gcd}(b,a-b)$ for all positive integers $a$ and $b$.)

In the first card, a general heuristic is presented (think backwards) and a technique for proving equality of two gcds. The next two cards show details of steps of the technique applied to the problem. Another example can be found in the second branch of “making and executing plans”, which contains a situation description for a user who has not already opened the other nodes (principle 4).As a second example, in Fig. 1, the first card shows a heuristic; the second two cards show a decompression into two cases where it applies. The last cards show the details of the technique applied to the problem.

Designing heuristic trees requires a proper understanding of the design criteria, good analysis of the involved problem, and pedagogical insight on how to support students. In a follow-up study, we address how teachers address these challenges as they design heuristic trees for their students (Bos et al., submitted).

Design of the Study: Help-Seeking Flowchart

The students used the help-seeking flowchart (Fig. 4), as designed by the first author and his student (Lemmink, 2019), to support the help-seeking process. The design is inspired on the help-seeking model by Aleven and colleagues (2006). It facilitates the phased approach taken from Pólya (1945) and refined by Schoenfeld (1985), referring to an orientation phase, a phase where those ideas are transformed into concrete plans that can be executed, and a completion phase, after the problem is solved. The chart suggests how and when to move between the phases of orientation and making and executing plans. It also included the movement from heuristic (general) support to more concrete hints.

Fig. 4

A flowchart for help-seeking (Lemmink, 2019)

In our research, the “general hints” refer to heuristics in the sense of Pólya, and the concrete hints to the hints and heuristic techniques in the heuristic tree. These labels “general” and “specific” are not entirely justified: “specific” should be thought of as “problem-specific”. The hints within the heuristic tree still go from general techniques and concepts towards more concrete decompressions of those applied specifically to the problem. For the general hints, students could consult a chart with a list of Pólya’s general heuristics that was in use during their studies, not just for this number theory course.

In the pilot study by Lemmink (2019), students tracked their route through a flowchart for each problem and marked the labels of the hints they used. Classroom observations, data from the collected charts, and hint click data from the digital learning environment together supported the conclusion that help-abuse and help-avoidance had nearly completely vanished.

Design: Classroom Implementation

The number theory course was planned in the spring of 2020 and the intervention involved the first 5 weeks of the course (all just before the outbreak of the corona crisis). The course’s digital learning environment contained various resources and activities, supplemented by a weekly 4 h interactive lecture. The core of the course was formed by weekly homework tasks consisting of nine to twelve problems each. For each of those fifty-five homework problems, the authors designed a heuristic tree according to the principles discussed above.

During each lecture, 30–60 min were available to work on the homework tasks in collaboration with peers in small groups. For this part of the meeting, a printed flowchart and a pawn were handed out to the students. Students were invited to move the pawn along the flowchart according to their state during the problem-solving process. They accessed the problems and the heuristic trees on their laptops and also received a list with general problem-solving heuristics in the style of Pólya for reference. In the first lecture, the students watched a short video clip in which the researchers explained how to use heuristic trees and the flowchart. In addition, the latter explained that the heuristic trees were an innovation in the course and part of a research project. While students worked on the exercises, the teacher was available for support—but he was asked to promote the use of the heuristic trees, if feasible.

Instruments

In the previous section, we listed five hypotheses A to E that provided tentative answers to our research questions. Our aim was to gather data related to these hypotheses and then analyse them from the perspectives of all five hypotheses. To do this, we used four instruments: a survey, in-class camera observations of two small groups of students working on the problems, an interview with those two groups at the end of the course, and an informal teacher’s log with classroom observations.

The survey consisted of four types of questions:

• To test hypotheses A and D, there were four questions about the frequency of usage of the three tools that were provided during the lectures: the flowchart, the heuristic trees, and a list with Pólya’s general heuristics.

• Relating to hypotheses A, C, and E, there were three agree/neutral/not agree choice questions about the perceived benefits of the heuristic trees.

• Eleven Likert scale questions about their experiences with these tools provide information about hypotheses A to D; see Table 2 below.

• In addition, there were eleven open questions about the reasons for the actual use or non-use, as well as about the perceived opportunities and constraints of these tools—the answers were coded afterwards and connected to our hypotheses.

The interview was semi-structured. Based on the analysis of the survey and the video, we prepared a list of questions where the responses would serve as triangulation of earlier findings.

Participants and Data

The participants in this study were students taking a course in number theory from the master’s programme for mathematics education at the Utrecht University of Applied Sciences. The students—approximately fifty—were all in-service teachers with a bachelor degree in mathematics education and varying numbers of years of experience as teachers in secondary education. For the survey and interviews, it was convenient that this population was accustomed to reflecting on their learning processes. The teacher of the course is the second author. The survey was filled in by twenty-three students.

A sub-group of two and a sub-group of three students volunteered for in-class camera observations, which were carried out three times: at the start, at the middle, and at the end of the course. At the end, these two groups were interviewed. The students gave their written consent for the observations and interview. During the course, one of the members of the two-person subgroup left the course and was replaced by a student from another group. The length of the observations and interviews is given in Table 1. Each video contained two or three help-seeking episodes, adding up to a total of thirteen.

Table 1 Observation time in minutes and seconds and number of observed help-seeking episodes

Data Analysis

The classroom observation videos were analysed in a process of analytical abstraction, which starts with tentative coding, then repacking and aggregating the data, and ends in a syntheses of all data into one explanatory framework (Miles & Huberman, 1994). The codes referred to aspects of the hypotheses A to E exhibited in the previous section. This way the codes allowed us to aggregate similar passages and quotations and to refer to them in support of the relevant hypothesis. Both researchers independently watched the videos and coded passages that were relevant to the hypotheses. These analyses were compared and discussed, which lead to an improved set of codes and another round of analysis of the selected passages.

From the survey, the scores on the Likert scale items were averaged and tabulated. For the open answers, we continued to use and expand the coding developed for classroom observation. Both researchers independently coded the answers to open questions in the survey. Then, they compared them and discussed their analysis and created a final coding. Since the codes were grounded in our hypotheses, we could use the codes to match students’ answers with our hypotheses.

In similar fashion, we analysed the interviews and teacher log for relevant passages connected to the hypotheses. On the bases of these coded fragment and answers for each hypothesis, we made a table displaying assembled support, consisting of teacher log info, coded video observation with frequencies, relevant result on the Likert scale items, coded answers to the open questions, and relevant interview passages. Each table presents data in support of the hypothesis with which the table is concerned. Data that oppose the hypothesis are presented and discussed under each of these tables.

Results

Table 2 shows the survey responses to the survey’s Likert scale items.

Table 2 Survey responses on the statements, using a 5-point Likert scale (–2: fully disagree, 2: fully agree) (The third column lists the hypothesis the statement provides information about. The results are only given for students ($N=15$) who indicated that they used the heuristic tree three times or more.)

Tables 3, 4, 5, 6, to 7 present support for the five hypotheses. After these tables with results, a short discussion follows that brings nuance to the supportive results. A selection of three episodes taken from the videos can found in Table 8. This serves two goals: to give an impression of students’ use of heuristic trees and help-seeking flowchart and as material to refer to in support of the hypotheses in the tables below. The names are anonymised.

Table 3 Results supporting hypothesis A (The support by heuristic trees allows students to work on problems without support of a teacher.)

Fig. 5

The frequency of use of heuristic trees at the start at near the end of the course as reported by students

Table 4 Results supporting hypothesis B (Heuristic trees are less flexible in the help they provide than a teacher. This impacts on the help-seeking process, in particular for below-average students.)

Table 5 Results supporting hypothesis C (Heuristic trees allow students to engage with a problem for a longer period. Supporting a good start with the problem and providing help in small steps—leaving following steps to be worked on—allow students to maintain ownership of the solution.)

Table 6 Results supporting hypothesis D (Heuristic trees allow for students to focus on general heuristics and compressed concepts, techniques, and statements, and on how to decompress and apply them in concrete situations.)

Table 7 Results supporting hypothesis E (Students improve their problem-solving and help-seeking behaviour. They have a more phased approach and consider several heuristic techniques before embarking on one.)

Table 8 Three characteristic help-seeking episodes

In contrast to the supportive results presented in Table 3, some students indicate that they (occasionally) have difficulty finding suitable help in the heuristic trees. There are some explanations for this.

• Some of these students indicated that they did not use the heuristic trees at all.

• Some students did not meet the required level for the course and struggled a lot with all problems. If one finds a problem hard, then the offered support can be challenging too.

• Success in a help-seeking episode depends on the ability to self-diagnose: first the decision whether help is needed and next the discussion why you are stuck and what help is needed.

We can say some more about self-diagnosis. In the survey, the statement a (“If I got stuck in a problem, I managed to find the cause”) got only moderate support with a relatively wide variation ($M=0.3, SD=1.1$). We observed many discussions in the videos on whether to turn to the heuristic tree or not. Sometimes students misjudge that they need general heuristic help when actually they need specific help on an algebraic step (see, for example, episode 1 in Table 8). With limited experience, it is simply hard to decide if an approach is still fruitful and why progress is stagnating. Nevertheless, we observe how heuristic trees can provide confirmation of a chosen approach (as in episodes 1 and 2). As a consequence, students can diagnose their approach as correct and sometimes they decide to continue the work without obtaining a new hint (as in episode 2).

The inflexibility of help in heuristic trees could play a role in various ways. Heuristic trees:

• 1. cannot accommodate to all solution strategies to a problem;

• 2. cannot adapt to the levels of all student: levels of presentation or language or the difficulty or ‘size’ of the steps;

• 3. do not adapt to the student’s notational choices;

• 4. cannot support the activation of all requisite pre-knowledge.

Table 4 presents moderate support for issues 1, 2, and 3, but not for issue 4. Issue 1 seems not a major problem. We hypothesise a few reasons:

• We know that some problems allowed more than one approach, but maybe students were all drawn to the one the heuristic tree supports.

• Students who need support search it before they have chosen an approach, and therefore adapt to the one in the heuristic tree.

• Students who use different approaches in problems did not use the heuristic trees for those problems.

In contrast to a point in Table 5, both groups A and B said that the heuristic trees also sometimes speed up the process, because relevant information is presented in the heuristic tree and does not need to be searched for in other sources. Of course, there might have been a positive effect of the camera observation on students’ engagement.

In the theoretical framework section, we hypothesised that experiencing the compression–decompression dynamic would support compression of knowledge. Nevertheless, it seems difficult to directly observe this—in the students’ verbal utterances—or measure whether knowledge has compressed in a quantitative way. We thought that this form of transfer of compressed knowledge might express itself in how far students would progress along a branch to find suitable help. However, only few students reported to experience this and most disagreed: statement h “As I built up experience with a subject, I did not need to proceed as much along a branch to find usable help on that subject” ($M=-0.2$, $SD=1.1)$.

A possible explanation for this is the following. Unfortunately, the tasks in the number theory course did not repeatedly require the same techniques so much. So, there was little opportunity for students to reuse their compressed techniques, concepts, and statements. Moreover, proper transfer could also mean that no help from the heuristic tree is needed in similar tasks. In the interview, group A mentioned that, whether you need to proceed along a branch is mostly determined by the difficulty of the subject: with difficult subjects, you proceed down the branches more often than with simpler subjects.

Even though the results in Table 7 suggest that students have acquired a more structured approach, the support tools did not improve students’ behaviour in the completion phase. This is evidenced both in interviews and in video observations. In the interview, the students from group B said that they were not convinced time would be well spent in this phase: they would rather move on to the next problem. However, when prompted, these students are able to come up with the added value, e.g. deciding what techniques in a solution are important to add to one’s “toolbox”. In some video episodes, students were confused about what the completion phase is for. For example, in two episodes, students thought you first come up with a method for the proof, and only write it down in the completion phase. Furthermore, when asked in the interview, students say the reason they do not do anything in the completion phase is that they are maybe too lazy or too much under time pressure to spent the necessary time and effort.

Discussion and Conclusion

We begin this section drawing conclusions with respect to our sub-questions.

• RQ1. How do heuristic trees and the help-seeking flowchart contribute to students’ efficient teacher-independent problem-solving?

  Table 3 provides support for hypothesis A, that students work much more independently, in the sense of “in the absence of a teacher”. Students find help in the heuristic tree—not with the teacher—when they need it. The questions on the back of the cards, the logical-ordering principle, and the phased-ordering principle help students navigate to the right help in the heuristic tree. The flowchart guides making the decision of when to find what kind of help. Nevertheless, students sometimes struggle to self-diagnose why they are stuck and what help is needed. Table 4 provides support for hypothesis B that the inflexibility of heuristic trees is occasionally a problem for below-average students. Since other students have less need of the teacher, the teacher experienced more time to support the below-average group.

• RQ2. How do heuristic trees and the help-seeking flowchart contribute to students’ engagement in problem-solving?

  Table 5 provides support for hypothesis C that the support offered by heuristic trees stimulates students to engage with the problem. When stuck, students no longer turn to the answering model, but use the heuristic tree to understand better the problem and find help in small steps. In particular, opening cards confirming steps already taken motivates students to continue to pursue an approach. The rationing and revelation principle ensures that students do not obtain more help than they ask for or need. This way they maintain maximal ownership of their solution.

• RQ3. How do heuristic trees and the help-seeking flowchart contribute to the way students structure the problem-solving process, in particular with respect to the role of compression and decompression?

  In Table 6, we find self-reported support for hypothesis D that heuristic trees invite students to focus on general heuristics and compressed concepts, techniques, and statements, and on how to decompress and apply them in concrete situations, as intended by the compression–decompression ordering principle. Table 7 provides support for hypothesis E that students improve their problem-solving and help-seeking behaviour. We conclude that providing help in a very structured way (logical, phased, and compression–decompression principles) invites students to develop a more structured approach themselves. The heuristic trees and flowchart help students internalise structural elements of the problem-solving process. In particular, heuristic trees become a framework for students to think about the structure of a problem approach, as witnessed by a student in the interview: “You can put a problem in a heuristic tree”, by which she meant that you can think about a problem approach as if you would make a heuristic tree for it yourself. However, we have not been able to find more objective evidence that compression has taken place within individual students.

Together, these conclusions draw a picture that answers our main question of how heuristic trees and the help-seeking flowchart influence students problem-solving behaviour. This study has found evidence supporting and nuancing the opportunities and constraints described by hypotheses A to E.

Is it worth the effort, spending hours designing heuristic trees for the problems in a course? First of all, this research shows that students can find help in the heuristic trees working at home or in the classroom, with less need of the teacher. This way the invested time is won back—which, in turn, could be invested in supporting students who cannot find enough help in the heuristic trees. Moreover, the trees can be reused each time a course is repeated, as happens with the trees for the number theory course in this study. Moreover, heuristic trees can be shared on-line with colleagues or anyone interested. Additionally, we have experienced how designing heuristic trees invites the designer/teacher to put more thought to the role of the problem in the course: what are students supposed to learn from this problem, and how do I plan to prepare them and myself for this?

Limitations

Our results do depend on some specific qualities of the population of students from a master’s degree in mathematics education. In the data collection, we benefited from their ability to reflect on learning processes and their willingness to work on the way they shaped their problem-solving processes. Nevertheless, there are no reasons to believe these results would not mostly transfer to other populations. On the contrary, the observations in this study are much in line with observations in previous studies with secondary school students (Bos, 2017; Lemmink, 2019).

Whether a similar study would produce similar results depends, to some extent, on the quality of the heuristic trees involved. Properly implementing the design heuristics turned out to be challenging for us, as it is for other researchers and teachers (Bos et al., submitted).The way the use of heuristic trees is experienced also depends on other aspects of the course, e.g. the preparation for teacher-independent problem-solving and the way concepts, techniques, and theorems are introduced. This should be taken into account in any study using heuristic trees. Other vulnerabilities include the dependency on IT and practical constraints: using a computer, a flowchart, a notebook, and exercise sheets all on one table could be cumbersome. Apart from these issues, we believe that combining video observation with the survey and interviews generated a reliable data collection.

Relation to Other Research

As mentioned before, Lester (2013) listed some problems in research on problem-solving instruction that we also address in this paper. In particular, he was concerned with the role of the teacher. Even though we do not focus on the role of the teacher, our study does focus on a tool that takes over the role of the teacher to some extent, thereby giving the teacher more time to fulfil the mentioned role for students who need most support. Secondly, Lester claimed that there is too much focus on individuals working on problems. We investigated small groups working on problems. We described the interaction between students and the tools. As we have discussed, in the dynamic of problem-solving in interaction with the heuristic trees, students take different roles. A third Lester concern is that problem-solving research is too little concerned with what really happens in classroom. This study is concerned both with classroom behaviour and also with what happens at home. Finally, Lester commented on the largely a-theoretical nature of problem-solving research. We hope to have convinced the reader of the importance of the theory of compression of mathematical knowledge for understanding problem-solving and how to support it.

Reiss and colleagues (Reiss et al., 2008) worked on an idea that is related to heuristic trees: heuristic worked examples. A heuristic worked example is one that makes explicit the heuristics of the problem-solving process. Including heuristic worked examples in the learning trajectory resulted in a positive effect on the reasoning and proof capacities in geometry of low- and average-achieving students. Heuristic trees are allowed to be used as a sort of heuristic worked example by opening all the cards. In the videos, we observed students displaying this behaviour of opening all the cards after solving the problem. As a student of group A said, “Now, I would also like to see how they approach the problem in the heuristic tree”. Notably, students opened all the cards only after they worked on the problem. We believe these attempts to solve the problem oneself are essential in learning. Nevertheless, heuristic trees encompass the heuristic worked example approach by allowing all cards to be opened. In this way, heuristic trees cross a boundary between inquiry-based mathematics teaching and the worked examples approach.

Heuristic trees are designed to support learning of abstract concepts, techniques, and statements. In particular, they are designed to support in cases where this abstraction is due to compression. Other tools and approaches have been designed to support learning to deal with abstraction, e.g. the teaching for abstraction model (White & Mitchelmore, 2010) and APOStheory (Arnon et al., 2014). If we compare the use of heuristic trees with these approaches (that are already very different among themselves), the main difference is that these set out to develop learning trajectories to support the various phases of developing abstract concepts or techniques (e.g. familiarity, similarity, etc.), whereas heuristic trees have as a goal to support tasks within an existing learning trajectory. We do not suggest to use heuristic trees as an alternative method, but instead suggest they could be combined. Heuristic trees could provide support in tasks that are part of activities, class tasks, and exercises (ACE) teaching cycle, and could be inspired by the genetic decomposition of the involved concepts.

Future Developments

In observed help-seeking episodes, browsing through the general Pólya heuristics did not help students. This observation is in-line with Schoenfeld’s (1985) results. We propose, therefore, to discard the general heuristics card, as the relevant heuristics can be integrated in the heuristic tree—thereby reducing the number of support tools from three to two. One could also try to incorporate Pólya’s heuristics into the visible questions on the card. The flowchart can be simplified accordingly. No longer would we suggest in the flowchart first to consult general heuristics and then specific techniques in the heuristic tree. To match the observed workflow better, we incorporated the suggestion to “click until you meet a hint that helps you further”. To improve the instruction for the completion phase, we made a few more changes: see the central two items in the flowchart in Fig. 6. Hopefully, these changes support students in understanding why the completion phase can be fruitful.

Occasionally, we have used hyperlinks in heuristic trees, but we think this possibility could be explored further. Reference to other digital learning resources, such as on-line lecture notes, instruction videos, or Wikipedia, could be used. This way a better integration of all resources within a course could be achieved. Further work is required to establish the impact of the use of heuristic trees on problem-solving achievement through a comparative study. Another interesting path of research, on which we have some preliminary results, is what happens when teachers in pre- or in-service training are set the task to design heuristic trees. This seems to lead to interesting didactical discussions and insight.

Fig. 6

Improved flowchart for help-seeking