1 Introduction

Compared to traditional mathematics paper textbooks, digital mathematics textbooks increasingly offer additional features, such as interactive diagrams, feedback, and formative assessment, and provide teachers and students with possibilities of personalization and individualization (Choppin et al., 2014; Rezat, 2020; Usiskin, 2018; Yerushalmy, 2005). Pepin et al. (2016) referred to these textbooks as “enhanced” e-textbooks. The additional features of enhanced e-textbooks are supposed to support the teaching and learning of mathematics (Yerushalmy, 2015).

The increasing implementation of enhanced e-textbooks is likely to change the learning experience of students. Research on the use of paper textbooks has shown that for the student, working with a textbook means to a large extent working on tasks (Pepin & Haggarty, 2001). If students need assistance, they can ask peers or the teacher at school or family members at home. Some textbooks offer a solution manual to which students can refer in order to check their results (Rezat, 2009). Maybe, in class, teachers take a look at the individual student’s progress and provide personal spontaneous feedback, relying on their own diagnoses (Johansson, 2007). After a phase of individual work in class or at home, the teacher may check the students’ results and provide individual feedback in the form of notes on the student’s work.

Working with enhanced e-textbooks implies that students may have access to additional supportive resources which involve them more actively in their interaction with mathematics (Naftaliev & Yerushalmy, 2013), and that they receive additional support in their learning processes, and automated feedback on their work. The implementation of such features in e-textbooks is aimed at supporting learning and improving instruction. At the same time, they change instruction. On the one hand, they take responsibility from the teacher in that the teacher is not the only one providing explanations and feedback. On the other hand, these additional features potentially change the role of students as readers and thus change students’ encounters with mathematics by providing multiple, linked, dynamic, and interactive representations of mathematical concepts (Kynigos, 2015). Students need to include these features productively in their learning processes, and teachers need to adjust their planning and teaching due to the presence of these additional features. Therefore, the additional features of e-textbooks are transformative and change the teaching and learning of mathematics. However, very little is still known about how these additional features of enhanced e-texbooks actually influence students’ learning processes when solving tasks from digital textbooks (Radović et al., 2020).

In this paper, the focus is on the role of automated feedback in the learning process. In the relevant literature, feedback is most commonly defined as information “provided by an agent (e.g., teacher, peer, book, parent, self, experience) regarding aspects of one’s performance or understanding” (Hattie & Timperley, 2007, p. 81). Its purpose is to cause change in behavior based on highlighting discrepancies between the actual and the intended performance (Hattie & Timperley, 2007; Molloy & Boud, 2014; Mory, 2004; Shute, 2008). Therefore, it is widely acknowledged as an important influential factor in learning and achievement (Hattie & Timperley, 2007).

Many studies have investigated the effectiveness of feedback, by focusing on variables of the feedback message, for example feedback content, timing of feedback, feedback complexity and amount of information provided (Mory, 2004; Narciss, 2006; Shute, 2008). The results of these studies are ambiguous (Shute, 2008; Van der Kleij et al., 2015). In order to determine to what extent various methods of providing item-based feedback in computer-based learning environments affect students’ learning outcomes, Van der Kleij et al. (2015) carried out a meta-study. They could substantiate a moderately large effect of elaborated feedback providing learners with additional information related to the solution of the task, and that elaborated feedback is more effective than feedback that evaluates the correctness of the answer or shows the correct answer. Despite these measurable effects of feedback on learning outcomes, Shute (2008) summarized that “the specific mechanisms relating feedback to learning are still mostly murky, with very few (if any) general conclusions” (p. 156).

In their synthesis of recent research on how technology can support effective feedback, Dawson et al. (2018) pointed out a possible reason, as follows:

In most cases, there is no clear indication of how the feedback inputs (e.g., comments on the assessment performance) are designed to impact on subsequent assessment or how the impact is to be measured. This calls into question the overall validity and comparability of many studies into technology and feedback; without knowing if a technology was used within a high-quality feedback design or not, it is difficult to conclude if the benefits of an approach are actually related to the technology. In addition, the composition or nature of the comments is sometimes less than clear in the feedback designs. Arguably, the impact of the feedback process is heavily dependent on the nature of the information being provided such as a focus on providing actionable comments and the clarification and use of clear performance standards (p. 36).

Additionally, Esterhazy and Damşa (2019) argued that students are actively engaged in making sense of feedback comments, and that therefore the meaning of the feedback is not unilaterally determined by the agent. Therefore, Molloy and Boud (2014) argued for “less preoccupation in what educators ‘do’ in giving feedback, such as how much information to give and at what time, and instead anticipate a shift toward a better understanding of how students seek, interpret and use data related to their learning and how programs are designed to foster this” (p. 414).

For these reasons it is important to gain a deeper insight into how students make sense of the feedback-messages they receive from enhanced e-textbooks. Therefore, the overall goal of the present study is to contribute to a better understanding of the mechanisms of how feedback actually works in the learning process and on students’ conceptual development. Since the large body of studies on the effectiveness of feedback is framed in the quantitative paradigm, which does not allow for fine grained analysis of students’ understanding of the feedback messages and their influences on students’ conceptual development, appropriate theoretical frameworks and methodological approaches to achieve the goal of the present study are missing. Therefore, the first step towards the goal is to develop an appropriate framework and methodology. Based on this framework, I carry out a qualitative in-depth analysis of two elementary students’ learning processes when solving a task from an enhanced e-textbook with automated feedback. The research question was as follows: how does automated feedback from an enhanced mathematics e-textbook influence primary students’ conceptual development?

2 Theoretical framework

Many models have been proposed to explain how feedback functions in the learning process based on results of quantitative studies on the effects of feedback (Narciss, 2006). However, rarely have studies investigated in detail how a particular feedback design actually affects students’ conceptual development. In this paper, I propose a framework that allows investigators to gain a detailed account of how feedback functions in student’s task related conceptual development, based on Rabardel’s (2002) theory of the instrument.

In the present study, feedback is conceptualized as an artifact within an activity-theoretical perspective. Artifacts, such as digital mathematics textbooks in general, or feedback in particular, influence the activity systems of which they are part (Vygotsky, 1997). The didactical tetrahedron (Rezat & Sträßer, 2012) is the activity system underlying this study. It describes a co-produced system of learning (Molloy & Boud, 2014) constituted by the teacher, the student, the mathematical content or concept materialized as a mathematical task, and the feedback from the enhanced e-textbook. The focus in this study is on the triangle between student—feedback—and the mathematical concept/task. In the next section, the characteristics of each of these constituents of the activity system are described in more detail for the scope of this study. In this section, I focus on how the interaction between student, feedback, and mathematics is conceptualized in order to develop a deeper understanding of how the feedback influences students’ learning processes. The interaction between student and feedback is conceptualized based on Rabardel’s (2002) theory of the instrument. The distinction between artifact and instrument, together with his conceptualization of the instrument, are the core of Rabardel’s theory. Artifacts are regarded as “anything which human beings create by the transformation of nature and of themselves” (Wartofsky, 1979, p. xiv) to be incorporated in human activity in order to support the solution of problems. Rabardel (2002) defines the instrument as “a composite entity made up of an artifact component (an artifact, a fraction of an artifact or a set of artifacts) and a scheme component (one or more utilization schemes, often linked to more general action schemes” (p. 86).

The instrument is developed by the user of the artifact in a process called instrumental genesis. This process of instrumental genesis consists of two opposite processes called instrumentalization and instrumentation. While instrumentalization relates to the development of artifact components of the instrument such as the selection, attribution of functions and adaptations, instrumentation processes relate to the development of utilization schemes. Rabardel’s conceptualization of instrumental genesis as two opposite processes provides concepts that allow researchers to unveil the influence of the artifact on the activity system and the user’s cognition. Instrumentalization enables the description of how features of the artifact are noticed by users and how users try to make use of them, while instrumentation focuses on how features of the artifact influence the user’s activity. Relating these insights to the goal of the activity allows researchers to draw conclusions in terms of how feedback as a feature of digital textbooks affords or constrains the activity, that is, the learning of mathematics. The goal of students’ activity in the present study was predefined by the situation. Students worked individually on mathematical tasks from an enhanced e-textbook. Thus, their goal was finding the correct answer to the task.

In order to develop a deeper understanding of students’ instrumental genesis connected to the feedback provided by the enhanced mathematics e-textbook in order to achieve this goal, a deeper understanding of students’ utilization schemes and how these are influenced by the artefact within the instrumentation process was necessary. With his theory of conceptual fields, Vergnaud (1996) provided a conceptualization of scheme within a “comprehensive framework for studying complex cognitive competences and activities, and their development through experience and learning” (p. 219). His concept of scheme has proved to be useful for describing cognitive aspects related to instrumentation processes within the instrumental approach (Drijvers, 2002; Rezat, 2013) since it allows elicitation of the dialectic relationship between activity and thought (Trouche, 2004). According to Vergnaud (1998) a scheme is “the invariant organization of behavior for a certain class of situations” (p. 167). The class of situations in the present study was defined by the mathematical task and the related mathematical concept. Vergnaud (1998) suggested that schemes are particularly characterized by “1. goals and anticipations; 2. rules of action, information seeking, and control; 3. operational invariants; 4. possibilities of inference” (p. 173). Among those characteristics he underlined the role of the two operational invariants “theorems-in-action” and “concepts-in-action”, because they represent the implicit knowledge in schemes. The difference between both operational invariants is that of relevance and truth. While “concepts-in-action are relevant, or not relevant, or more or less relevant, to identifying and selecting information”, “a theorem-in-action is a proposition held to be true by the individual subject for a certain range of the situation variables” (Vergnaud, 1996, p. 225) and thus can be either true or false.

While the mathematical target concept is defined by the task, students’ operational invariants are their individual concepts and theorems that they apply when solving the task. These are not necessarily expressed explicitly but may be implicit in their actions. This feature is exactly what is described by the attribute ‘in-action’. Generally, the purpose of feedback is “to reduce discrepancies between current understandings and performance and a goal” (Hattie & Timperley, 2007, p. 86). A student’s goal was described earlier as “finding the correct answer to the task”. The purpose of feedback within our framework can be described as reducing the discrepancies between students’ operational invariants and the target concept. Focusing the analysis on students’ concepts- and theorems-in-action and how they are altered as a response to the feedback therefore allows us to describe how students’ conceptual development is instrumented by the feedback. A related analysis of the instrumentalization of the feedback enables a detailed account of the aspects of the feedback that are instrumentalized. Figure 1 provides an overview of the activity system and relevant theoretical notions that were analyzed in the present study.

Fig. 1
figure 1

Theoretical framework

Full size image

Enhanced e-textbooks usually combine different types of automated feedback (Rezat, 2019, 2020). In order to develop a differentiated understanding of how feedback influences students’ learning processes, different types of feedback are distinguished based on a systematization by Shute (2008). Shute (2008) systematized different types of feedback according to feedback complexity.

3 Methodology

3.1 Participants and data collection

In the overarching project, data were collected by videotaping 117 third grade students from 12 different German primary schools in the region around a German city, working with one enhanced e-textbook for primary level available on the German market. The students were selected randomly, depending on their own and their parents’ agreement to participate in the study. All students were working individually on all kinds of different tasks from the e-textbook in an experimental setting, and were asked to think aloud. An interviewer was present during data collection and repeatedly asked students to explain what they did and why they did it, in order to prompt and sustain their thinking aloud. The students that participated in the study were working with the enhanced e-textbook for the first time. Prior to the task that was used in this analysis they were introduced to the features and handling of the e-textbook and already had been working on a few other tasks. The interviewer also sometimes assisted with handling the digital textbook and drew attention to some features. This setting was chosen in order to be able to observe students’ instrumental genesis of the feedback.

3.2 The digital textbook

In this study, the enhanced e-textbook “Denken und Rechnen interaktiv” published by one of the biggest German publishing companies for curriculum materials (Westermann Group) was used. Currently, it is one of the few mathematics e-textbooks for the primary level on the German market that offers interactive answering formats, such as multiple choice or drag and drop, and automated feedback. The textbook is a digitalized version of a traditional and widely used paper textbook offered by the same publisher. The structure and the contents of the printed and the digital versions are analogous. Compared to the traditional textbook the digital version offers different kinds of feedback (categorized according to Shute, 2008), as follows:

  1. a.

    After entering an answer, the correctness of the answer is evaluated by verification feedback. This is combined with the possibility of two retries (try again feedback). If there is only one answer to enter, the textbook shows a green field with a positive feedback message or a red field with a negative one. If there are multiple entries to make, the textbook provides error flagging feedback in that it highlights the correct answers in green and the wrong answers in red. The wrong answers disappear before the next trial, the correct answers remain.

  2. b.

    If the student has not succeeded after the second trial, a symbol with a lightbulb appears on the lower right-hand side of the screen. A click on the symbol provides a hint, cue or prompt for solving the task. According to Shute‘s systematization, this is a type of elaborated feedback (hints/cues/prompts).

  3. c.

    If the student has not succeeded in answering the task correctly in the third trial, the correct result is shown (correct feedback).

  4. d.

    After solving a set of tasks, summative feedback is provided, which provides information about the time needed for working on the task and a summary of the number of tasks that were solved in the first, the second, and the third trial, respectively, or not at all.

  5. e.

    The textbook offers a lexicon of mathematical terms and procedures, which is accessible throughout the whole process of working on a task. Mathematical terms in the wording of the task are directly linked with the lexicon. This is also categorized as a kind of elaborated feedback.

In summary, the enhanced e-textbook used in this study is generic for digital mathematics textbooks on the market. It provides verification feedback combined with try-again feedback, error flagging feedback, correct response feedback, and elaborated feedback. This appears to be a combination of feedback types that is already the maximum that is offered b German digital mathematics textbooks (Rezat, 2020).

3.3 The task

The focus of this paper is on learning processes related to the concept of impossible, likely, and certain events in the context of probability. Altogether 24 children in our study were observed while working on the probability task presented in Fig. 2 and a second analogous task (2b) that asks for the number of trials that is needed to certainly draw a red ball from the box. The task requires to differentiate between impossible, (un-)likely, and certain events. The standards of the German state, where the study took place, expect the competence that students at the end of grade four are able to describe the probability of simple events as certain, likely, impossible, always, (in-)frequently, never. In the two tasks, students are asked to determine the number of trials that are necessary to certainly draw a blue/red ball from an urn represented by a picture of a box with four red and eight blue balls. This task requires counting the maximum number of undesirable outcomes and adding 1.

Fig. 2
figure 2

Task 2a on page 85 (www.denken-und-rechnen-inderaktiv.de) Translation [SR]: [heading] Chance and probability—drawing balls; [task] Lia draws one ball at a time and does not return it. How many times does she have to draw a ball until she certainly draws a blue ball. Give reasons in your notebook. [answers] once, twice, 3-times, 4-times, 5-times, 6-times, 7-times, 8-times, 9-times, 10-times, 11-times, 12-times. [button at lower left]: skip

Full size image

After the second wrong trial, the lightbulb on the lower right-hand side appears. Clicking on this lightbulb opens the hint provided in Fig. 3:

Fig. 3
figure 3

Hint/elaborated feedback provided by the e-textbook related to task no. 2a on page 85 (www.denken-und-rechnen-interaktiv.de) Translation [SR]: [heading] Hint; 1. Look how many red and blue balls are in the box at the beginning. 2. Imagine, Lia draws a red ball first. How many times could she draw another red ball after that? Keep in mind that Lia pulls a ball and does not return it into the box. 3. Consider: What is the smallest number of draws until there are no red balls in the box but only blue ones?

Full size image

The learners in our case studies were not introduced to these kinds of tasks and representations of urns by the teacher. However, before working on this task they worked on other tasks with similar urn representations. Since the aim was, to investigate how students make sense of the feedback and how it influences their learning of mathematics, this setting was chosen in order to reduce the influence of other factors, such as previous instruction by the teacher. However, it is important to note that these were experimental conditions with the common limitations in terms of ecological validity.

3.4 Data analysis

The video recordings were transcribed according to the rules for simple transcripts by Kuckartz (2014) and analyzed in terms of instrumentalization and instrumentation. In order to identify instances of instrumentalization, the analysis determined whether students used different types of feedback and which function they attributed to the feedback in their learning process. The function they attributed to the feedback was inferred from their actions following the feedback message. If it was noticeable that they reacted to the feedback message and were rethinking their answer, this was taken as an indicator that they had instrumentalized the feedback as a means for learning.

The analysis of the instrumentation by the textbook focused on reconstructing students’ concepts-in-action and theorems-in-action as the implicit knowledge in their schemes. Qualitative content analysis according to Mayring (2010, 2015) was applied in order to analyze students’ actions and their verbal explanations of their actions, for the purpose of explicating the operational invariants.

“Explicating procedures” (Mayring, 2015, p. 373) were used to formulate concepts- and theorems-in-action that explained students’ actions. Explication was accomplished by a “narrow context analysis” (Mayring, 2015, p. 373) by relating different passages in the transcript in order to infer possible concepts- or theorems-in-action.

4 Results

In this paper, I present the analysis of two cases, Kim and Ben. These cases were selected, because they showed typical obstacles when working with tasks from digital textbooks with feedback in its currently widespread form.

4.1 The case of Kim

1

K

(reading) (24 s.)

2

I

What are you thinking about?

3

K

(…) Well, Lia picks a ball from the box and does not put it back in and then one has to find out how many times she must have reached into it to draw herself a blue ball. Hm (pondering). Once, I believe, because there are many blue ones in it and only a few red ones

4

I

Uhugh (affirmative)

5

K

(…) Or maybe twice, I am not so sure. (5 s.) I think it’s twice

6

I

Why?

7

K

Because they can be mixed up. So that she draws a red one first and then a blue one (11 s.) (selects TWICEa , taps on OK)

8

TB

feedback: SORRY, WRONG

9

I

What happened?

10

K

It is wrong

11

I

Why, or try again

12

K

(9 s.) Once

13

I

Why?

14

K

Because, if she is lucky, she draws a blue one in her first trial. (…) (selects ONCE, taps on OK)

15

TB

feedback: SORRY, WRONG. The feedback disappears, the task is completely visible on the screen and the symbol of a lightbulb appears on the lower right-hand side of the screen

16

I

(5 s.) And now? (25 s.) Maybe something appeared that was not there before and that might help you

17

K

Ehm (contemplating), clicking on this book?

18

I

Is there maybe something else?

19

K

The lightbulb. (taps on the lightbulb symbol. A tip appears on the screen) (14 s.)

20

I

What happened?

21

K

Mmh, there are tips

22

I

Aha. … Does the tip help you? … (30 s.) … Do you understand the tip?

23

K

Yes, well, ehm, if, ehm / tip 3 is a / well, they can / because there are only three red ones in the box and then she could have drawn four red ones and then a blue one and therefore I think it ‘s five (selects FIVE TIMES, taps on OK)

24

TB

feedback: YES, THAT IS CORRECT

25

K

(taps on CONTINUE)

26

TB

Task 2b appears on the screen

27

K

(17 s.) I believe she draws (..) once (..) a red ball

28

I

Why do you believe that?

29

K

(..) It’s a feeling

30

I

Mmh

31

K

I am thinking (..)

32

I

Yes

33

K

If she (11 s.)

34

I

What are you reflecting upon? You have already said that you believe it’s once

35

K

Ehm, if she has to grab multiple times in the box (12 s.) I believe (.) ten times, because (.) maybe she could draw nine blue ones and then the first red one. It’s probably the other way around now

36

I

What do you mean by the other way around?

37

K

Well, before I had four red ones and one blue one and then I assumed it was five times and now I think it is the other way around, inverted I mean, that I remove the blue ones first and then one more red one. (selects 10 TIMES, taps on OK)

38

TB

feedback: NO, THIS IS NOT ABSOLUTELY CORRECT

39

K

(sighs) (11 s.)

40

I

What did you just say? How many times does she have to draw?

41

K

There, where she drew blue ones?

42

I

No, right now. You just explained how many times she

43

K

Well, about ten times, because it was the same with the red ones. But now, I could (.) also (..) half of nine, five or four

44

I

Half? How did you come up with half?

45

K

Well, when I remove these (points at the blue balls in the figure), well the blue ones are nine and then add one red one and then 10 divided by five (5 s.) (selects 5 TIMES, taps on OK)

46

TB

feedback: NO, NOT YET CORRECT

47

K

(sighs) (6 s.) I think (.) when I answered ten he said it is not absolutely correct and (.) well, I would not go below five, because when I answered five he said not correct and when I answered ten he said not absolutely correct

48

I

Ah, okay

49

K

Eleven, twelve, nine, eight (.) I am not so (8 s.) (moves her finger over the tablet, pulls her hand back)

50

I

So, now you are thinking about which number close to ten you are choosing?

51

K

Yes. Well, I think, it will not go down to six (..) (selects 9 TIMES)

52

I

Why have you selected nine?

53

K

Ehm (..) because it is close to ten. (taps on OK)

54

TB

feedback: SUPER!

55

I

So, have you been right?

56

K

Yes. (taps on EVALUATE)

  1. K Kim, I Interviewer, TB textbook
  2. aText from the enhanced e-textbook is cited in capital letters

After reading the task, a long pause, and the question of the interviewer concerning what she is thinking about (l. 2), Kim repeats the task in her own words (l. 3). She guesses the answer that Lia only has to draw once to get a blue ball and reasons that this is because there are many blue balls and only a few red balls in the box. Then she sways between the two answers that Lia has to draw a ball once or twice (l. 5). The reason she provides to explain her swaying is that the balls can be mixed up in the box and that it is also possible then that Lia draws a red ball first and a blue one afterwards.

She enters the answer TWICE and receives the verification feedback that her answer is wrong (l. 7–8). She takes some time to think about the correct answer and returns to her first idea that Lia only needs to draw once from the box in order to get a blue ball. She reasons that, if Lia is lucky, she can already draw a blue ball in her first trial (l. 14).

After entering the answer and receiving the verification feedback that this answer is wrong, she takes a long pause to think. Triggered by the hint of the interviewer (l. 16, l. 18), she taps on the lightbulb and opens the tip (elaborated feedback). Caused by the question of the interviewer about whether she understands the tip, she gives the correct answer and explains how to find it (l. 23).

After entering the correct answer and following the positive verification feedback, she starts working on the second task, which now asks her to find the number of times Lia has to draw a ball from the box until she certainly gets a red ball. Kim starts guessing that Lia has to draw a ball once from the box in order to get a red ball and justifies this with her intuition (l. 29). She hesitates (l. 31–33) and following questions from the interviewer she explains that she is considering whether Lia has to draw a ball ten times, because it could be the other way around, compared to the previous task, and Lia now has to remove all the blue balls before she gets the first red one (l. 35–37). She enters 10 TIMES and receives the verification feedback that her answer is not absolutely correct. She starts to rethink her answer and thinks aloud about two possible divisions she could carry out with her answer 10 TIMES (l. 43–45). The reasoning behind these is not comprehensible to the interviewer, but Kim does not further explain them and decides to enter 5 TIMES. She receives the verification feedback that this is not correct. Hereafter, she starts to identify limits of the interval, in which she expects the correct answer and reasons based on the wording of the feedback messages she received (l. 47). She seems to expect the correct answer to be in the interval from 5 to 12 (l. 47–49) and decides to enter 9 TIMES (l. 52), because it is close to ten. She gets affirmative verification feedback from the textbook.

4.1.1 Instrumentalization

Following the prompt (l. 9) by the interviewer to think aloud, it becomes apparent that Kim instrumentalizes the verification feedback as an assessment of her answer and rethinks her answer each time she receives negative verification feedback from the e-textbook. Furthermore, she instrumentalizes the tip in the first task after her second incorrect answer, as a means to get further support to solve the task correctly. While the use of the tip is also triggered by the interviewer (l. 16, 18), there is no indication in the transcript that the interviewer influenced the instrumentalization of the tip in terms of attributing a function to it.

In the second task, Kim also instrumentalizes the precise wording of the KR-feedback-message as additional information that could guide her to answer the task correctly (l. 47–53).

4.1.2 Instrumentation

Kim’s process of solving the task is instrumented by the feedback from the textbook. First of all, it is observable that the negative verification feedback initiates that she rethinks her answer each time it occurs. How this affects her concepts- and theorems-in-action was analyzed in more detail, as follows.

In line 3, Kim argues that “there are many blue ones in it and only a few red ones”, but that “they can be mixed up” (l. 7). Therefore, she concludes, that Lia “draws a red one first and then a blue one” (l. 7). The theorem-in-action that guides Kim’s answer in this first attempt to solve the task could be formulated as If there are (much) more blue balls than red balls in the box, then it is (almost) certain that Lia will draw a blue ball from the box in her first or second trial. The underlying concept-in-action is rather the concept of (high) probability than that of certainty. In her second attempt to solve the task, Kim argues that Lia could be lucky and draw a blue ball from the box in her first trial (l. 14). This supports that her concept-in-action seems to be that of (high) probability. After reading the hint (elaborated feedback), Kim’s theorem-in-action changes. In line 23, Kim explains that since there are altogether four red balls in the box, Lia “could have drawn four red one and then a blue one”. The change in her wording from “I believe” (l. 3) and “if she is lucky” (l. 14), which refer to her personal guess and to chance, to “she could have drawn” (l. 23), indicates that she finally considers the worst case scenario in order to make a claim about certainty. The related theorem-in-action could be formulated as If Lia wants to certainly draw a blue ball from the box, then she has to remove all the red balls first and in her next trial she will certainly draw a blue ball. Thus, the concept-in-action of (high) probability seems to have been replaced by the concept of certainty as “number of unfavorable outcomes + 1”. This concept-in-action seems to be persistent throughout Kim’s first attempt to solve the next task. In her explanations in line 37 (“I had four red ones and one blue one and then I assumed it was five”; “I remove the blue ones first and then one more red one”) she relates to her concept-in-action of certainty as “number of unfavorable outcomes + 1”, but seems to miscount the number of unfavorable outcomes, i.e. the number of blue balls in the box (“the blue ones are nine and then add one red one”, l. 45; “about ten times”, l. 43). Due to the negative verification feedback she now omits this concept-in-action and starts to carry out calculations that are not comprehensible (“half of nine”, l. 43; “10 divided by 5”, l. 47). In the following, her mathematical concepts-in-action seem to be replaced by concepts-in-action that are related to the precise wording of the verification feedback-message, in particular to her concept-in-action of the wording “not absolutely correct”. Kim interprets the difference between the two verification feedback-messages she has received as relevant information regarding her answers: “when I answered five he said not correct and when I answered ten he said not absolutely correct” (l. 47) Her concept-in-action of “not absolutely correct” triggers her to look for the correct answer in the neighborhood of her wrong solution that was evaluated with this message (“Eleven, twelve, nine, eight”, l. 49; “I think, it will not go down to six”, l. 51). Thus, the concept-in-action could be explicated as the feedback message ‘not absolutely correct’ indicates that the correct solution lies in a neighborhood of the not absolutely correct solution. While Kim’s explicit reasoning why she has decided on nine times (“because it is close to ten”, l. 53) might be triggered by the interviewer’s prompt in line 50 (“So, now you are thinking about which number close to ten you are choosing? “) her utterances in lines 49 (“Eleven, twelve, nine, eight”) and 51 (“I will not go down to six”) support that she is herself is looking for the correct answer in the neighborhood of ten.

4.2 The case of Ben

1

B

(reading 32 s.) How am I supposed to calculate this? I mean,

2

I

Do you not know, what you have to do?

3

B

No, I (.), how am I supposed to calculate this, I mean, she draws one in her first trial

4

I

Mhm, she draws a ball

5

B

Yes

6

I

Exactly. Read the task one more time

7

B

(reading 18 s.) I think, once, because the blue balls are already on top

8

I

Ok. Just try it, what you think

9

B

enters ONCE, taps on OK

10

TB

feedback: NO, THAT IS NOT ABSOLUTELY CORRECT

11

I

It’s not absolutely correct

12

B

(looks at the screen 20 s.)

13

I

Would it be helpful to get some support now?

14

B

No, mmh, I just don’t know (..) on top there already is a blue ball, and when she reaches into the box for the first time, she already would have to get a blue ball

15

I

Mhm. Perhaps, yes, hm

16

B

(enters TWICE) If she draws from the bottom, then, yes (taps on OK)

17

TB

feedback: NO, THAT IS NOT ABSOLUTELY CORRECT

18

B

Huh? (taps on the symbol of the lightbulb on the lower right-hand side of the screen, the tip shows up on the screen)

19

I

Now you are using the tip

20

B

(reading 5 s.) Oh, I see, she draws the red ones first? Ok. (enters 5 TIMES, taps on OK)

21

TB

feedback: CORRECT!

22

I

Did the tip help you to find the correct answer?

23

B

Yes, because she had drawn the red ones on the bottom first

24

I

And what did you think?

25

B

I first thought, that, because the, because the blue ones are on top and she has to draw a blue one, that she gets a blue one immediately

26

I

Mhm. Yes. Ok

27

B

That’s what I thought

28

I

Yes. So, this is a second task

29

B

(reading 14 s., enters ONCE, taps on OK)

30

TB

feedback: SORRY, WRONG

31

I

Why did you choose once now?

32

B

Because, in the other task, she draws from the bottom first

33

I

Mhm

34

B

(His fingers move above the screen. 4 s., enters 9 TIMES, taps on OK)

35

TB

feedback: MARVELOUS!

36

I

Why have you chosen nine times now?

37

B

Because, now she took the ones on top first

38

I

Ok, the blue ones. Yes. Well done

  1. B Ben, I Interviewer, TB textbook

After Ben reads the task, he conveys that he does not know how to calculate the answer (l. 1–3). After being asked to read the task once more, he expresses that he believes that Lia will draw a blue ball from the box in her first trial, because the blue balls are on top (l. 7). He enters this answer and gets the feedback that it is not absolutely correct. After taking 20 more seconds to think, and being asked by the interviewer if he needs some help, he denies. Then he repeats his belief that Lia only needs one trial to get a blue ball, because the blue balls are on top (l. 12–14). He enters TWICE as an answer and gets the verification feedback that this is not absolutely correct. Ben seems to be confused and immediately opens the tip provided by the e-textbook (l. 18). After reading the tip he finds the correct answer and explains that he did not know that Lia draws the red balls from the bottom first. After he receives positive verification feedback, the second task shows up on the screen. Ben reads it and enters the answer that Lia has to draw once in order to get a red ball. He receives the verification feedback that his answer is wrong (l. 29–30). He argues that he thought that Lia now only has to draw once in order to get a red ball, because in the first task, she picked the balls from the bottom first (l. 32). In his second trial, he enters 9-times and receives positive verification feedback. He explains that Lia now picks the upper blue balls from the box first and then gets a red ball.

4.2.1 Instrumentalization

Ben also instrumentalizes the verification- and elaborated feedback messages to rethink his answers. Besides his instrumentalization of the feedback, Ben further instrumentalizes the picture with the urn and the spatial arrangement of eight blue and four red balls in order to find the correct answer of the task.

4.2.2 Instrumentation

At the beginning of the transcript, Ben asks twice “How am I supposed to calculate this?” (l. 1, 3). This question seems to be an expression of his concept-in-action of a mathematical task: Mathematical tasks require calculation. However, even without knowing the procedure to calculate the correct answer, he engages in solving the task. In his first and second attempt to solve the task, his answer is grounded in inferences from the position of the balls in the box. He argues that “the blue balls are already on top” (l. 7) and therefore Lia “already would have to get a blue ball” (l. 14) “when she reaches into the box for the first time” (l. 14). Therefore, the theorem-in-action, which seems to guide his first and second attempt to answer the task could be formulated as If the blue balls are on top it is (almost) certain that one would draw a blue ball from the box. “Almost” is added, because he seems to omit certainty in his second attempt and considers the possibility of drawing a red ball from the bottom first (l. 16).

After reading the tip provided by the e-textbook his theorem-in-action is adjusted. This is apparent in his utterance “she draws the red ones first” (l. 20) and a similar utterance in line 23 combined with his selection of the answer “5 TIMES” (l. 20). This theorem-in-action could be explicated as If the blue balls are on top and Lia wants to certainly draw a blue ball from the box then she takes the red balls from the bottom of the box first and the number of trials she needs to draw a blue ball with certainty is determined by < number of red balls >  + 1. This theorem-in-action is based on two concepts-in-action: (1) the particular spatial arrangement of the blue and red balls in the box combined with (2) a particular order in which the balls are drawn from the box. In his first attempt to solve the second task it becomes apparent that these two concepts-in-action remain relevant when solving the task. The theorem-in-action that guides his first attempt to answer the second task could be reconstructed from his utterance in line 32 “Because, in the other task, she draws from the bottom first”. It could be expressed as If the blue balls are on top and Lia draws the red balls from the bottom first and she wants to draw a red ball with certainty from the box then she certainly draws a red ball from the box in her first trial. After receiving negative verification feedback, his reasoning is adjusted to “Because, now she took the ones on top first” (l. 37). This indicates that also his theorem-in-action was adjusted to If the blue balls are on top and Lia wants to draw a red ball with certainty from the box then she takes the blue balls from the top of the box first and the number of trials she needs to draw a red ball with certainty is determined by < number of blue balls >  + 1.

The explication of Ben’s theorems-in-action shows that these are biased by two salient concepts-in-action: (1) the spatial arrangement of the blue and red balls in the urn, and (2) the particular order in which the balls are drawn from the urn. These concepts-in-action are persistent throughout the episode as becomes apparent in Ben’s justification of his correct answer to the second task (“now she took the ones on top first”, l. 37). It is likely that they will remain even after he found the correct answer to the second task.

5 Discussion

The qualitative in-depth analysis of the two cases provided a glimpse of how automated feedback from a German enhanced e-textbook is instrumentalized by students and influences students’ conceptual development (in the sense of instrumentation) when individually solving a task from the textbook.

The cases share some similarities. In both cases, verification-feedback initiates that the students rethink their answers. However, since they are not provided with any information that could be helpful in this process they start to guess and only slightly adjust their answers from their first trial based on slightly adjusted concepts- and theorems-in-action. Both, Kim and Ben start with the concept of high probability, which is persistent throughout their first and second trials.

In both cases, the elaborated feedback supports students in adjusting their theorems-in-action and finding the correct answer accordingly. Thus, the elaborated feedback seems to support the further development of the concept of certain events in both cases.

However, the analysis revealed that in both cases students instrumentalize information that is not relevant for the task. In the case of Kim this was the verification-feedback-message NOT ABSOLUTELY CORRECT and in the case of Ben this was the arrangement of the balls in the box, and the order in which they are approached by the person acting in the task.

Kim’s careful interpretation of the feedback message shows that she seems to appreciate and instrumentalize any additional information that she can get in order to find the correct answer. However, there is a mismatch between the actual information content of the feedback message and the one suggested by its wording. The wording suggests that the feedback provides information about the degree of derivation of the student’s answer from the correct answer. The actual information content is simply that the answer is wrong. Instrumentalizing the information suggested by the wording could be misleading for the students as in the case of Kim. Based on her instrumentalization of the feedback-wording Kim comes to the correct answer and the feedback even supports her instrumentalization. This is likely to contribute to the consolidation of her utilization scheme of the feedback.

For Ben, the spatial arrangement of the balls in the symbolic representation of the urn is very salient and evokes concepts-in-action, which are related to this spatial arrangement. Even after finding the correct answer to the second task, these concepts-in-action bias his theorem-in-action and seem to remain related to his concept of certainty.

It might be argued, that this misleading information is not an issue with e-textbooks and feedback in general, but an issue with the particular e-textbook and even with the particular task—an issue that could easily be omitted by reformulating the task and the feedback-message. However, I argue that these cases exemplify how readily students will try to make sense of information that is not relevant for the solution of the task. This is only natural, since conceptual learning is about refining a concept by differentiating between the relevant and the irrelevant aspects of a concept (Prediger, 2008; Tall & Vinner, 1981). Therefore, it is part of the teaching–learning process to develop and specify student’s concepts-in-action that guide their perception and activation of relevant schemes. However, these two cases show how difficult it might be for the developers of e-textbooks to take all possible instrumentalizations of irrelevant information by students into account when developing an e-textbook. No matter how carefully the tasks in e-textbooks and the feedback messages are designed, it is very likely that they still contain irrelevant information, which might become salient in students’ solution processes. Therefore, tasks, feedback messages, and diagrams need to be designed very carefully and as unambiguously as possible.

Assuming that the availability of feedback will increase students’ individual work with e-textbooks and that teachers will delegate the evaluation of students’ answers to the textbook and only check the summarized results of students’ work, this analysis showed that these students in third grade are likely to develop misconceptions if they are left on their own. While the summarized feedback shows in both cases that students succeeded, it can be questioned whether their conceptual development of the concept of certain events was actually successful.

These two cases also show that mere quantitative insights into the effects of feedback are not sufficient, but that feedback can have very different and individual effects. This insight supports Dawson et al.’s (2018) argument that the question of how feedback is designed to match students’ needs is an important issue when evaluating the effectiveness of feedback, and that this point needs to be made explicit in related studies. The analysis has shown how students differ in their individual sense making of feedback messages. This result supports that feedback needs to be regarded as a process rather than only as information provided by an agent. Only the analysis of the whole feedback process is likely to reveal the hurdles that appear in the learning process and how feedback affords or constrains students’ conceptual development.

This case study exemplifies some of the challenges and hurdles that students encounter in the learning process when working with digital textbooks on their own. Therefore, I argue that tasks for e-textbooks including feedback should be developed in design-research cycles, in order for educators to become aware of students’ understanding of the tasks, the provided feedback messages, and how they afford or constrain their learning processes. In its detailed reconstruction of the two students’ conceptual development, this study again supports the findings that mere knowledge of response feedback is not sufficient (Jaehnig & Miller, 2007; Van der Kleij et al., 2011). In the case of multiple try feedback, elaborated feedback in terms of hints, cues or prompts should already be provided after the first wrong answer. As was shown in the analysis, the elaborated feedback seemed to have an effect on the two students’ conceptual development. Furthermore, it is important that students be provided with unambigious feedback messages that do not contain more information than is actually relevant. Finally, students’ age needs to be considered carefully in the development of e-textbooks, paying attention to how students at different ages are actually capable of learning mathematics individually from the e-textbook.

The study has some limitations to consider. It was a case study in an experimental setting. Only two students working with one task from one particular German mathematics e-textbook were studied. The automated feedback from the e-textbook was not designed according to the scientific state of the art. Limitations in the types of feedback implemented in the e-textbook are well known. However, it is an enhanced e-textbook based on a widely used paper textbook published by a well-established publishing company. Therefore, the study mirrors what students are actually exposed to when they learn mathematics with e-textbooks in German primary schools. In this sense, the study is ecologically valid. However, the experimental setting limits the ecological validity of the study; i. e., students might interact differently with the enhanced e-textbook in their natural environment without the presence of an interviewer. In the case of Kim, the interviewer even drew attention to the elaborated feedback. Otherwise, Kim might not even have noticed it. Furthermore, it could be expected that the teachers in German primary schools would introduce students to new content and concepts before they work with the textbooks individually. Rarely would students have to learn new concepts solely from the textbook. Consequently, the findings are not generalizable. However, they indicate obstacles that students actually faced in their interaction with a task from an enhanced e-textbook and related feedback. The findings also suggest that teachers have to be very careful in relying on the potential of the feedback.

From a theoretical perspective, the present study expanded the range of instrumental geneses to be analyzed based on the instrumental approach (Rabardel, 2002) and Vergnaud’s concept of scheme. Adding to the large body of mainly quantitative research on the effectiveness of feedback, the qualitative analysis based on the instrumental approach provided a detailed account of how students made sense of the feedback and how it affected their conceptual development related to a particular mathematical concept and a specific situation. Instrumentalizing the instrumental approach for a study on the effectiveness of feedback has shown how students are actively engaged in making sense of feedback comments and that therefore the meaning of the feedback is not unilaterally determined by the agent (Esterhazy & Damşa, 2019); and thus this study has contributed to the “shift toward a better understanding of how students seek, interpret and use data related to their learning and how programs are designed to foster this” as postulated by Molloy and Boud, (2014, p. 414).