Exploring the Contribution of Gestures to Mathematical Argumentation Processes from a Semiotic Perspective
A multimodal perspective on mathematics thinking processes is addressed through the semiotic bundle lens and considering a wide notion of sign drawing from Vygotsky’s works. Within this frame, the paper focuses on the role of gestures in their interaction with the other signs (speech, in particular) and investigates the support they can provide to mathematical argumentation processes. A case study in primary school in the context of strategic interaction games provides data to show that gestures can support students in developing argumentations that depart from empirical stances and shift to a hypothetical plane in which generality is addressed. In this regard, by combining synchronic and diachronic analysis of the semiotic bundle, specific features of gestures are pointed out and discussed: the semiotic contraction, the condensing character of gestures, and the use of gesture space in a metaphorical sense.
KeywordsArgumentation Gestures Multimodality Semiotic contraction Semiotic bundle
At the turn of the millennium, in 2000, the provocative essay Where Mathematics Comes From by George Lakoff and Rafael Núñez pointed out the crucial role of perceptual and bodily aspects on the formation of abstract concepts, including mathematical concepts (Lakoff and Núñez 2000). The new stance emphasized sensory and motor functions, as well as their importance for successful interaction with the environment. Criticizing the platonic idealism and the Cartesian mind–body dualism, Lakoff and Núñez advocated that all kinds of ideas, including the most sophisticated mathematical ideas, are founded on our bodily experiences and develop through cognitive metaphorical mechanisms.
The book aroused a great interest in mathematics education and prompted many research studies highlighting the role of bodily and kinesthetic experiences in mathematical learning (Arzarello and Robutti 2008; de Freitas and Sinclair 2014; Edwards 2009; Ferrara 2014; Nemirovsky 2003; Radford 2014; Roth 2009; for an overview, see Gerofsky 2015).
More recently, embodied stances seem to receive a certain confirmation by neuroscientific results on “mirror neurons” and “multimodal neurons,” which are neurons firing when subjects performs actions, when they observe somebody else doing the same action, and when they imagine it (Gallese and Lakoff 2005). On the basis of these results, Gallese and Lakoff (2005) provide a new theoretical account on how the brain works, according to which “action and perception are integrated at the level of the sensory-motor system and not via higher association areas” (p. 459). In particular, such an integration would appear to be crucial not only for motor control, but also for planning actions, an activity typical of what is generally understood as “thinking.”
The terms multimodal and multimodality come therefore to indicate a feature of human cognition opposed to “modularity.” On the other hand, in the communication field the term multimodal is used with reference to multiple modalities that we have to communicate and express meanings to our interlocutors: words, sounds, images, and so on (Kress 2004). These communicative affordances have been acquiring increasing attention due the diffusion of new technological affordances, which are constantly developing new possibilities of interaction with them through our body.
In this paper, in line with Radford et al. (2009), multimodality refers to the importance and mutual co-existence of a variety of cognitive, material and perceptive modalities or resources in the mathematics teaching-learning processes, and more in general in the formation of mathematical meanings: “These resources or modalities include both oral and written symbolic communication as well as drawing, gesture, the manipulation of physical and electronic artifacts, and various kinds of bodily motion” (pp. 91–92). Including the embodied aspects in the analysis of mathematical thinking and learning brought to the fore the study of gestures as an important cognitive and communicative manifestation.
On the other hand, the attention to embodied and multimodal aspects needs to come to terms with the consideration of the social, historical, and cultural aspects in the genesis of mathematical concepts (Schiralli and Sinclair 2003; Radford et al. 2005). Mathematics is indeed “inseparable from the symbolic instruments” and the act of knowing is a “culturally shaped” phenomenon (Sfard and McClain 2002, p. 156) in which use of tools and signs play an important role.
What specific contribution can gestures, when they are considered as signs in semiotic bundles, provide to students’ argumentation processes?
In the next sections, the theoretical framework for the research is presented: It is constituted by theoretical elements and results from gesture studies in psychology and by a semiotic perspective for multimodality grounded on Vygotsky’s account of signs and on the semiotic bundle notion. Afterwards, selected data from a case study in primary school in the context of strategic interaction games will be analyzed and discussed according to an analytical generalization stance.
30.2 Gestures as Multimodal Resources
Gestures accompanying discourses are a widespread phenomenon (not only Italian!), as the pioneering work by Kendon has documented since the 80s (Kendon 1980). Since then, psychological and psycholinguistic studies have been stressing that speech and gestures are closely linked and that gesturing is relevant in communication and thinking processes (McNeill 1992, 2005; Goldin-Meadow 2003).
McNeill (1992) found speech and gestures to be closely linked in many respects: They are temporally synchronous in phonological (the central phase of the gesture coinciding with the peak of the phonological phrase), semantic (at the meaning level), and pragmatic (their function in the discourse) aspects. Also, in child development, gesture and speech proceed together. At the cognitive level, some scholars have identified their function being important in lightening the working memory, offering the possibility for cognitive resources to do their best to reorganize (Goldin-Meadow et al. 2001).
These cognitive interpretations provide elements that can explain, for example, why we gesticulate in telephone conversations (de Ruiter 1995), why when we are prevented gesturing our discourse becomes less fluid, or why even blind from birth use gestures while speaking. These phenomena cannot be explained only in terms of interpersonal communicative dimension, and so gestures are claimed to have a constitutive role also in thinking processes.
Vygotsky (1934/1986) had already stressed the constitutive role of language in thinking by saying that “thought is not merely expressed in words; it comes into existence with them” (p. 218); since then psychological studies on gestures have pushed in the direction of extending this constitutive role of language to the speech-gesture unity. Quoting McNeill (1992), we can say that “gestures do not just reflect thought but have an impact on thought. Gestures, together with language, help constitute thought [emphasis original]” (p. 242). It is within this Vygotskian hypothesis that I frame the role of gestures in mathematical activities.
Iconic: if they bear a relation or resemblance to the semantic content of discourse (for example, inclining two hands to indicate a roof);
Metaphoric: similar to iconic gestures, but with the pictorial content presenting an abstract idea that has no physical form (a classical example is the hand in the act of holding an object, when referring to idea of “a certain topic” in the discourse);
Deictic: if they indicate objects, events, or locations in the concrete world.
Beats: if they contribute to stress some parts of the discourse.
Deictic gestures are usually performed with the extended forefinger (sometimes with hand-held objects, such as a pen) and are also called pointings. Apparently simple, pointing is indeed a complex act. Besides concrete pointings (such as indicating a book on the table), research has also identified abstract pointings, when the hand or fingers are extended in the space as to indicate something, but the space it actually empty. In McNeill’s (1992) interpretation, “the speaker appears to be pointing at empty space, but in fact the space is not empty; it is full of conceptual significance. Such abstract deixis implies a metaphoric use of space in which concepts are given spatial forms” (p. 173). Such a classification is not based on the physical features of gesture, but by considering the relationships with contextual information: this entails that the interpretative process needs to take into account the broader context in which a gesture is performed. A second remark concerns the fact that the same gesture may belong to more than one category; therefore, the categories have to be considered dimensions along which a gesture can be featured, more than in a classificatory view.
By discovering the catchments created by a given speaker, we can see what this speaker is combining into larger discourse units – what meanings are being regarded as similar or related and grouped together, and what meanings are being put into different catchments or are being isolated, and thus are seen by the speaker as having distinct or less related meanings. (McNeill et al. 2001, p. 10)
Catchments may therefore be of great importance because they can give information on the underlying meanings in speech and dynamics. In a classroom setting, studying the catchments could provide clues on the evolution of meanings in students. In addition, catchments can contribute to the organization of an argument at a logical level, as discussed in Arzarello and Sabena (2014).
30.3 A Semiotic Approach to Multimodality
The significance of semiosis for mathematics education lies in the use of signs; this use is ubiquitous in every branch of mathematics. It could not be otherwise: The objects of mathematics are ideal, general in nature, and to represent them—to others and to oneself—and to work with them, it is necessary to employ sign vehicles, which are not the mathematical objects themselves but stand for them in some way. (Presmeg et al. 2016, pp. 1–2)
The invention and use of signs as auxiliary means of solving a given psychological problem (to remember, compare something, report, choose, and so on) is analogous to the invention of tools in one psychological respect. The signs act as instruments of psychological activity in a manner analogous to the role of a tool in labor. (Vygotsky 1931/1978, p. 52)
From this perspective, signs are considered in their functional role as psychological tools that allow the subjects to reflect and plan actions and act as cultural mediators (Radford and Sabena 2015). This is a very general idea of a sign that does not assign prescriptions on what can be a sign and which specific features it should have: A gesture can also be considered a sign, as Vygotsky himself highlighted in his famous example of the pointing gesture to illustrate the internalization process starting from the meaning assigned by the mother to the child’s hand movement (Vygotsky 1931/1978).
In order to include gestures as well as other more classical registers, Arzarello developed the semiotic bundle construct (Arzarello 2006; Arzarello et al. 2009) as a system made of different signs (or semiotic resources) and their mutual relationships that are produced by students and possibly the teacher during mathematics activities: words (spoken or written), written diagrams, gestures, tools, and so on. Similarly to Radford’s idea of “semiotic system” (Radford 2002), the semiotic bundle includes both the classical registers, with precise and codifiable rules of productions and transformation (Duval 2006), and the embodied ones, allowing us to provide a semiotic account of the multimodal processes occurring while learning and teaching mathematics. An example can be constituted by students’ words, gestures, and drawn figures while solving a geometrical problem.
A systemic character, revealed by a synchronic analysis of the relationships between the different kinds of signs at a certain moment (like a sort of “semiotic picture”)
A dynamic nature revealed by a diachronic analysis focusing on the evolutions of signs and of their transformations over time (a sort of “semiotic movie”)
Synchronic and diachronic analysis—which are distinguished only for the sake of analysis—are performed by considering closely the video recordings from classroom activities students are engaged in, together with their multimodal transcripts (i.e., transcripts that include not only words but also record gestures and other kinds of signs). It is interesting to remark how the possibilities offered by new technologies for the study of the interaction between students and the teacher in the classroom gave a boost to considering multimodal aspects in mathematics learning. Although in the early 90s attention to the “classroom discourse” in the teaching learning of mathematics had already emerged, it was the use of video recordings that opened up the possibility to observing phenomena that had hitherto been unnoticed due to its undetectability. Gestures and other embodied resources have thus begun to be considered among the resources through which communication and conceptualization are realized. In other words, the strong push towards new theories that has come from methodological aspects has required new analytical tools, such as the semiotic bundle.
In the following, the semiotic bundle lens is applied to empirical data on mathematical argumentation processes carried out by primary students. The analysis will use a fine-grain focus on video-recording data with the aim of identifying and theorizing key gestural phenomena that play a role in such processes.
30.4 A Case Study: The Race to 20
The case study is based on a strategic interaction game called “Race to 20,” which was used by Brousseau to illustrate the theory of didactical situations (Brousseau 1997). The case study is part of a design-based research project on the use of strategy games for fostering problem-solving and argumentation processes from primary to secondary school.
The game is played by two players who struggle to reach the number 20 by adding alternatively small numbers. Specifically, the first player chooses a number between 1 and 2, then the second player must add 1 or 2 to the previous number and say the result, then the first player adds 1 or 2, and so on. The player saying 20 wins the game. In game theory, it is a perfect information game with complete information, based on sequential decision-making. As the reader may know or can check, the winning strategy consists of starting as the first player and in following the number sequence 2-5-8-11-14-17-20. In this kind of game, the player needs to determine, for any move by the opponent player, the right move in order to win the game. These processes may be related to the logical scheme of coordinating a universal qualifier with an existential one, as it is the case in many mathematics theorems.
This semiotic template was introduced by the teacher in order to allow the students to keep record of both the winning numbers and the added ones. This record was meant to support them in determining regularities that are at the base of the winning strategy.
The discussion is engineered right after the students have finished a classroom team tournament, in which representatives of each team have played the matches at the blackboard (the last match is shown in Fig. 30.1). The teacher initiates the discussion by making explicit the goal of providing a strategy to win the game and justifying it.
We will focus in detail on the specific contribution of some children—Giulio, Eliana, and Elisa—but first let us give some contextual information, in order to make the analysis understandable.
Marta You have to get first to 14 and then to 17. Because if you do 14 plus 1 and you get 15 and then you do plus 2 and you get to 17, then . . . you do plus 1 to arrive to 18 and the other does 2 and gets to 20. Whereas if from 14 you do plus 2, you arrive at 16 and the other one does plus 1 to arrive at 17, the other 19 if he does plus 2, you do plus 1, and you get 20. So however, from 14 to 17 you arrive anyway to 20.
In other cases, they rely on the empirical observation of what did actually happen during the tournament. Through backward induction, the number 11 begins to be identified as a winning number and related to 14 and 17:
11 maybe is an important number, because maybe my team adds 2 and it is 13, the other team adds 1 and arrives at 14, I add 1, 15, they add 2 and it is 17.
At this point, after about 20 min of classroom discussion, Giulio proposes a general rule to identify all winning numbers:
I think that for the winning numbers you always remove 3: from 20 you remove 3 and you arrive at 17; from 17 you remove 3 and you arrive at 14, I think that another winning number could be 11, could be . . . 8, could be . . . 5, could be . . . 2.
Giulio’s strategy identifies all winning numbers starting from the winning result, 20, and moving backwards through repeated subtractions in a process of backward induction, as described in game theory.
Multimodal transcript of the first part of Giulio’s strategy
I thik that for the winning numbers you always remove 3
you remove 3
and you arrive at 17
Secondo me dato che i numeri vincenti si toglie sempre 3
e arrivi a 17
Open hand moving from right to left
Three extended fingers
Three extended moving from right to left
Abstract pointings downwards
When saying “you always remove 3,” Giulio moves his hand from right to left (from the child’s perspective): This movement can be interpreted as indicating subtraction, with reference to the number line, which is often used in the Italian curriculum. With this interpretation, the gesture can be classified as a metaphoric gesture indicating subtraction. I remark that in the original Italian version, Giulio uses the term togli, which is used both in everyday contexts to say “remove, take away” and in mathematical context in primary school to indicate “subtract.”
right-to left movement → subtraction
three fingers → number 3
When completing the sentence and uttering the winning numbers, Giulio is performing abstract pointing gestures downwards that are co-timed with the uttered numbers (in Table 30.1, the case of 17 is reported in the last column).
If we consider the whole sequence, we see that the subtraction of 3 is repeated in order to obtain 14; this repetition is also expressed by the repetition of the same gesture configuration of the three fingers extended (pictures are not reported for reasons of space). The same metaphorical gesture is hence repeated in a catchment expressing that number 3 is always subtracted, in order to get all the winning numbers.
Afterwards (“I think that another . . .”), we can notice a type of semiotic contraction occurring within the semiotic bundle: Speech reduces, mentioning only the winning numbers, and shifts to a hypothetical level (“could be . . .”); also, gestures appear to reduce in their movements, ending up with quick abstract pointing left and downwards co-timed with the utterance of the winning numbers (8, 5, and 2).
At this point, the teacher asks Giulio to explain his idea. Here it is the verbal transcription of Giulio’s argument:
Explain well this idea.
Because . . . that is I don’t know, if I arrive at 2 . . . I don’t know, I begin, I make 1, no I make 2, he arrives and makes 1, I put 2 and I arrived at 5, which I think is another winning number . . . yes, arrived at 5 . . . it is a winning number, I think. Then . . . he adds 2, say, I add 1 and I arrived at 8, which is another winning number. She adds 1, I add 2 and I arrive at . . . 11, which is a winning number. He adds 2, I add 1, and I arrive at 14, which is another winning number, he adds 1 I add 2, we arrive at 17 which is a winning number, he adds 1 or 2, I add 1 or 2, and I win.
The subtraction turns now into an onward movement that starts from the very first move (number 2) of an imagined match between himself and another player. This movement is produced by means of a repetition of the same linguistic structure: “He adds . . . I add . . . and I arrive at . . . , which is a winning number.” This repetition is not just a mere repetition of words, but is performed with a rhythmical structure in sound, which is preserved along the entire sentence and contributes to convey the general character of the found rule (similar to what was discussed in algebraic context in Radford et al. 2007).
Multimodal transcript of the first part of Giulio’s argument
I begin . . . , I make 2
he arrives and puts 1
I put 2
and I arrived at 5
Io inizio . . . , faccio 2
lui arriva lì e mette 1
Io metto 2
e sono arrivato a 5
Two fingers pointed out
Hand held open upwards as containing something
Two fingers pointed out
Hand held open upwards as containing something
While uttering the moves of an imagined match, Giulio is performing again two kinds of metaphoric gestures. When indicating his own moves, the gesture indicates the uttered numbers by pointing out the correspondent number of fingers, i.e., two fingers when saying 2 (Pictures a and c in Table 30.2). When referring to the other players’ moves or to the obtained result, the hand is held open upwards as containing something (Pictures b and d in Table 30.2): In this case the metaphoric reference is made to underline a certain kind of generality of the uttered numbers.
Multimodal genericity conveyed within the semiotic bundle
The gesture is again metaphoric and the semiotic bundle of words and gestures underlines that the number 2, chosen to indicate the opponent’s move, is to be considered as one possibility among others (all the possible moves): It is one generic move. This is indeed a very delicate logical relationship to manage: the articulation between a universal qualifier (for any move from my opponent player) with an existential one (the move that I choose after him). We see that the gesture-speech combination allows the students to successfully manage it.
From this moment on, when uttering the imagined moves by the two players, abstract pointing gestures are enacted with left-right spatial alternation, which indicates visually the alternation between the two players in the game (Table 30.3, Pictures b and c). This spatial alternation can be interpreted as helping the student to keep control of the argument at the local level, that is to say, to control the choice of the moves and counter-moves in the imagined sequence. The gesture spatial alternation is repeated several times (catchment) for any couples of moves and counter-moves, realizing the same rhythm of the accompanying words. As indicated by McNeill (2005), gesture catchments provide the discourse with cohesion. In this case, it contributes in structuring the entire argument at a global level.
The written template through which the game has been played has possibly helped Giulio in developing his strategy, working as an interiorized tool. As a matter of fact, when saying the general rule, he is looking at the blackboard (Table 30.1, Picture a), where the record of last match is still written (see Fig. 30.1). We remark that this match had not been played according to Giulio’s strategy and that after this initial moment, we do not find any explicit reference to performed matches, for instance, with pointing gestures to the blackboard or to his notebook: This could be another index of the general level reached by Giulio in his argument.
The discussion focuses then on Giulio’s strategy. Some students immediately agree with Giulio and produce their own argumentations, such as Eliana:
I agree with Giulio because practically any time you have to reach a lucky number you must add 3, because first you add 1 and then you add 2 or first you add 2 and then add 1.
Eliana’s argument on the strategy of adding 3
you must add 3
because first_you add 1
and then you add 2
or first you add 2
and then add 1
devi aggiungere 3
perché prima aggiungi 1
e poi aggiungi 2
o prima aggiungi 2
e poi aggiungi 1
Right hand turning from left to right
Left hand moving from left to right
Right hand moving from left to right
Left hand moving from left to right
Right hand moving from left to right
Eliana explains also where this number 3 comes from, i.e., the combination of the possible subsequent moves of the two players. When uttering the numbers added to compose 3, she moves first the left hand from left to right (Picture b in Table 30.4), then the right hand with the same movement (Picture c in Table 30.4); she mentions the two possible combinations (1 + 2 or 2 + 1) and repeats the gestural combination in a catchment (Pictures c and d in Table 30.4).
As in the case of Giulio, we can notice the spatial alternation left-right as a metaphoric reference for the alternation of the two players, and again we can notice a catchment. But differently from the case of Giulio, now the fact that there are two alternating players is expressed by Eliana only through her gesture, because in her speech she uses always the pronoun “you,” possibly in impersonal sense.
left-to-right movement → addition
spatial alternation → players alternation
Right after Eliana, Elisa intervenes:
So overall . . . if you play . . . you add 3 every time, and so if you can arrive at the numbers that there are 3 [pointing out index and thumb; Picture a in Table 30.5], that is if . . . Overall it is 3 [shifting the pointed fingers from left to right; Picture b in Table 30.5], because if you add 1 [placing the pointed fingers at her left; Picture c in Table 30.5] and the other adds 2 [placing the pointed fingers at her right, Picture d in Table 30.5], if you add 2 and the other adds 1 [repeating the sequence with pointed fingers at her left and then right as in Pictures c and d in Table 30.5], overall it is 3 [shifting again the pointed fingers as in Picture b in Table 30.5] and so you must be able to pick the numbers that are . . .
At a distance . . .
. . .of 3.
Elisa’s condensing gesture
and so if you can arrive at the numbers so that there are 3
that is if . . . overall it is 3
because if you add 1
and the other adds 2
e quindi se tu riesci ad arrivare ai numeri in cui…in cui ci sono 3
cioè se… in tutto fa 3
perché se tu aggiungi 1
e l’altro aggiunge 2
Gesture performed with two pointed fingers as holding a little stick
Holding-stick gesture shifted from left to right
Holding-stick gesture placed at left
Holding-stick gesture placed at right
When saying “overall it is 3,” the holding-stick gesture is shifted from left to right (Picture b in Table 30.5): The left-to-right movement indicates that the fixed distance (of 3) allows one to pass from a winning number to the following one in the sequence. The girl is keeping her eyes towards the blackboard, where the last match played is still written, according to the template chosen by the teacher (Fig. 30.1). In this template, the subsequent moves are written one after the other, in a horizontal way. The horizontal movement of Elisa’s gesture may be interpreted against this background, suggesting that the semiotic choice of the teacher has been useful for developing the students’ argument. At the same time, the gesture may be referring metaphorically to an addition on the interiorized number line. We find another example of a gesture condensing different meanings through its metaphoric references and its dynamism. Through the condensing character of the gestures and in synergy with speech, the different meanings come to be connected to build an important part of the argument.
Through a gesture repetition or catchment, the condensing gesture is then combined with spatial alternation referring again to the two players (Pictures c and d in Table 30.5): The catchment provides support in shifting from a relationship between moves to a relationship between numbers that can justify the moves and indicates that such a distance does not vary over the entire winning sequence in the game.
This paper adopted a multimodal perspective on mathematics teaching and learning processes and chose a semiotic tool in order to address it: the semiotic bundle, with its wide notion of sign drawing from Vygotsky’s works, and its systemic and dynamic features. It focused in particular on the role of gestures in interaction with the other semiotic resources used in the classroom—speech first of all, but also written signs—and addressed primary students’ mathematical argumentation processes in the context of strategic interaction games.
Through a case study and qualitative-interpretative analysis, it has been shown that gestures may contribute to carrying out argumentations that depart from empirical stances and shift to a hypothetical plane in which generality is addressed. From this case study, we can get also some insights on how gestures can do this. In particular, specific features have been identified: semiotic contraction, the condensing character of gestures, and the use of gesture space in a metaphorical sense combined with catchments. They will be briefly discussed, referring to the data analysis reported above.
When Giulio identifies and/or expresses (we do not have sufficient data to determine) the general rule of “always removing 3,” we see his sentences becoming shorter and shorter and at the end just expressing the winning numbers, accompanied by abstract pointing gestures (Table 30.1, Fig. d). This is a type of semiotic contraction that has also been found in other contexts and at different ages, such as pattern generalization and function graphs (Sabena et al. 2005; Sabena 2007). From an epistemological point of view, semiotic contraction characterizes modern mathematical symbolism, and from a cognitive point of view it is a precious mechanism. Radford (2008) relates contraction to focusing attention to the elements that are relevant for a certain situation and to a deeper level of consciousness: “Contraction is the mechanism for reducing attention to those aspects that appear to be relevant. This is why, in general, contraction and objectification entail forgetting. We need to forget to be able to focus” (p. 94).
Semiotic contraction can be found also in what Vygotsky (1934/1986) calls “inner speech,” which is described discussing language as a paradigmatic signs system. Inner speech is described at a structural level by syntactic reduction and phasic reduction and at semantic level by agglutination. Syntactic reduction is a specific form of abbreviation that curtails the subjects of sentences and leaves pure predication. Syntactic articulation results are therefore minimized to the pure juxtaposition of predicates. Phasic reduction consists of minimizing the phonetic aspects of speech, namely curtailing the words themselves (for example, writing “u” instead of “you”). Agglutination consists in combining words, gluing different meanings (concepts) into one expression (for example, “highway,” formed by “high” and “way”). Nowadays, instant messaging communication systems on our smartphones extensively exploits semiotic contractions (combined with additional iconic features, such as the emoticons), typically in informal communication by people sharing most of the contextual information.
With syntax being reduced, Vygotsky (1934/1986) claims that semantics undertakes a contrary movement, with meaning coming to the fore: “With syntax and sound reduced to a minimum, meaning is more than ever in the forefront. Inner speech works with semantics, not phonetics” (p. 244).
Giulio, with the right-to-left gesture with three fingers pointed out, indicates both the number 3 and subtraction (Table 30.1, Pictures b and c).
Elisa, with two fingers pointed as if she were holding a little stick, shifted from left to right, which may interpreted as indicating both a fixed distance and the fact that this distance allows one to pass from one winning number to the next one (Table 30.5, Pictures b and c). The co-timed speech specifies that this distance is 3, obtained as the sum of 1 and 2.
In both examples, gestures condensed or blended two meanings by combining a dynamic component with the hand shape: This dynamic feature has been observed also by Calbris (2011) in what she calls “polysign gestures.” In previous studies in the mathematical domain (Sabena 2007, 2008, 2010), the condensing or blending character of gestures has been identified in functions and graphs contexts and associated with iconic features of gestures. In the reported study, this feature is shown in an arithmetic domain and associated with the metaphoric feature of gestures, as McNeill (1992) classified them. Condensing two different meanings, each of these gestures establishes two different kinds of metaphorical references, one of which calls into play the number line, a didactical tool suggested in the Italian curriculum. By exploiting space in order to reason about numbers, the number line itself has a metaphorical nature. A double or even multiple blending process seems therefore to be activated by some metaphorical gestures typical of the mathematical domain. This theme requires deepening the reflection of what “metaphorizing” means at a cognitive and at a semiotic level, and further research is needed (for preliminary results using cognitive metaphors and blended spaces, see Sabena et al. 2016).
Metaphoricity appears to be related also to the use of gesture space with spatial alternation, as we have seen in Giulio and in Eliana. Giulio moves his hand left and right when mentioning the moves and countermoves in his imagined match (Table 30.3, Pictures b and c), while Eliana alternates her left and right hand for the same purpose (Table 30.4, Pictures b and c and Pictures d and e). Through gesture spatial alternation, empty spaces acquire meanings, which in the data appear related to a sort of local level, either in the game (imagined subsequent moves, in the case of Giulio), or in the mathematical argument (numbers to add to compose 3, in the case of Eliana).
As seen in the analysis, such a spatial alternation is repeated many times, realizing what in gesture studies is called a catchment and is interpreted as providing the discourse with cohesion (McNeill 2005). In this case study, gesture catchment is interpreted as supporting the students in structuring the entire argument at a global level. Previous results about how opposite spatial locations are exploited gesturally to indicate mutually excluding cases seem to confirm this interpretation (Arzarello and Sabena 2014).
For the sake of analysis, the different gestural features contributing to providing general meaning and structure to the argumentation process have been discussed here one after the other. However, as can be observed going back to the data analysis, many of these features intertwine; furthermore, the analysis of gestures needs to take into account the entire semiotic bundle. For example, only a systemic analysis of words and gestures can show how, even if it is describing a certain hypothetical match between himself and another player, Giulio’s argument contains essential aspects conveying generality: the rhythmical repetition of the same linguistic structure, accompanied with a corresponding catchment (Tables 30.2 and 30.3); the use of generic words accompanied by a generic gesture (Table 30.3, Picture a); and the use of abstract pointings while uttering the possible moves (Table 30.3, Pictures b and c). In the case of Eliana’s argument, it is striking to observe how gestures and words complete each other in a synchronic way.
If we analyze the children’s contribution in a diachronic way, further observations may be drawn that provide elements to describe the classroom discussion evolution in a multimodal perspective. To give an example, it is interesting to see how Giulio’s spatial alternation with his right hand evolves in Eliana’s alternation of the two hands one after the other (see Table 30.4); in this latter case, the subject in her speech is not changing (it is always “you”), showing a tension towards the arithmetical relationships rather than on the strategic game interaction. This paves the way to the following Elisa’s intervention about the “numbers so that there are 3.”
A final consideration is reserved for the didactical implications of such a fine-grained analysis. In this paper, little attention has been devoted to the didactical variables of the situation. Of course, the teacher’s choices are never neutral with respect to the use of any semiotic resource in the classroom, gestures included. An example in the data is the semiotic template through which the Race to 20 has been presented to the students and through which they play the game (Fig. 30.1). We have seen that this choice—which resonates with the didactic tool of the number line—has provided an essential tool for the students not only to play the game, but also for developing argumentations about how to win it. In particular, Elisa’s multimodal argument about the “distance of 3” between winning numbers shows a relation to the semiotic written template through which the game was played. The results of this analysis appear therefore to offer elements for validating the choice of the teacher. It is beyond the scope of the analysis, however, to discuss why for Elisa (and for some students) it did work, whereas for others, further reflection was needed. Classroom discussion appears indeed to be a suitable means for allowing the development of multimodal argumentations such as the one described, in which the students may exploit gestures as semiotic resources. This requires, of course, that gestures are considered legitimate in the classroom (as it happens in the analyzed case: The teacher supports Elisa in her multimodal argument by considering the contribution of her gesture and offering her the missing word). Even more, the teacher can contribute to classroom mathematical activity through her gestures in order to make the mathematical discourse evolve towards culturally established mathematics forms (see the “semiotic game” in Arzarello et al. 2009). Ongoing research indicates that the teacher can have an important role in the evolution of signs within the semiotic bundles and in building “multimodal semiotic chains” that make mathematical meaning progress through argumentation processes (Maffia and Sabena 2015, 2016). Further extensive research is still needed in order to unveil and exploit fully the potentiality of gestures as didactical means in the classroom.
- Arzarello, F. (2006). Semiosis as a multimodal process. In L. Radford & B. D’Amore (Guest Eds.), Revista Latinoamericana de Investigación en Matemática Educativa, Special issue on semiotics, culture, and mathematical thinking (pp. 267–299).Google Scholar
- Arzarello, F., Paola, D. Robutti, O., & Sabena, C. (2009). Gestures as semiotic resources in the mathematics classroom. Educational Studies in Mathematics, 70(2), 97–109.Google Scholar
- Arzarello, F., & Robutti, O. (2008). Framing the embodied mind approach within a multimodal paradigm. In L. English, M. Bartolini Bussi, G. Jones, R. Lesh, & D. Tirosh (Eds.), Handbook of international research in mathematics education (2nd ed., pp. 720–749). Mahwah, NJ: Erlbaum.Google Scholar
- Arzarello, F., & Sabena, C. (2014). Analytic-structural functions of gestures in mathematical argumentation processes. In L. D. Edwards, F. Ferrara, & D. Moore-Russo (Eds.), Emerging perspectives on gesture and embodiment (pp. 75–103). Charlotte, NC: Information Age Publishing, Inc.Google Scholar
- Brousseau, G. (1997). Theory of didactical situations in mathematics. Dordrecht: Kluwer.Google Scholar
- Calbris, G. (2011). Elements of meaning in gesture. Amsterdam: John Benjamins Publishing Company.Google Scholar
- de Freitas, E., & Sinclair, N. (2014). Mathematics and the body. Cambridge, UK: Cambridge University Press.Google Scholar
- de Ruiter, J. P. (1995). Why do people gesture at the telephone? In M. Biemans & M. Woutersen (Eds.), Proceedings of the center for language studies opening academic year ’95–96 (pp. 49–56). Nijmegen: Center for Language Studies.Google Scholar
- Duval, R. (2006). A cognitive analysis of problems of comprehension in a learning of mathematics. Educational Studies in Mathematics, 61, 103–131.Google Scholar
- Edwards, L. D. (2009). Gestures and conceptual integration in mathematical talk. Educational Studies in Mathematics, 70(2), 127–141.Google Scholar
- Ferrara, F. (2014). How multimodality works in mathematical activity: Young children graphing motion. International Journal of Science and Mathematics Education, 12(4), 917–939.Google Scholar
- Gallese, V., & Lakoff, G. (2005). The brain’s concepts: The role of the sensory-motor system in conceptual knowledge. Cognitive Neuropsychology, 22, 455–479.Google Scholar
- Gerofsky, S. (2015). Approaches to embodied learning in mathematics. In L. D. English & D. Kirshner (Eds.), Handbook of international research in mathematics education (3rd ed.). New York: Routledge.Google Scholar
- Goldin-Meadow, S. (2003). Hearing gesture. How our hands help us think. Cambridge, Massachusetts, and London, England: The Belknap Press of Harvard University Press.Google Scholar
- Goldin-Meadow, S., Nusbaum, H., Kelly, S. D., & Wagner, S. (2001). Explaining math: Gesturing lightens the load. Psychological Science, 12, 516–522.Google Scholar
- Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), The relation between verbal and nonverbal communication (pp. 207–227). The Hague: Mouton.Google Scholar
- Kress, G. (2004). Reading images: Multimodality, representation and new media. Information Design Journal, 12(2), 110–119.Google Scholar
- Lakoff, G., & Nùñez, R. (2000). Where mathematics comes from: How the embodied mind brings mathematics into being. New York: Basic Books.Google Scholar
- Maffia, A., & Sabena, C. (2015). Networking of theories as resource for classroom activities analysis: The emergence of multimodal semiotic chains. In C. Sabena & B. Di Paola (Eds.), Teaching and learning mathematics: Resources and obstacles, Proceedings of the CIEAEM 67, Quaderni di ricerca didattica, 25-2 (pp. 405–417). Aosta, July 20–24, 2015.Google Scholar
- Maffia, A., & Sabena C. (2016). Teacher gestures as pivot signs in semiotic chains. In C. Csikos, A. Rausch, & J. Szitànyi (Eds.), Proceedings of 40th Conference of the International Group for the Psychology of Mathematics Education (Vol. 3, pp. 235–242). Szeged, Hungary: PME.Google Scholar
- McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Google Scholar
- McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press.Google Scholar
- McNeill, D., Quek, F., McCullough, K.-E., Duncan, S., Furuyama, N., Bryll, R., … Ansari, R. (2001). Catchments, prosody and discourse. Gesture, 1(1), 9–33.Google Scholar
- Nemirovsky, R. (2003). Three conjectures concerning the relationship between body activity and understanding mathematics. In N. A. Pateman, B. J. Dougherty, & J. T. Zillox (Eds.), Proceedings of the 27th Conference of the International Group for the Psychology of Mathematics Education (Vol. 1, pp. 105–109). Honolulu, HI: PME.Google Scholar
- Presmeg, N., Radford, L., Roth, W.-M., & Kadunz, G. (2016). Semiotics in mathematics education. ICME-13 Topical Surveys. https://doi.org/10.1007/978-3-319-31370-2_1.
- Radford, L. (2002). The seen, the spoken and the written. A semiotic approach to the problem of objectification of mathematical knowledge. For the Learning of Mathematics, 22(2), 14–23.Google Scholar
- Radford, L. (2008). Iconicity and contraction: A semiotic investigation of forms of algebraic generalizations of patterns in different contexts. ZDM, 40(1), 83–96.Google Scholar
- Radford, L. (2014). Towards an embodied, cultural, and material conception of mathematics cognition. ZDM—The International Journal on Mathematics Education, 46, 349–361.Google Scholar
- Radford, L., Bardini, C., & Sabena, C. (2007). Perceiving the general: The semiotic symphony of students’ algebraic activities. Journal for Research in Mathematics Education, 38(5), 507–530.Google Scholar
- Radford, L., Bardini, C., Sabena, C., Diallo, P., & Simbagoye, A. (2005). On embodiment, artifacts, and signs: A semiotic-cultural perspective on mathematical thinking. In H. L. Chick & J. L. Vincent (Eds.), Proceedings of the 29th Conference of the International Group for the Psychology of Mathematics Education (Vol. 4, pp. 113–120). Melbourne: University of Melbourne, PME.Google Scholar
- Radford, L., Edwards, L., & Arzarello, F. (2009). Beyond words. Educational Studies in Mathematics, 70(3), 91–95.Google Scholar
- Radford, L., & Sabena, C. (2015). The question of method in a Vygotskian semiotic approach. In A. Bikner-Ahsbahs, C. Knipping, & N. Presmeg (Eds.), Approaches to qualitative research in mathematics education (pp. 157–182). New York: Springer.Google Scholar
- Roth, W. M. (Ed.). (2009). Mathematical representation at the interface of body and culture. Charlotte, NC: Information Age Publishing.Google Scholar
- Sabena, C. (2007). Body and signs: A multimodal semiotic approach to teaching-learning processes in early Calculus (Ph.D. dissertation). University of Torino, Italy.Google Scholar
- Sabena, C. (2008). On the semiotics of gestures. In L. Radford, G. Schumbring, & F. Seeger (Eds.), Semiotics in mathematics education: Epistemology, history, classroom, and culture (pp. 19–38). Rotterdam, The Netherlands: Sense Publishers.Google Scholar
- Sabena, C. (2010). Are we talking about graphs or tracks? Potentials and limits of ‘blending signs’. In M. M. F. Pinto & T. F. Kawasaki (Eds.), Proceedings of the 34th Conference of the International Group for the Psychology of Mathematics Education (Vol. 4, pp. 105–112). Belo Horizonte, Brazil: PME.Google Scholar
- Sabena, C., Krause, C., & Maffia, A. (2016). L’analisi semiotica in ottica multimodale: dalla costruzione di un quadro teorico al networking con altre teorie. Relazione al XXXIII Seminario Nazionale di ricerca in didattica della matematica Giovanni Prodi, Rimini 28–30 Gennaio 2016. http://www.airdm.org/sem_naz_2016_25.html.
- Sabena, C., Radford, L., & Bardini, C. (2005). Synchronizing gestures, words and actions in pattern generalizations. In Proceedings of the 29th Conference of the International Group for the Psychology of Mathematics Education (Vol. 4, pp. 129–136). Melbourne, Australia: University of Melbourne, PME.Google Scholar
- Schiralli, M., & Sinclair, N. (2003). A constructive response to ‘Where mathematics comes from’. Educational Studies in Mathematics, 52(1), 79–91.Google Scholar
- Sfard, A., & McClain, K. (2002). Analyzing tools: Perspectives on the role of designed artifacts in mathematics learning. Journal of the Learning Sciences, 11(2&3), 153–161.Google Scholar
- Vygotsky, L. S. (1931/1978). Mind in society. The development of higher psychological processes. In M. Cole, V. John-Steiner, S. Scribner, & E. Souberman (Eds.). Cambridge, MA: Harvard University Press.Google Scholar
- Vygotsky, L. S. (1934/1986). Thought and language (Revised edition, A. Kozulin, Ed., Trans.). Cambridge, MA: MIT Press. (Original work published in 1934).Google Scholar
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.