Keywords

30.1 Introduction

At the turn of the millennium, in 2000, the provocative essay Where Mathematics Comes From by George Lakoff and Rafael Núñez pointed out the crucial role of perceptual and bodily aspects on the formation of abstract concepts, including mathematical concepts (Lakoff and Núñez 2000). The new stance emphasized sensory and motor functions, as well as their importance for successful interaction with the environment. Criticizing the platonic idealism and the Cartesian mind–body dualism, Lakoff and Núñez advocated that all kinds of ideas, including the most sophisticated mathematical ideas, are founded on our bodily experiences and develop through cognitive metaphorical mechanisms.

The book aroused a great interest in mathematics education and prompted many research studies highlighting the role of bodily and kinesthetic experiences in mathematical learning (Arzarello and Robutti 2008; de Freitas and Sinclair 2014; Edwards 2009; Ferrara 2014; Nemirovsky 2003; Radford 2014; Roth 2009; for an overview, see Gerofsky 2015).

More recently, embodied stances seem to receive a certain confirmation by neuroscientific results on “mirror neurons” and “multimodal neurons,” which are neurons firing when subjects performs actions, when they observe somebody else doing the same action, and when they imagine it (Gallese and Lakoff 2005). On the basis of these results, Gallese and Lakoff (2005) provide a new theoretical account on how the brain works, according to which “action and perception are integrated at the level of the sensory-motor system and not via higher association areas” (p. 459). In particular, such an integration would appear to be crucial not only for motor control, but also for planning actions, an activity typical of what is generally understood as “thinking.”

The terms multimodal and multimodality come therefore to indicate a feature of human cognition opposed to “modularity.” On the other hand, in the communication field the term multimodal is used with reference to multiple modalities that we have to communicate and express meanings to our interlocutors: words, sounds, images, and so on (Kress 2004). These communicative affordances have been acquiring increasing attention due the diffusion of new technological affordances, which are constantly developing new possibilities of interaction with them through our body.

In this paper, in line with Radford et al. (2009), multimodality refers to the importance and mutual co-existence of a variety of cognitive, material and perceptive modalities or resources in the mathematics teaching-learning processes, and more in general in the formation of mathematical meanings: “These resources or modalities include both oral and written symbolic communication as well as drawing, gesture, the manipulation of physical and electronic artifacts, and various kinds of bodily motion” (pp. 91–92). Including the embodied aspects in the analysis of mathematical thinking and learning brought to the fore the study of gestures as an important cognitive and communicative manifestation.

On the other hand, the attention to embodied and multimodal aspects needs to come to terms with the consideration of the social, historical, and cultural aspects in the genesis of mathematical concepts (Schiralli and Sinclair 2003; Radford et al. 2005). Mathematics is indeed “inseparable from the symbolic instruments” and the act of knowing is a “culturally shaped” phenomenon (Sfard and McClain 2002, p. 156) in which use of tools and signs play an important role.

This paper takes a semiotic stance to analyze gestures not as isolated variables, but rather as part of the multimodal resources at the students’ disposal in order to bridge the gap between everyday experience and formal mathematics. The multimodal resources will be considered as signs entering in meaning-making processes, and will be analyzed through the semiotic bundle lens (Arzarello 2006). Previous research from this perspective has suggested that gestures can contribute not only to the semantic content of mathematical ideas but also to the logical structure that organizes them in mathematical arguments (Arzarello and Sabena 2014). In this line of research and adopting a case study methodology, the following research question will be addressed:

What specific contribution can gestures, when they are considered as signs in semiotic bundles, provide to students’ argumentation processes?

In the next sections, the theoretical framework for the research is presented: It is constituted by theoretical elements and results from gesture studies in psychology and by a semiotic perspective for multimodality grounded on Vygotsky’s account of signs and on the semiotic bundle notion. Afterwards, selected data from a case study in primary school in the context of strategic interaction games will be analyzed and discussed according to an analytical generalization stance.

30.2 Gestures as Multimodal Resources

Gestures accompanying discourses are a widespread phenomenon (not only Italian!), as the pioneering work by Kendon has documented since the 80s (Kendon 1980). Since then, psychological and psycholinguistic studies have been stressing that speech and gestures are closely linked and that gesturing is relevant in communication and thinking processes (McNeill 1992, 2005; Goldin-Meadow 2003).

McNeill (1992) found speech and gestures to be closely linked in many respects: They are temporally synchronous in phonological (the central phase of the gesture coinciding with the peak of the phonological phrase), semantic (at the meaning level), and pragmatic (their function in the discourse) aspects. Also, in child development, gesture and speech proceed together. At the cognitive level, some scholars have identified their function being important in lightening the working memory, offering the possibility for cognitive resources to do their best to reorganize (Goldin-Meadow et al. 2001).

These cognitive interpretations provide elements that can explain, for example, why we gesticulate in telephone conversations (de Ruiter 1995), why when we are prevented gesturing our discourse becomes less fluid, or why even blind from birth use gestures while speaking. These phenomena cannot be explained only in terms of interpersonal communicative dimension, and so gestures are claimed to have a constitutive role also in thinking processes.

Vygotsky (1934/1986) had already stressed the constitutive role of language in thinking by saying that “thought is not merely expressed in words; it comes into existence with them” (p. 218); since then psychological studies on gestures have pushed in the direction of extending this constitutive role of language to the speech-gesture unity. Quoting McNeill (1992), we can say that “gestures do not just reflect thought but have an impact on thought. Gestures, together with language, help constitute thought [emphasis original]” (p. 242). It is within this Vygotskian hypothesis that I frame the role of gestures in mathematical activities.

Gesture studies have provided other tools of analysis, such as categories for their classification. McNeill (1992) has classified gestures as:

  • Iconic: if they bear a relation or resemblance to the semantic content of discourse (for example, inclining two hands to indicate a roof);

  • Metaphoric: similar to iconic gestures, but with the pictorial content presenting an abstract idea that has no physical form (a classical example is the hand in the act of holding an object, when referring to idea of “a certain topic” in the discourse);

  • Deictic: if they indicate objects, events, or locations in the concrete world.

  • Beats: if they contribute to stress some parts of the discourse.

Deictic gestures are usually performed with the extended forefinger (sometimes with hand-held objects, such as a pen) and are also called pointings. Apparently simple, pointing is indeed a complex act. Besides concrete pointings (such as indicating a book on the table), research has also identified abstract pointings, when the hand or fingers are extended in the space as to indicate something, but the space it actually empty. In McNeill’s (1992) interpretation, “the speaker appears to be pointing at empty space, but in fact the space is not empty; it is full of conceptual significance. Such abstract deixis implies a metaphoric use of space in which concepts are given spatial forms” (p. 173). Such a classification is not based on the physical features of gesture, but by considering the relationships with contextual information: this entails that the interpretative process needs to take into account the broader context in which a gesture is performed. A second remark concerns the fact that the same gesture may belong to more than one category; therefore, the categories have to be considered dimensions along which a gesture can be featured, more than in a classificatory view.

Furthermore, gestures are sometimes characterized by repetition: distinct features of a gesture recur over the length of a discourse (although not necessarily in consecutive gestures), and the recurrence can be signaled by the form of the hand shape, its location, orientation, motion, rhythm, and so on (McNeill et al. 2001). This phenomenon is called catchment and may be related to discourse cohesion:

By discovering the catchments created by a given speaker, we can see what this speaker is combining into larger discourse units – what meanings are being regarded as similar or related and grouped together, and what meanings are being put into different catchments or are being isolated, and thus are seen by the speaker as having distinct or less related meanings. (McNeill et al. 2001, p. 10)

Catchments may therefore be of great importance because they can give information on the underlying meanings in speech and dynamics. In a classroom setting, studying the catchments could provide clues on the evolution of meanings in students. In addition, catchments can contribute to the organization of an argument at a logical level, as discussed in Arzarello and Sabena (2014).

30.3 A Semiotic Approach to Multimodality

The choice of adopting a semiotic approach to study the role of multimodality and gestures in mathematical activities stems basically from two considerations. The first is epistemological, dealing with the assumption that mathematical objects are not directly perceivable using our senses and need by their nature to be mediated by signs, such as the graph of a function for the function concept. Indeed, signs and transformations between them are at the heart of the mathematical activities:

The significance of semiosis for mathematics education lies in the use of signs; this use is ubiquitous in every branch of mathematics. It could not be otherwise: The objects of mathematics are ideal, general in nature, and to represent them—to others and to oneself—and to work with them, it is necessary to employ sign vehicles, which are not the mathematical objects themselves but stand for them in some way. (Presmeg et al. 2016, pp. 1–2)

The second consideration is psychological, concerning how meanings are formed and evolve. In Vygotsky’s account of human cognitive development (or cultural development), signs play a crucial role (Vygotsky 1931/1978). By virtue of their social meaning, signs serve individuals as a way to exert voluntary control on their behavior, in a way similar to the way that road signs signal events to individuals to regulate their conduct. Analogous with tools in labor activities, signs work, on the individual psychological level, as “stimuli-means” standing for some characteristic or aspect of the socially shared experience and steering one’s own mental processes:

The invention and use of signs as auxiliary means of solving a given psychological problem (to remember, compare something, report, choose, and so on) is analogous to the invention of tools in one psychological respect. The signs act as instruments of psychological activity in a manner analogous to the role of a tool in labor. (Vygotsky 1931/1978, p. 52)

From this perspective, signs are considered in their functional role as psychological tools that allow the subjects to reflect and plan actions and act as cultural mediators (Radford and Sabena 2015). This is a very general idea of a sign that does not assign prescriptions on what can be a sign and which specific features it should have: A gesture can also be considered a sign, as Vygotsky himself highlighted in his famous example of the pointing gesture to illustrate the internalization process starting from the meaning assigned by the mother to the child’s hand movement (Vygotsky 1931/1978).

In order to include gestures as well as other more classical registers, Arzarello developed the semiotic bundle construct (Arzarello 2006; Arzarello et al. 2009) as a system made of different signs (or semiotic resources) and their mutual relationships that are produced by students and possibly the teacher during mathematics activities: words (spoken or written), written diagrams, gestures, tools, and so on. Similarly to Radford’s idea of “semiotic system” (Radford 2002), the semiotic bundle includes both the classical registers, with precise and codifiable rules of productions and transformation (Duval 2006), and the embodied ones, allowing us to provide a semiotic account of the multimodal processes occurring while learning and teaching mathematics. An example can be constituted by students’ words, gestures, and drawn figures while solving a geometrical problem.

The semiotic bundle is characterized by two key features:

  • A systemic character, revealed by a synchronic analysis of the relationships between the different kinds of signs at a certain moment (like a sort of “semiotic picture”)

  • A dynamic nature revealed by a diachronic analysis focusing on the evolutions of signs and of their transformations over time (a sort of “semiotic movie”)

Synchronic and diachronic analysis—which are distinguished only for the sake of analysis—are performed by considering closely the video recordings from classroom activities students are engaged in, together with their multimodal transcripts (i.e., transcripts that include not only words but also record gestures and other kinds of signs). It is interesting to remark how the possibilities offered by new technologies for the study of the interaction between students and the teacher in the classroom gave a boost to considering multimodal aspects in mathematics learning. Although in the early 90s attention to the “classroom discourse” in the teaching learning of mathematics had already emerged, it was the use of video recordings that opened up the possibility to observing phenomena that had hitherto been unnoticed due to its undetectability. Gestures and other embodied resources have thus begun to be considered among the resources through which communication and conceptualization are realized. In other words, the strong push towards new theories that has come from methodological aspects has required new analytical tools, such as the semiotic bundle.

In the following, the semiotic bundle lens is applied to empirical data on mathematical argumentation processes carried out by primary students. The analysis will use a fine-grain focus on video-recording data with the aim of identifying and theorizing key gestural phenomena that play a role in such processes.

30.4 A Case Study: The Race to 20

The case study is based on a strategic interaction game called “Race to 20,” which was used by Brousseau to illustrate the theory of didactical situations (Brousseau 1997). The case study is part of a design-based research project on the use of strategy games for fostering problem-solving and argumentation processes from primary to secondary school.

The game is played by two players who struggle to reach the number 20 by adding alternatively small numbers. Specifically, the first player chooses a number between 1 and 2, then the second player must add 1 or 2 to the previous number and say the result, then the first player adds 1 or 2, and so on. The player saying 20 wins the game. In game theory, it is a perfect information game with complete information, based on sequential decision-making. As the reader may know or can check, the winning strategy consists of starting as the first player and in following the number sequence 2-5-8-11-14-17-20. In this kind of game, the player needs to determine, for any move by the opponent player, the right move in order to win the game. These processes may be related to the logical scheme of coordinating a universal qualifier with an existential one, as it is the case in many mathematics theorems.

I will refer to data from a classroom discussion in Grade 4. The discussion follows some lessons in which the students played the game in pairs by writing the added numbers on the top of arrows from left to right and the results in a line from left to right (similar to Fig. 30.1).

Fig. 30.1
figure 1

Semiotic template introduced by the teacher to play the Race to 20

Full size image

This semiotic template was introduced by the teacher in order to allow the students to keep record of both the winning numbers and the added ones. This record was meant to support them in determining regularities that are at the base of the winning strategy.

The discussion is engineered right after the students have finished a classroom team tournament, in which representatives of each team have played the matches at the blackboard (the last match is shown in Fig. 30.1). The teacher initiates the discussion by making explicit the goal of providing a strategy to win the game and justifying it.

We will focus in detail on the specific contribution of some children—Giulio, Eliana, and Elisa—but first let us give some contextual information, in order to make the analysis understandable.

As the discussion starts, numbers 14 and 17 are soon identified as winning numbers: In some cases justifications are based on the possible moves of the two players, as Marta states:

Marta You have to get first to 14 and then to 17. Because if you do 14 plus 1 and you get 15 and then you do plus 2 and you get to 17, then . . . you do plus 1 to arrive to 18 and the other does 2 and gets to 20. Whereas if from 14 you do plus 2, you arrive at 16 and the other one does plus 1 to arrive at 17, the other 19 if he does plus 2, you do plus 1, and you get 20. So however, from 14 to 17 you arrive anyway to 20.

In other cases, they rely on the empirical observation of what did actually happen during the tournament. Through backward induction, the number 11 begins to be identified as a winning number and related to 14 and 17:

Diego::

11 maybe is an important number, because maybe my team adds 2 and it is 13, the other team adds 1 and arrives at 14, I add 1, 15, they add 2 and it is 17.

At this point, after about 20 min of classroom discussion, Giulio proposes a general rule to identify all winning numbers:

Giulio::

I think that for the winning numbers you always remove 3: from 20 you remove 3 and you arrive at 17; from 17 you remove 3 and you arrive at 14, I think that another winning number could be 11, could be . . . 8, could be . . . 5, could be . . . 2.

Giulio’s strategy identifies all winning numbers starting from the winning result, 20, and moving backwards through repeated subtractions in a process of backward induction, as described in game theory.

The strategy is expressed verbally in general terms, without simulations of moves, and it is accompanied by several gestures. Table 30.1 reports the transcript of the initial part of Giulio’s utterance, enriched with the gesture component and the Italian original words. The underlined words indicate that they are co-timed with the shown gesture and the same convention will be used in the following tables.

Table 30.1 Multimodal transcript of the first part of Giulio’s strategy
Full size table

When saying “you always remove 3,” Giulio moves his hand from right to left (from the child’s perspective): This movement can be interpreted as indicating subtraction, with reference to the number line, which is often used in the Italian curriculum. With this interpretation, the gesture can be classified as a metaphoric gesture indicating subtraction. I remark that in the original Italian version, Giulio uses the term togli, which is used both in everyday contexts to say “remove, take away” and in mathematical context in primary school to indicate “subtract.”

The hand movement from right to left is repeated when the rule is applied in order to identify which are the winning numbers. From Table 30.1 (Pictures b and c) it can be noticed that when Giulio is saying “from 20 you remove 3,” his hand is moving leftwards. Now, three fingers are pointed out, and overall we find two metaphorical references condensed in a single gesture:

  • right-to left movement → subtraction

  • three fingers → number 3

When completing the sentence and uttering the winning numbers, Giulio is performing abstract pointing gestures downwards that are co-timed with the uttered numbers (in Table 30.1, the case of 17 is reported in the last column).

If we consider the whole sequence, we see that the subtraction of 3 is repeated in order to obtain 14; this repetition is also expressed by the repetition of the same gesture configuration of the three fingers extended (pictures are not reported for reasons of space). The same metaphorical gesture is hence repeated in a catchment expressing that number 3 is always subtracted, in order to get all the winning numbers.

Afterwards (“I think that another . . .”), we can notice a type of semiotic contraction occurring within the semiotic bundle: Speech reduces, mentioning only the winning numbers, and shifts to a hypothetical level (“could be . . .”); also, gestures appear to reduce in their movements, ending up with quick abstract pointing left and downwards co-timed with the utterance of the winning numbers (8, 5, and 2).

At this point, the teacher asks Giulio to explain his idea. Here it is the verbal transcription of Giulio’s argument:

Teacher::

Explain well this idea.

Giulio::

Because . . . that is I don’t know, if I arrive at 2 . . . I don’t know, I begin, I make 1, no I make 2, he arrives and makes 1, I put 2 and I arrived at 5, which I think is another winning number . . . yes, arrived at 5 . . . it is a winning number, I think. Then . . . he adds 2, say, I add 1 and I arrived at 8, which is another winning number. She adds 1, I add 2 and I arrive at . . . 11, which is a winning number. He adds 2, I add 1, and I arrive at 14, which is another winning number, he adds 1 I add 2, we arrive at 17 which is a winning number, he adds 1 or 2, I add 1 or 2, and I win.

The subtraction turns now into an onward movement that starts from the very first move (number 2) of an imagined match between himself and another player. This movement is produced by means of a repetition of the same linguistic structure: “He adds . . . I add . . . and I arrive at . . . , which is a winning number.” This repetition is not just a mere repetition of words, but is performed with a rhythmical structure in sound, which is preserved along the entire sentence and contributes to convey the general character of the found rule (similar to what was discussed in algebraic context in Radford et al. 2007).

Gestures are constantly present, starting from the first simulated move to the last (winning) one. Table 30.2 reports some pictures of gestures accompanying the very first part of Giulio’s sentence.

Table 30.2 Multimodal transcript of the first part of Giulio’s argument
Full size table

While uttering the moves of an imagined match, Giulio is performing again two kinds of metaphoric gestures. When indicating his own moves, the gesture indicates the uttered numbers by pointing out the correspondent number of fingers, i.e., two fingers when saying 2 (Pictures a and c in Table 30.2). When referring to the other players’ moves or to the obtained result, the hand is held open upwards as containing something (Pictures b and d in Table 30.2): In this case the metaphoric reference is made to underline a certain kind of generality of the uttered numbers.

At a certain point, when mentioning an opponent move, Giulio uses a linguistic expression that in English may be translated as “say” or “for example” (in Italian it is tipo) and that can be interpreted as expressing the germs of the concept of “any number.” When uttering “say,” the student performs a gesture consisting of an open hand quickly turned around (Table 30.3, Picture a).

Table 30.3 Multimodal genericity conveyed within the semiotic bundle
Full size table

The gesture is again metaphoric and the semiotic bundle of words and gestures underlines that the number 2, chosen to indicate the opponent’s move, is to be considered as one possibility among others (all the possible moves): It is one generic move. This is indeed a very delicate logical relationship to manage: the articulation between a universal qualifier (for any move from my opponent player) with an existential one (the move that I choose after him). We see that the gesture-speech combination allows the students to successfully manage it.

From this moment on, when uttering the imagined moves by the two players, abstract pointing gestures are enacted with left-right spatial alternation, which indicates visually the alternation between the two players in the game (Table 30.3, Pictures b and c). This spatial alternation can be interpreted as helping the student to keep control of the argument at the local level, that is to say, to control the choice of the moves and counter-moves in the imagined sequence. The gesture spatial alternation is repeated several times (catchment) for any couples of moves and counter-moves, realizing the same rhythm of the accompanying words. As indicated by McNeill (2005), gesture catchments provide the discourse with cohesion. In this case, it contributes in structuring the entire argument at a global level.

The written template through which the game has been played has possibly helped Giulio in developing his strategy, working as an interiorized tool. As a matter of fact, when saying the general rule, he is looking at the blackboard (Table 30.1, Picture a), where the record of last match is still written (see Fig. 30.1). We remark that this match had not been played according to Giulio’s strategy and that after this initial moment, we do not find any explicit reference to performed matches, for instance, with pointing gestures to the blackboard or to his notebook: This could be another index of the general level reached by Giulio in his argument.

The discussion focuses then on Giulio’s strategy. Some students immediately agree with Giulio and produce their own argumentations, such as Eliana:

Eliana::

I agree with Giulio because practically any time you have to reach a lucky number you must add 3, because first you add 1 and then you add 2 or first you add 2 and then add 1.

Eliana makes explicit that winning or “lucky” numbers can be reached by adding 3 (while Giulio mentioned subtraction by 3) and produces an argument for this by referring to the two numbers 1 and 2 that can be played in the game. When she says “you must add 3,” she accompanies her speech with a gesture with the right hand turning from left to right (Table 30.4, Picture a), which may be referring to the addition on the number line.

Table 30.4 Eliana’s argument on the strategy of adding 3
Full size table

Eliana explains also where this number 3 comes from, i.e., the combination of the possible subsequent moves of the two players. When uttering the numbers added to compose 3, she moves first the left hand from left to right (Picture b in Table 30.4), then the right hand with the same movement (Picture c in Table 30.4); she mentions the two possible combinations (1 + 2 or 2 + 1) and repeats the gestural combination in a catchment (Pictures c and d in Table 30.4).

As in the case of Giulio, we can notice the spatial alternation left-right as a metaphoric reference for the alternation of the two players, and again we can notice a catchment. But differently from the case of Giulio, now the fact that there are two alternating players is expressed by Eliana only through her gesture, because in her speech she uses always the pronoun “you,” possibly in impersonal sense.

Again, we may identify two metaphoric components condensed in a single gesture:

  • left-to-right movement → addition

  • spatial alternation → players alternation

Right after Eliana, Elisa intervenes:

Elisa::

So overall . . . if you play . . . you add 3 every time, and so if you can arrive at the numbers that there are 3 [pointing out index and thumb; Picture a in Table 30.5], that is if . . . Overall it is 3 [shifting the pointed fingers from left to right; Picture b in Table 30.5], because if you add 1 [placing the pointed fingers at her left; Picture c in Table 30.5] and the other adds 2 [placing the pointed fingers at her right, Picture d in Table 30.5], if you add 2 and the other adds 1 [repeating the sequence with pointed fingers at her left and then right as in Pictures c and d in Table 30.5], overall it is 3 [shifting again the pointed fingers as in Picture b in Table 30.5] and so you must be able to pick the numbers that are . . .

Teacher::

At a distance . . .

Elisa::

. . .of 3.

Elisa accompanies her speech with a gesture performed with two fingers pointed as if they were holding a little stick. This gesture is performed for the first time when she says “there are 3” (Picture a in Table 30.5) and is kept until the end of the sentence. It indicates metaphorically a fixed distance, similar to what has been described in an early calculus context in previous studies (Arzarello et al. 2009; Sabena 2007, 2008). The word distance is never uttered (it will be uttered immediately later, after a prompt from the teacher): The gesture is complementing her words and providing further meaning to her multimodal discourse.

Table 30.5 Elisa’s condensing gesture
Full size table

When saying “overall it is 3,” the holding-stick gesture is shifted from left to right (Picture b in Table 30.5): The left-to-right movement indicates that the fixed distance (of 3) allows one to pass from a winning number to the following one in the sequence. The girl is keeping her eyes towards the blackboard, where the last match played is still written, according to the template chosen by the teacher (Fig. 30.1). In this template, the subsequent moves are written one after the other, in a horizontal way. The horizontal movement of Elisa’s gesture may be interpreted against this background, suggesting that the semiotic choice of the teacher has been useful for developing the students’ argument. At the same time, the gesture may be referring metaphorically to an addition on the interiorized number line. We find another example of a gesture condensing different meanings through its metaphoric references and its dynamism. Through the condensing character of the gestures and in synergy with speech, the different meanings come to be connected to build an important part of the argument.

Through a gesture repetition or catchment, the condensing gesture is then combined with spatial alternation referring again to the two players (Pictures c and d in Table 30.5): The catchment provides support in shifting from a relationship between moves to a relationship between numbers that can justify the moves and indicates that such a distance does not vary over the entire winning sequence in the game.

30.5 Discussion

This paper adopted a multimodal perspective on mathematics teaching and learning processes and chose a semiotic tool in order to address it: the semiotic bundle, with its wide notion of sign drawing from Vygotsky’s works, and its systemic and dynamic features. It focused in particular on the role of gestures in interaction with the other semiotic resources used in the classroom—speech first of all, but also written signs—and addressed primary students’ mathematical argumentation processes in the context of strategic interaction games.

Through a case study and qualitative-interpretative analysis, it has been shown that gestures may contribute to carrying out argumentations that depart from empirical stances and shift to a hypothetical plane in which generality is addressed. From this case study, we can get also some insights on how gestures can do this. In particular, specific features have been identified: semiotic contraction, the condensing character of gestures, and the use of gesture space in a metaphorical sense combined with catchments. They will be briefly discussed, referring to the data analysis reported above.

When Giulio identifies and/or expresses (we do not have sufficient data to determine) the general rule of “always removing 3,” we see his sentences becoming shorter and shorter and at the end just expressing the winning numbers, accompanied by abstract pointing gestures (Table 30.1, Fig. d). This is a type of semiotic contraction that has also been found in other contexts and at different ages, such as pattern generalization and function graphs (Sabena et al. 2005; Sabena 2007). From an epistemological point of view, semiotic contraction characterizes modern mathematical symbolism, and from a cognitive point of view it is a precious mechanism. Radford (2008) relates contraction to focusing attention to the elements that are relevant for a certain situation and to a deeper level of consciousness: “Contraction is the mechanism for reducing attention to those aspects that appear to be relevant. This is why, in general, contraction and objectification entail forgetting. We need to forget to be able to focus” (p. 94).

Semiotic contraction can be found also in what Vygotsky (1934/1986) calls “inner speech,” which is described discussing language as a paradigmatic signs system. Inner speech is described at a structural level by syntactic reduction and phasic reduction and at semantic level by agglutination. Syntactic reduction is a specific form of abbreviation that curtails the subjects of sentences and leaves pure predication. Syntactic articulation results are therefore minimized to the pure juxtaposition of predicates. Phasic reduction consists of minimizing the phonetic aspects of speech, namely curtailing the words themselves (for example, writing “u” instead of “you”). Agglutination consists in combining words, gluing different meanings (concepts) into one expression (for example, “highway,” formed by “high” and “way”). Nowadays, instant messaging communication systems on our smartphones extensively exploits semiotic contractions (combined with additional iconic features, such as the emoticons), typically in informal communication by people sharing most of the contextual information.

With syntax being reduced, Vygotsky (1934/1986) claims that semantics undertakes a contrary movement, with meaning coming to the fore: “With syntax and sound reduced to a minimum, meaning is more than ever in the forefront. Inner speech works with semantics, not phonetics” (p. 244).

We may observe that gestures, because of their spatial and kinesthetic nature, do not need processes of agglutination to combine meanings, as some languages do: It is their enactment itself that may produce the same result of combining meanings as agglutination does. A specific form of semiotic contraction characterizing gestures is in fact what I call blending or condensing gestures, which are gestures expressing (at least) two different meanings. We have seen two examples above:

  • Giulio, with the right-to-left gesture with three fingers pointed out, indicates both the number 3 and subtraction (Table 30.1, Pictures b and c).

  • Elisa, with two fingers pointed as if she were holding a little stick, shifted from left to right, which may interpreted as indicating both a fixed distance and the fact that this distance allows one to pass from one winning number to the next one (Table 30.5, Pictures b and c). The co-timed speech specifies that this distance is 3, obtained as the sum of 1 and 2.

In both examples, gestures condensed or blended two meanings by combining a dynamic component with the hand shape: This dynamic feature has been observed also by Calbris (2011) in what she calls “polysign gestures.” In previous studies in the mathematical domain (Sabena 2007, 2008, 2010), the condensing or blending character of gestures has been identified in functions and graphs contexts and associated with iconic features of gestures. In the reported study, this feature is shown in an arithmetic domain and associated with the metaphoric feature of gestures, as McNeill (1992) classified them. Condensing two different meanings, each of these gestures establishes two different kinds of metaphorical references, one of which calls into play the number line, a didactical tool suggested in the Italian curriculum. By exploiting space in order to reason about numbers, the number line itself has a metaphorical nature. A double or even multiple blending process seems therefore to be activated by some metaphorical gestures typical of the mathematical domain. This theme requires deepening the reflection of what “metaphorizing” means at a cognitive and at a semiotic level, and further research is needed (for preliminary results using cognitive metaphors and blended spaces, see Sabena et al. 2016).

Metaphoricity appears to be related also to the use of gesture space with spatial alternation, as we have seen in Giulio and in Eliana. Giulio moves his hand left and right when mentioning the moves and countermoves in his imagined match (Table 30.3, Pictures b and c), while Eliana alternates her left and right hand for the same purpose (Table 30.4, Pictures b and c and Pictures d and e). Through gesture spatial alternation, empty spaces acquire meanings, which in the data appear related to a sort of local level, either in the game (imagined subsequent moves, in the case of Giulio), or in the mathematical argument (numbers to add to compose 3, in the case of Eliana).

As seen in the analysis, such a spatial alternation is repeated many times, realizing what in gesture studies is called a catchment and is interpreted as providing the discourse with cohesion (McNeill 2005). In this case study, gesture catchment is interpreted as supporting the students in structuring the entire argument at a global level. Previous results about how opposite spatial locations are exploited gesturally to indicate mutually excluding cases seem to confirm this interpretation (Arzarello and Sabena 2014).

For the sake of analysis, the different gestural features contributing to providing general meaning and structure to the argumentation process have been discussed here one after the other. However, as can be observed going back to the data analysis, many of these features intertwine; furthermore, the analysis of gestures needs to take into account the entire semiotic bundle. For example, only a systemic analysis of words and gestures can show how, even if it is describing a certain hypothetical match between himself and another player, Giulio’s argument contains essential aspects conveying generality: the rhythmical repetition of the same linguistic structure, accompanied with a corresponding catchment (Tables 30.2 and 30.3); the use of generic words accompanied by a generic gesture (Table 30.3, Picture a); and the use of abstract pointings while uttering the possible moves (Table 30.3, Pictures b and c). In the case of Eliana’s argument, it is striking to observe how gestures and words complete each other in a synchronic way.

If we analyze the children’s contribution in a diachronic way, further observations may be drawn that provide elements to describe the classroom discussion evolution in a multimodal perspective. To give an example, it is interesting to see how Giulio’s spatial alternation with his right hand evolves in Eliana’s alternation of the two hands one after the other (see Table 30.4); in this latter case, the subject in her speech is not changing (it is always “you”), showing a tension towards the arithmetical relationships rather than on the strategic game interaction. This paves the way to the following Elisa’s intervention about the “numbers so that there are 3.”

A final consideration is reserved for the didactical implications of such a fine-grained analysis. In this paper, little attention has been devoted to the didactical variables of the situation. Of course, the teacher’s choices are never neutral with respect to the use of any semiotic resource in the classroom, gestures included. An example in the data is the semiotic template through which the Race to 20 has been presented to the students and through which they play the game (Fig. 30.1). We have seen that this choice—which resonates with the didactic tool of the number line—has provided an essential tool for the students not only to play the game, but also for developing argumentations about how to win it. In particular, Elisa’s multimodal argument about the “distance of 3” between winning numbers shows a relation to the semiotic written template through which the game was played. The results of this analysis appear therefore to offer elements for validating the choice of the teacher. It is beyond the scope of the analysis, however, to discuss why for Elisa (and for some students) it did work, whereas for others, further reflection was needed. Classroom discussion appears indeed to be a suitable means for allowing the development of multimodal argumentations such as the one described, in which the students may exploit gestures as semiotic resources. This requires, of course, that gestures are considered legitimate in the classroom (as it happens in the analyzed case: The teacher supports Elisa in her multimodal argument by considering the contribution of her gesture and offering her the missing word). Even more, the teacher can contribute to classroom mathematical activity through her gestures in order to make the mathematical discourse evolve towards culturally established mathematics forms (see the “semiotic game” in Arzarello et al. 2009). Ongoing research indicates that the teacher can have an important role in the evolution of signs within the semiotic bundles and in building “multimodal semiotic chains” that make mathematical meaning progress through argumentation processes (Maffia and Sabena 2015, 2016). Further extensive research is still needed in order to unveil and exploit fully the potentiality of gestures as didactical means in the classroom.