Metaphors in Computer Programming Education

At its core, a computer program comprises of words and characters representing a series of events that will be executed by a computer. At first glance, some of the code lines might seem self-explanatory, since they sometimes reflect human actions, such as print () or read (). However, underneath the surface of the computer there are no human actions, rather a network of electrons performing the work. Instructions such as ‘jump’ or ‘run’ will obtain their meaning by simple references to embodied experiences (Colburn & Shute, 2008), while an expression such as ‘killing a process’ will have its source in an abstract sense of death. Such conceptual metaphors (e.g., Lakoff & Johnson, 1999) provide for a more tangible discussion on computer programming. Over time, scholars have explored the use of metaphors in other domains of computer science education to identify how, for example, code is described and used in educational environments and educational literature (e.g. Dufva & Dufva 2016; Hendry, 2006; Larsson, In press), how students employ gestures in their reasoning about programming (Manches et al., 2020) or how metaphors can be used as a basis for interaction design (Carroll et al., 1988; Hurtienne et al., 2010).

In his paper Computer Metaphors: Approaches to Computer Literacy for Educators, Peelle (1983) advocates the use of a variety of metaphors (e.g., THE COMPUTER AS A PERSONFootnote 1 and THE COMPUTER AS A TOOL) when introducing computer literacy in general education. Using the notion that “metaphors cultivate the mind” (p. 91), Peelle argues that the use of metaphors relating to already established knowledge will help students to comprehend new concepts within a computer science context. His main thesis in the paper is that, by introducing and moving beyond said metaphors, teachers will be able to work together with computer manufacturers to devise suitable systems for facilitating students’ learning. Partly confirming this line of reasoning, Colburn & Shute (2008) attribute metaphors with the ability for students to use their previous understandings from one area and map these onto computer programming. Consequently, metaphors will always play a big part in how computers and computer-related concepts are communicated, structured, and learnt (Colburn & Shute, 2008).

Recently, researchers have started conducting empirical studies exploring the use of metaphors in computer programming education. For example, Pérez-Marín et al., (2020) present evidence in support of their hypothesis that employing metaphors when teaching computer programming helps children to develop computational thinking. They do, however, emphasise that teachers might need support when developing such metaphors. Further support for this hypothesis is found in Chibaya (2019), where students’ programming performance was compared based on how specific programming concepts had been presented, either grounded on technical terms generally used in textbooks or adopting a metaphor-based method (e.g., variables presented in terms of containers). In an adjacent study, comparing the effects of oral allegories and metaphors when working with complex programming tasks, Hidalgo-Cespedes et al., (2018) found that metaphors are not always a useful learning tool in all contexts. Rather, it is a matter of matching a specific source domain to the right programming task. Here, the authors note that it would be fruitful to explore students’ vocabulary in relation to their programming performance. Further studies on the topic of programming metaphors have investigated students’ use of metaphors when solving programming tasks. For example, Agirbas (2018) found that allowing students to form their own “design metaphors” facilitated the design process itself, in that the metaphor places natural constraints on how to solve problems that would otherwise have a vast number of possible solutions. Also, the metaphors helped the students to develop versatility in their reasoning when solving problems. Altogether, these studies support the use of metaphors in education being beneficial for students’ general computer literacy as well as their specialised programming skills. Nevertheless, little is known about teachers’ spontaneous use of metaphors in speech and gestures when teaching programming.

Aims and Research Question

The overarching aim of this study is to explore three teachers’ use of conceptual metaphors (Lakoff & Johnson, 1999), expressed in speech and gestures, in a computer programming classroom. The following research questions will be raised and answered.

  1. 1.

    How do the teachers’ conceptual metaphors relate to specific programming concepts?

  2. 2.

    How can teachers’ use of conceptual metaphors provide tangible qualities of abstract programming concepts?

Theoretical Framework

Previous Research

The role of gestures in combination with spoken language have been used to obtain insights of metaphors in different areas of STEM education (e.g. Alibali & Nathan 2012; Dreyfus et al., 2015; Ping et al., 2021). Other studies within the field of computer programming, for example Manches et al., (2020) show promising and interesting results in terms of how computing constructs are communicated by students using both gestures and verbal utterances. When asked to explain three computing concepts – algorithm, loops and conditional statements – students display two different embodied metaphors (Manches et al., 2020). The first one, COMPUTING CONSTRUCTS AS PHYSICAL OBJECTS, where the students performed a grasping or pinching gesture when explaining the concepts, indicating that computing constructs such as code or data were treated as physical objects. The second one, COMPUTING PROCESSES AS MOTION ALONG A PATH, where students used gestures in three different directions: up to down, left to right and pathway forwards from the body. The students also demonstrated a combination of these two metaphors when, for example, “pinching an imaginary object and moving the pinched hand down in progressive steps” (Manches et al., 2020, p. 7). Similar gestures have been described by Larsson et al., (2021) in a classroom setting. Here, the authors state that the teacher in their study, generally used vertical hand movements while speaking about code-lines. These results are in line with another small-scale study, which explored teachers’ use of embodied representations in a computing classroom (Solomon et al., 2020), in which the authors conclude that the teachers’ gestures make abstract concepts vivid, for example by helping students to visualise the otherwise invisible logic of a “list”.

Building on studies such as the above, we argue that studying teachers’ gestures has the potential to give new insights into how computer science is communicated in a programming classroom. The inclusion of gesture analysis seems to provide additional information to analyse students’ and teachers’ embodied reasoning on computer programming. Nevertheless, there are still theoretical and methodological issues that need to be addressed.

Metaphor and gesture

Metaphorical language could be described as the type of language used when a speaker does not mean what is literally being said (Gibbs & Colston, 2012). Of particular interest in this study are the conceptual metaphors, as described by for example Lakoff and Johnson (1999); metaphors that display mappings between sensorimotor experiences and abstract phenomenon (e.g., “the program tells the user to type” or “the computer just crashed” or “I had to kill all processes”). Williams (2008) suggests that, within an instructional context, “gestures are used to map conceptual elements profiled in speech to specific structures […]. When these mappings cross distinct domains, they can produce metaphoric construals of structure in the world” (p. 30). Consequently, gestures – in the same way as verbal utterances (e.g., Gibbs & Colston 2012) or textual expressions (e.g., Grady 1997) – could be seen asa cognitive vehicles for connecting experience and abstract phenomenon. Here, the gesture bears the grounding of a metaphor (Chui, 2011; Jensen & Greve, 2019; Williams, 2008) and serves as a bodily enactment of the source domain of a metaphor. Cienki & Müller (2008) argue that:

“[it] is certainly often the case that metaphoric gestures depict the abstract in terms of the concrete; yet, we would like to point out that metaphoricity is not reduced to conceptualizing the abstract in terms of the concrete. Rather metaphor is a cognitive procedure of understanding one thing in terms of another and hence may also apply to two concrete entities – such as for instance characterizing a woman’s body gesturally in terms of an hourglass, or when accounting verbally for all kinds of objects in terms of body parts, as is the case in expressions such as: the foot of a mountain, the leg of a table, or the arm of chair. Therefore, we are calling metaphoric gestures the ones which have the potential to engage an active cross-domain mapping, that is, the cognitive process of understanding something in terms of something else.” (pp. 485–486.)

In this paper, gestures are viewed as spontaneous movements of the hands, head and upper torso that reflect thought, language and cognition (e.g. Cienki 2016; Kang & Tversky, 2016; Kendon, 2007; McNeill, 1992; Müller, 2019). By adding spatial elements to verbal utterances (verbo-spatial metaphors) or providing utterances with additional imagery (verbo-gestural metaphors), the communicative burden is divided between gestures and speech. Consequently, gestures play a vital role in our day-to-day communication (Cienki & Müller, 2008). Furthermore, gestures can add nuances and additional information to verbal expressions (Cienki, 2016; McNeill, 2008), and are hence a significant part of people’s full-body interaction with their peers and their environment (Gibbs, 2019; Müller, 2009; Sweetser, 2007).

Based on the above, one could say that a teachers’ way of speaking and gesturing (embodying a metaphor) is an articulation of his understanding of computer programming within a specific environment (enacting a metaphor), which is – at least partially – motivated by embodied experiences (a metaphor embedded in the environment), while at the same time being a consequence of the ecology he is in (an extension of a metaphor) (Gibbs, 2019; Jensen & Greve, 2019). Hence, considered jointly, gestural and verbal behaviour should be seen as a metaphorical performance; a performative action that is taking place in real-world (physical and cultural) ecologies that change over time (Gibbs, 2019) argues that “This ecologically ingrained view [of metaphor] is increasingly recognised as the foundation for all human thought and action… minds are now seen as embodied, enacted, embedded, and extended” (p. 43). Consequently, gestures should be seen as a phenomenon that is fully integrated into our conceptual system (e.g. Chui 2011; Gibbs, 2017; Hostetter & Alibali, 2019) and should hence be considered inseparable from speech (Cuccio & Fontana, 2017).

Studying teachers’ lecturing from this perspective consequently means putting communication into context. Consequently, metaphor and gestures in education ought to be studied in a broader way, instead of merely focusing on how the underlying conceptual metaphors structure language. Rather, it is a question of exploring how the surrounding environment either motivates or constrains speakers’ metaphor use (Gibbs, 2019; Jensen & Greve, 2019; Kok & Cienki, 2016; Müller, 2019). This infers that observing how teachers enacts metaphors related to programming concepts in a particular situation will enable us to explore how teachers’ understanding of programming concepts is embodied and extended on that occasion, i.e., the way in which they use gestures and speech, and what tangible qualities of abstract programming concepts they provide.

Methods

To meet the aim of this study, we have visited three different Swedish upper-secondary schools, in which teachers were lecturing on computer programming. To identify teachers for this study, we addressed the headmasters at three upper secondary public schools located in three different municipalities in the same region in Sweden. In the following section, we will provide a thorough description of the methodological aspect of the study. This includes an introduction to the context of the study, an overview of the data collection, our ethical considerations, and the theoretical underpinnings of the analytical approach of the study. Furthermore, the section Includes a step-by-step description of our analytical process.

The context of the study

All content addressed in the observed lessons of this study are parts of a mandatory programming course of the Technology programme in Swedish upper-secondary education; a programme that aims to prepare students for higher education in fields such as engineering or computer science. In Sweden, the syllabus gives teachers a large degree of autonomy when planning content and structure of their lessons. Hence, the observed lessons differed in terms of the learning objectives and the development software and programming language used. In general, this variance reflects the way in which Swedish upper secondary education is formally structured and operationalised. The teachers, Lennie, Robert, and Johannes, were selected based on the programmes and courses they were teaching at the time of the data collection, rather than their personal background. Hence, the programming teachers have different backgrounds in relation to educational level and prior education. Consequently, the selected teachers for this study reflect some of the variety among programming teachers in Sweden.

Lennie and his classroom

Lennie has a teaching degree in computer science and English. Furthermore, he has worked with pedagogical development at his school. He has also been involved in extracurricular activities relating to programming. Aside from teaching this course, he also teaches subjects such as web programming and project management. During the observed lesson, Lennie spends most of his lecturing time near his whiteboard, positioned in front of his class (Fig. 2). On his left, Lennie has a stationary computer placed on a rostrum. The computer screen is casted to the students’ workstations, allowing him to demonstrate his code examples while coding in real-time. This technology is used during the final part of the lecture, when the students are seated at desks positioned along the walls, having access to one stationary computer per student.

Fig. 2
figure 1

Lennie performing two different gestures: (1) extending his thumb while saying “one thing” and (2) using his thumb to “draw” what can be interpreted as a list of chores while saying “number of things”

Full size image

Johannes and his classroom

Johannes has a teaching degree in music. A large portion of his teaching is based around experiences gained when working with the development and maintenance of large software systems. During the observed lesson, Johannes spends the initial part of his lecture seated at a desk in front of a laptop computer (Fig. 4). In front of him, positioned slightly to the left, the students are seated in three rows, facing him and a projector surface on which Johannes shares his screen. Towards the end of the lesson, Johannes raises a few of the issues that the students have encountered when working with today’s assignment. This spontaneous lecture sequence is performed standing between the desk and the projector surface.

Fig. 4
figure 2

Johannes performing a motion with his hand while saying “check”. The motion can be divided into three parts (left to right) where (1) Johannes opens his hand, (2) creates a space between his thumb and index finger and, (3) constrains the area between his thumb and his index finger

Full size image

Robert and his classroom

Robert’s main experience as a programmer derives from his post-doctoral studies in chemistry. During that period, he had to write his own applications to facilitate his research. As this is his first job as a teacher, Robert only has seven months of classroom experience. Furthermore, he has not trained as a teacher. Robert spends all his lecturing time standing next to a laptop computer that has been placed on a rostrum. Connected to the computer is a projector that mirrors the screen. At times he uses the whiteboard, but his attention is mainly turned towards the code projected onto the projector surface.

Data collection

The data for this study comprises video recordings of the three teachers giving lectures on computer programming. In total, approximately 120 min of video were recorded during the lessons. Two tripod-mounted cameras were used to capture the teachers from multiple angles. The cameras were placed on each side of the classroom at an angle of approximately 45 degrees in relation to the teacher. Together, they ensured that the teacher was always in focus without interfering with the teacher’s ability to move freely at any time. The data was collected with the intention of capturing as much as possible of the natural interaction between the teacher and the students. Due to the risk of making the teachers aware of their gesturing, the teachers had not been informed about the interest of this study in advance. The only instruction they were given prior to the observation was to perform a normal lecture in front of their regular class.

Ethical considerations

Cohen et al., (2017) state that “Educational researchers must take into account the effects of the research on participants: they have a responsibility to participants to act in such a way as to preserve their dignity as human beings.” (p. 112). To fulfil this principle, we have, in accordance with the Swedish Research Council’s Good Research Practice (Vetenskapsrådet, 2017), strived to be open with all aspects of our research to both the informants of the study and the students in the classroom. We have guaranteed our informants (1) anonymity, (2) voluntary participation, and (3) the possibility to withdraw their participation during the data collection and analysis process. Based on this, all teachers and students have given their written and informed consent to participate in the study. As the premise of this study was to capture the teacher’s gestures in a naturalistic environment, we chose not to inform them about the exact aims of the study beforehand, an act that could be considered counterintuitive in relation to good research practice. However, by (1) formulating and presenting the teachers with a vague aim prior to the study, and (2) asking them to take part in a post-observation interview to discuss their behaviour in the classroom, we ensured the possibility to reveal the precise aim for this study. Following the interview, the teachers gave us their oral consent to continue with the analysis process. Consequently, we have made conscious and continuous efforts to follow both the ethical principles for good research practice and the General Data Protection Regulation (GDPR) to protect the personal data of our informants.

Data Selection and Construction

The data selection for this study was performed on three levels: a contextual level, a conceptual level and a data construction level (Fig. 1). Firstly, all video recordings from the classroom observations were watched back in real time with the aim of creating an initial understanding of the video data and the classroom context (Cienki, 2016). We notated central passages in the videos, in which programming content were in focus. Secondly, Wondershare Filmora9 was used as a tool during the gesture identification process. At first, only vivid gestures and gestures of an expected nature were identified, but the more we “got to know” the teachers’ individual gesturing style (Cienki, 2016), the more we were able to notice subtle details and nuances in gestural behaviour. At this conceptual level, gestures accompanying programming or gestures that could be inferred as important for programming concepts were selected as data for the study. Continuing, a coarse log of events was created, containing timecodes, simple descriptions of gestures and references of when specific programming concepts were being discussed. Following, crucial events were selected to bring forward for further analysis. The events were selected, partly on subject content and partly on variations in gestural behaviour among the teachers. The events were then transcribed verbatim and analysed as described below.

Fig. 1
figure 3

A description of the data selection and construction process for this study

Full size image

Analytical process

The verbal data for this study has been analysed based on an adaptation of the Metaphor Identification Procedure (MIP), a procedure originally developed by the Pragglejaz Group (2007) to provide a systematic way of identifying metaphoric language in the context of a “real-life” text corpus. However, in the case of this study, the context of a text corpus has been replaced with a classroom context and excerpts of texts have their counterparts in verbal utterances. To further structure the analysis, the study employs elements of the Metaphor Identification Guidelines for Gestures (MIG-G) (Cienki, 2016) to explore the teachers’ gestures. Underpinning both procedures are the assumptions that metaphor is not a question of “either-or”. Rather metaphor is a scalar property that varies on an individual and contextual basis (Cienki, 2016; Pragglejaz Group, 2007). Both adaptations of MIP and MIG-G follow a similar protocol to determine whether a word or a gesture is being used as a metaphor within a given context. In the case of MIP, this is based on comparing the meaning of a word in a context to the meaning of the same word in another context. For example, a computer can “crash”, but as the word “crash” has a more literal meaning in another context (e.g., a vehicle colliding violently with an obstacle or another vehicle), it can be determined as being a metaphor in a computer context. In the case of MIG-G however, identifying a gesture that is used as a metaphor is a question of identifying physical resemblances between the form and/or motion of the gesture and a corresponding word. For example, a hand shaped as a cup has physical resemblances with a cup where one can place an object, hence the gesture can be determined as a metaphor version of a container.

To demonstrate the analysis process, we provide an example from one of our classrooms. Here, Lennie is at the beginning of the programming lecture on retrieving and formatting data. Standing in front of his class, Lennie has started to introduce an analogy between the structure of a function and doing household chores (Fig. 2). He tells the students to imagine being told to clean their rooms. “Cleaning your room doesn’t mean one thing …” he says. Following a short pause, Lennie continues: “It means a number of things, right?” While speaking about what may be involved in cleaning the room, he raises his left hand to chest height, using his thumb to emphasise “one thing”. He then raises his right hand to a head-high position and extends his index finger to the left while using his left hand (thumb out) to illustrate what appear to be instructions that could be on a list of household chores.

The analysis process can now be described as follows: Firstly, MIP suggests that – as tidying one’s room has a more basic meaning in a house context compared to in a computer programming context – Lennie is using all verbal expressions relating to cleaning as metaphors for instructions in a program. Secondly, as there are physical resemblances between Lennie’s words and gesturers (raising the thumb for “one thing” and moving the thumb downwards for “a number of things”) MIG-G suggests that the gestures should be determined as being used as metaphors. Consequently, as Lennie’s words and gestures correspond with each other, it is reasonable to argue that the gestures provide additional imagery to Lennie’s speech, resulting in a verbo-gestural metaphor. Hence, this is one example where the verbal and the gestural metaphor share “the communicative burden” (Cienki & Müller, 2008).

Results and Analysis

Following is a presentation of four events that has been deemed representative of the three teachers’ metaphor performance. The examples have been chosen with the ambition of displaying the variety of ways in which the teachers use their hands in front of the class. Furthermore, events have been selected in relation to their respective programming content and also their personal gesturing style (see Cienki 2016). Consequently, our selection is based on qualitative differences in the teachers’ gestural behaviour in a naturalistic setting, rather than any quantifiable relations to programming concepts. Each example will be presented as follows: Firstly, we will provide a short presentation of the context in which the event is taking place. Secondly, we will present a short excerpt from our video-recorded observationFootnote 2, followed by a sequence of images representing how the gestures have been performed. Thirdly, we will describe the gestures in text. This will later be followed by an analysis of each section in which we will discuss our observations in relation to relevant theoretical and methodological literature on the topic.

Retrieving data

The following event is based around a sequence where Lennie is lecturing on the basics of retrieving and formatting data from a database, before publishing parts of it on a website. Lennie is using his computer during this section of the lecture (Fig. 3), meaning that the students are seated at their workstations facing away from Lennie. At the start of the sequence, Lennie is telling the students that they need to understand how the received data is structured to be able to handle it. He continues by saying “We need to do that [understand the structure] because we will actually take care of data”. During the event, Lennie performs a series of gestures that can be described as two separate parts in which (1) Lennie forms his right hand into a cup-like shape (Fig. 3, left-most image) and (2) lowers and closes his hand (Fig. 3, middle and right-most image).

Fig. 3
figure 4

Lennie performing a motion with his hand while saying “take care of data”. The motion can be divided into three parts (left to right) where (1) Lennie opens his hand, (2) lowers his hand while “grabbing” data and, (3) raises his hand while closing it

Full size image

“Take care of data”, in Swedish, literally corresponds to “take your hand around the data” or “grab hold of the data”. Hence, MIP suggests that – as it is impossible to literally grab, nor hold data – the verbal expression is here to be determined used as metaphors. Furthermore, MIG-G suggests – as this passage reveals a strong physical resemblance between Lennie’s speech and his corresponding gestures – that his gesture sequence is to be considered as being used as a metaphor. Consequently, we argue that this is an event where Lennie is enacting the whole experience of retrieving an object and placing a hand around it, all in synchronicity with speech. Therefore, the inherently abstract concept “data” becomes something that one can “hold on to” and possibly also shape, move, split and so on. This indicates that Lennie’s metaphoric actions are motivated and simulated in relation to his own experiences of catching and holding objects and that he – at least in this event – conceptualises data as a physical entity that can be grabbed and moved from one place to another (Cuccio & Fontana, 2017; Gibbs, 2006, 2019). From this we argue that data, in this event is based around verbal and gestural versions of the conceptual metaphor DATA IS A PHYSICAL OBJECT.

Validating data

In our second example, Johannes has started to introduce the assignment of the day by discussing to the issue of validating data. He instructs the students to “choose a method and check there isn’t anything…”. At the same time, Johannes starts lowering, while at the same time opening his hand (Fig. 4, left-most image), and later landing it slightly to his right hand with his thumb and index finger placed in such way that they form a confined area (Fig. 4, middle and right-most image). Suddenly, Johannes hesitates as if he wants to use another set of words, as he says “that the text field is empty in” while making a circular motion, landing his hand chest high, in front of his body. (Fig. 5).

Fig. 5
figure 5

Johannes performing a chest high, forward gesture, beginning with an open hand, and ending in a pinching pose. The gesture is performed in proximity to his speech, however, not in synchronicity

Full size image

As in our first event, this example is built around handling data. In this case, MIG-G suggests that Johannes – seemingly holding data in his hand while later opening his hand and putting it on display – is enacting a gestural version of a text-field in one of two ways: either (1) the open hand is – by physical resemblance referring to a visual representation of a part of the screen (Fig. 4, middle image) or (2) the whole sequence of hand movements is a gestural version of looking at metaphoric data (DATA IS A PHYSICAL OBJECT). Therefore, MIG-G is not able to provide a conclusive interpretation of Johannes’s gestures.

In the second stage of this event (Fig. 5) MIP suggests that the expression “text field” – as the word field generally refers to a (typically bounded) piece of land – is used as a metaphor for an “area” in which to place data. Hence, we argue that validating data, in this event, is partly based on Johannes’s experiences of examining either a part of a computer screen or physical objects. Johannes’s use of the expression “text field” implies – as metaphorical objects can be placed in metaphorical fields – that this is an event that reflects Johannes looking at data that is within a bounded region, hence it is a verbo-gestural conceptual metaphor A TEXT FIELD IS A BOUNDED REGION with DATA IS A PHYSICAL OBJECT.

For-loop

In our next example, Robert is speaking about how to design a for-loop as part of a larger program designed to solve a mathematical problem. The objective of said loop is to investigate the number of characters in a string variable and return that number for later use. While speaking, Robert is projecting his computer screen showing his own solution to the problem onto the whiteboard, enabling him to walk the students through his own code, step-by-step. On one occasion he says: “I have placed a for-loop here […] that should loop through…”.

In this short event, the term “loop” is spoken about in two ways, firstly as a noun and secondly as a verb (in Swedish: “loop” and “loopa”). Hence, judging from Robert’s grammar, a loop is either an “object” that one can place somewhere in a program or an activity that can be performed by an actor, for example a computer or a computer program. Robert’s corresponding gestures are both similar in gesture-form (circular) and in height (chest height) but are performed in different directions; while speaking about the loop as an object, Robert gestures inwards (Fig. 6), and while speaking about loop as a process, he gestures in an outward motion (Fig. 7). We suggest that this example reflects an event that two types of visuo-spatial metaphors with two different functions can either (1) provide the loop with the properties of an object that can be placed into the program, or (2) serve to describe an ongoing process, where the verb “to loop” gets a visuo-spatial dimension from the gesture (see Kok & Cienki 2016). Consequently, based on a combination of MIP and MIG-G, it is reasonable to state that Robert’s gestures act as “grammatical ques” that can set two homonymic computing concepts apart.

Fig. 6
figure 6

Robert performing an outwards circular gesture while saying “for-loop here”

Full size image
Fig. 7
figure 7

Robert performing an inwards circular gesture while saying “loop through”

Full size image

The program will move on

Our final example concerns Johannes, sitting at his desk, explaining the purpose of a specific command in the programming language Swift. During the lesson the students have been working on validating data, and they are now supposed to learn how to get the program to continue working after the validation process has been completed. Johannes tells the students that “one.next will make the program move on [Swedish: hoppa vidare.]”.

In Swedish, the expression “move on” corresponds with “jump further”, and as in our first example, this is an event that shows a strong correlation between Johannes’ speech and gestures. MIP suggests that the word “jump” in this context is used as a metaphor, since the more literal meaning refers to when we use our legs and move our body upwards and/or forward. Moreover, MIG-G suggests that the gesture is used as a metaphor for the computer program “moving on”. The jumping gesture (Fig. 8) ads a spatial dimension to his verbal “jump” metaphor, hence this is yet another example of a verbo-spatial programming metaphor that indicates that Johannes is speaking about a process (see Author-a). The direction of the metaphor indicates that Johannes is structuring the execution of a computer program in a sagittal motion (a horizontal motion outwards from his body) rather than the vertical code lines that are present on the computer screen. We interpret this as Johannes sees himself as being a part of, or at least being able to follow the program, i.e., embodying the code (Gibbs, 2019).

Fig. 8
figure 8

Johannes performing a motion with his hand while saying “move on”. The gesture is performed in a sagittal direction

Full size image

Discussion

The result of this study reveals that the observed teachers employ gestures in two ways: (1) to provide imagery to a verbal utterance to form verbo-gestural metaphors or (2) to add spatial properties to a verbal metaphorical or non-metaphorical utterance to form a verbo-spatial metaphor. These phenomena have been extensively described in previous literature by scholars such as McNeill (2008) and Cienki & Müller (2008) but have not to our knowledge been studied in a naturalistic classroom environment. Our three teachers have developed a plethora of ways to communicate programming and are thus able to adapt their metaphor performance to the situation and to the environment (e.g., Gibbs 2019).

The result of this study reveals one commonly enacted verbo-gestural metaphor, DATA IS A PHYSICAL OBJECT, where the teachers’ gestures generally depict holding something in their hands. This in turn will enable the teachers to elaborate on the concepts further by, for example, grabbing hold of data, moving it from one place to another or opening a containment to observe data. It will also, as in the case of Johannes, provide conceptual grounds for jumping from one place to another. These types of gestures are well in line with gestures identified by Manches et al., (2020), in which students tended to employ pinching or weighing gestures while discussing data in an interview situation. Furthermore, our results support Pérez-Marín et al., (2020) and Chibaya (2019) in the sense that teachers’ metaphor performance plays a big part in their teaching practice.

One example of special interest would be when Lennie is “taking care of [received] data”, where we are able – with a high degree of confidence – to suggest that his metaphoric actions are based around a physical object that he is able to catch and alter its form. Consequently, data – at least to Lennie – is not any object. It has specific properties which, in turn, are grounded in Lennie’s unique experience of interacting with objects in a physical world. It could be argued, however, that data – a well-established technical term in programming – should be considered non-metaphoric (Hanks, 2006). Still, based on our complementary analysis of both speech and gestures, we argue that – as all teachers enact data as being a concrete object rather than electrons in a box – the term should be treated analytically as an important programming metaphor.

In this study, we have also identified events where gestures used as metaphors provide spatiality for verbal utterances. This has made it possible for Johannes to set a computer program in motion, Robert to provide an imaginary space in which to place a for-loop, and for Lennie to use his hand to grab data from a database. Hence, it is fair to say that spatiality is a central aspect of computer programming. However, judging from the results of this study, it is not the gestures alone that that enhance and emphasise these aspects of programming; it is the gestures in combination with the speech. Similar results have been reported by for example Manches et al., (2020), stating that computing processed tends to be conceptualised in terms of motion, and Larsson, Stolpe & Johansson Falk 2021 stating that sagittal gestures may reflect the function of code rather than the representation on the screen.

In relation to verbo-spatial metaphors, the results of this study reveal situations where the teachers’ gestures are used to add spatial properties to spoken concepts. Here, the gesture can serve one of two purposes: (1) to set up imaginary spaces or planes in which the teacher can “place” objects or point to specific places, or (2) to enact metaphors related to a process or a procedure. Moreover, these types of gestures may occur in close proximity to each other, where the first gesture “sets the stage” for another (Cienki & Müller, 2008). It would be possible to place a concept in a room, as Johannes does in our example of the text field.

Closely related to the spatiality of a computer program is the issues of agency. As previously mentioned, Johannes gestures a pathway on which his program will “move on”. In this enactment of the metaphor, his hand is positioned inside the program, and hence has agency in this verbo-spatial metaphor. By contrast, as seen in the function example where Johannes merely “check” the state of a text field, his hand has no agency at all. As we see it, this is what separates gestures corresponding to a process (enacting the metaphor as if one were the agent within the metaphor) and those corresponding to a position within the code (employing a metaphor in reference to another agent, e.g., a part of the code). These results are in line with what has previously been described by Larsson, In press in a study concerning written language.

The result of this study also reveals one example where separate metaphors expressed in gestures and speech are combined to conceptualise a complex programming concept: the validation of data. We argue that Johannes, by combining the metaphors A TEXT FIELD IS A BOUNDED REGION and DATA IS A PHYSICAL OBJECT, provides the student with a conceptual ground based on space and object. Similar combinations of metaphors have been described extensively in previous literature (e.g., Grady 1997; Lakoff & Johnson, 1999; Dreyfus et al., 2015).

Even though it is complicated to present a general and conclusive analysis of how programming concepts are structured in the classroom, we argue that it is possible to accumulate enough information to make well informed suggestions about what might experientially motivate the structure of a programming concept or, for that matter, an entire lecture. We do so by employing established methodologies for identifying metaphors and come to conclusions based on empirical findings and theoretical literature from research areas such as Cognitive Linguistics and cognitive grammar. We consider gesture and speech as equal components of how we structure computing concepts. Hence, the key component in our approach to real life metaphors is our close attention to the relationships between gesture and speech. In that sense, this study adds to earlier attempts at using metaphor theory and gesture analyses in STEM education research (Alibali & Nathan, 2012; Dreyfus et al., 2015; Manches et al., 2020).

Conclusions and Implications

This study aims to explore how teachers’ gestures correspond to programming concepts and what impact this has on how these concepts are structured. In response to this aim, we have analysed lecture sequences with respect to verbal and gestural metaphors. Based on the observations in this study, we conclude that data should be considered one of the most graspable aspects of programming, despite its metaphorical origin. All three of our observed teachers structure data – an abstract concept that can be neither seen nor manipulated by the use of a hand – as a physical object that can be moved, thrown, returned, sent, manipulated, etc., even though they use both metaphoric and non-metaphoric expressions. In order to structure data as graspable and movable objects, the teachers need to somehow structure a scene in which to move the data. From the results of this study, we conclude that this is done by utilising metaphorical gestures. Furthermore, such gestures will be able to provide a metaphor with agency, making it possible to enact computer-related metaphors using the body as an educational resource. This suggests that gestures play a pivotal role in teachers’ enactment of abstract, yet conventionalised, programming concepts. This also implies that gesture can provide students with the underlying structures of an established technical term even if the teacher does not provide it to them explicitly, and as data is but one of many conventionalised concepts in computer science, this type of research may bring clarity to the key aspects of programming.

One limitation of this study is that it cannot predict whether the metaphors at hand contribute to or facilitate students’ learning in the programming classroom. To further understand the way Lennie, Johannes and Robert teach would require research that follows the interaction between teachers and their students over time. This would reveal whether and how the students respond to their teachers’ gestures, and whether they themselves adopt the teachers’ gesturing styles and metaphors. Yet another aspect that needs to be accounted for is questions regarding students’ perspectives in relation to the teacher. Within the context presented in this study, the students see the teacher in a mirror-inverted manner. This means that the students must take on the teacher’s perspective in order to interpret gestures correctly. However, in a setting where the teacher is sitting next to the student, such a perspective taking does not exist. Future research on such phenomena could increase the accumulated knowledge regarding the topic and would also develop new methodological approaches within education research. Such research could also affect for example the design of educational technology, or the way code is communicated in text-books or represented on the computer screen.

In summary, the study shows that analysing gestures and corresponding speech is a powerful analytical tool with which to explore how programming concepts are conceptualised and communicated. Furthermore, as gestures and metaphors, on a conceptual level, can be seen as enactments of a teacher’s personal knowledge and beliefs (Larsson & Stolpe, 2022), we suggest that this type of methodological approach could serve as a way to highlight critical aspects of a teacher’s pedagogical content knowledge (PCK). The result of this study shows the delicate work of trying to elucidate what is metaphor and what is not (see also Cienki 2016; Gibbs, 2017; Gibbs & Colston, 2012). In line with scholars (e.g., Gibbs 2019; Müller, 2008), we infer that this partly has to do with the qualitative property of metaphor, but also, what is metaphor differs from person to person and situation to situation. To address this issue, this study has employed a combination of two established methodologies, MIP and MIG-G, to increase the accuracy and transparency of our analysis. As our results indicate, the MIP and MIG-G complement each other in taking both verbal language and gestures into account. Moreover, we can also give plausible explanatory models for how these structures are being experientially motivated (Gibbs, 2019; Müller, 2019). Work to develop and refine such methodology is currently in progress (Larsson et al., 2022). Moreover, we argue that the results reported indicate that the teacher’s gestural behaviour may be affected by the (technological) artefacts surrounding them. Altogether, this shows that investigating teachers’ full-body interaction in relation to the affordances of the surrounding environment in will add valuable knowledge to the field of educational research as well as cognitive linguistics.