Introduction

Education researchers have argued that students’ development of representational competence is central to achieving conceptual understanding and science literacy (diSessa, 2004; Lemke, 2003, 2004; Prain & Tytler, 2012). In line with Lemke (2003), we use the term representational competence as the ability to construct and interpret links between an object (a real-world object, a concept, a quantity, a process or a phenomenon), its representation and its meaning.

Research on representational competence highlights the role of multiple representations in student learning processes. Learning with multiple representations encompasses using simultaneously two or more representational forms of the same concept, such as a text about a specific concept which is accompanied by pictures of that concept (Ainsworth, 2006). According to Opfermann et al. (2017), when students are offered different sources of information, they can choose the one that best fits them and the requirements of the provided task. In addition, when the information is packed into a representation taking advantage of several different modes (words, symbols or images on a page, moving images or sound on a screen, gestures, etc.), the representation is multimodal, in the sense that it may ‘involve the simultaneous coordination of any or all of the senses and any modes of action’ (Hutchins, 2010, p. 434).

Education researchers suggest that students need to participate in activities where they learn how to construct and interpret disciplinary-specific representations (Ainsworth, 2008a, 2008b; Prain & Tytler, 2012), while they should, in turn, be given the opportunity to learn about and with new representations other than the sanctioned representations asserted in standard educational material (diSessa, 2004). Students’ ability to generate their own representations of phenomena and concepts, so-called student-generated representations, is an important aspect of representational competence. According to Prain and Tytler (2012), students’ construction of their own representations allows them to focus on the purpose and relevance of representations in particular contexts and facilitates their understanding of relationships between objects, their representations and their meaning. In contrast to such studies where student-generated representations are explicitly prompted by the teachers, Steier et al. (2019) argue that cases where students spontaneously generate representations have received insufficient attention in science education research. They have found that students often come up with new representations in a collaborative environment and that the development of unprompted representations is part of the process of making sense of a physical concept or phenomenon.

A particular type of student-generated representation is the use of gestures and enactment. Roth (2000), drawing on Kendon (1985), defines gesture as a visible bodily action used in the process of communication. Gestures can add more layers to representational competence than plain speech, playing an important role during the process of understanding the relationships between an object, its representation and its meaning during construction of a scientific discourse (Roth, 2000; Roth & Welzel, 2001). This perspective can be extended from a focus on gestures to multimodal forms of communication where students interact physically with each other, add sounds or make use of objects from their vicinity to convey their ideas. By coordinating such representational forms, students may produce something similar to a theatre scene, that is an enactment, where they play out different aspects of a physical phenomenon or concept (McNeill, 1992; Scherr, 2008).

Computers and other digital devices have long been acknowledged as useful tools for constructing representations (diSessa, 2004) and to support students’ cooperation (Steier et al., 2019). As digital devices become more powerful, they can be used to link multiple representations using the same or different modes, creating more sophisticated and complex representations (White & Pea, 2011). Research on students’ learning processes in digital-based learning environments often explores students’ interaction with digital interfaces and with each other and shows that students put considerable interpretive effort in order to make sense of involved concepts and representations (Stahl, 2006).

The context of the present study is a collaborative activity of upper-secondary school students who are introduced to a computer simulation of the concept of friction. Research shows that microscopic factors (e.g. surface topography), rather than macroscopic factors (e.g. mass), can provide a causal mechanism of friction for students (Cheong et al., 2019).

The study aims to explore students’ use and interpretation of a provided representation, a computer simulation of friction, and generation of their own representations, with a particular focus on their gestures and enactment of the phenomenon. Against this background, we address the following research questions:

  1. 1.

    How do pairs of students use and explore a provided representation to make sense of microscopic aspects of friction?

  2. 2.

    What triggers pairs of students to come up with their own representations when making sense of a computer simulation of friction?

Gestures and Enactment as Student-Generated Representations

In this section, we expand the account of students’ gestures and enactment in relation to scientific phenomena from two angles. We consider student-generated gestural representations as a means of dealing with interpretational problems, but also recognise that exploring students’ gestures and enactment can be used to analyse their sensemaking processes.

An important contribution to psychology research on the interplay between spoken language and gestures stems from research by McNeill (1992). He argues that gestures are not only an accessory to speech, but also have an impact on thinking. The idea that ‘gestures and speech convey information of the same scene’ (McNeill, 1992, p. 79) has influenced the work of several science education researchers. Studies on students’ use of gestures while interpreting the relationship between an object’s representation and its meaning have focused on (a) static representations, such as drawings and maps (Roth, 2000; Scherr, 2008; Steier et al., 2019), (b) computer-based dynamic representations, such as dynamic drawings and simulations (Adams, 2008; Euler et al., 2019; Roth, 2000) and (c) physical models, such as a wooden model of a bridge (Roth, 2000; Roth & Welzel, 2001). These studies show different aspects of gestures that play an important role in communication and thinking. For example, Scherr (2008) examined students’ use of gestures while learning projectile motion in mechanics. When students were discussing the velocity of an object at the top of its trajectory, one of them moved her left hand in a half parabola shape, stopped the hand at the vertex and then let the hand drop straight down to illustrate the path of the object in the case of zero velocity at the top of the trajectory. The half parabola gesture was accompanied by incomplete and quiet sentences twice before the students actually spoke out loud and made her ideas publicly accessible through both gestures and speech. This led Scherr to conclude that students’ gestures indicate their pre-articulated ideas and facilitate construction of new ideas offering ‘one source of evidence of students’ engagement in constructive thinking’ (Scherr, 2008, p. 4). Euler et al. (2019) present a case study involving a pair of students as they reason about binary star dynamics. At one point, the students act out a rotation of a partner dance, constructing an explanatory model of the binary star system. In the process of meaning making, the partner dance provides shared embodied images which can persist long enough to serve as a hub around which further dialogue can take place. These studies suggest that there is a growing interest in science education in the processes through which students construct unprompted representations in the context of collaboration, and a common finding of these studies is the shift across modalities that arise during the interpretation process. Students were observed to frequently move between digital and non-digital representations, between static and dynamic representations and between spoken language and gestures when trying to make sense of the provided representation.

In addition to viewing student-generated gestural representations as a means of dealing with interpretational problems, some of the above studies also use students’ gestures and enactment as tools for analysing their sensemaking processes. For example, in the study conducted by Steier et al. (2019), groups of students were trying to understand that the shortest flight path may be represented by a curved line on a global map. To understand this, a student constructed a new representation of the motion between two locations. He shaped his left hand into a half-sphere tracing a possible trajectory of the flight path using his right index finger. Steier et al. (2019) argue that when students have difficulty expressing ideas within the given context, they may look for new ways to make their understanding available to collaborators, such as improvising gestural representations. The scholars also suggest that students’ improvised gestural representations can be used as analytic concepts for researchers to productively unfold meaning-making processes within collaborative learning environments. This suggestion confirms Scherr’s (2008) view that studying an individual student’s gestures in a group activity provides insights into his or her unique contributions to the group’s generation of ideas. Also, in the context of students using computer-based representations, the analysis of gestures can be a powerful tool for untangling student understanding through digital representations (Adams, 2008). These studies also suggest a growing interest in the students’ construction of gestural representations and enactment during collaborative meaning making processes of representations. In this vein, the present study contributes to our understanding of students’ development of representational competence by examining how they generate unprompted representations, in particular gestures and enactments, in interpreting the links between a physical phenomenon and its provided digital representation in a collaborative learning environment.

Learning with and about Digital Simulations

Drawing on Smetana and Bell (2012) and Clark et al. (2009), we define computer simulations as computer-generated, dynamic representations of real-world phenomena or idealised models that allow users to explore the implications of manipulating or modifying parameters within the representation. Simulations are dynamic representations in that they ‘display processes that change with respect to time’ (Ainsworth & VanLabeke, 2004, p. 241). For example, they might display a block moving on a table or electrons moving around an atomic nucleus. In addition, simulations often include multiple representations, such as images, graphs and written captions, which are usually displayed simultaneously (Ainsworth & VanLabeke, 2004).

Digital simulations may be used for instructional purposes as they ‘allow learners to construct their own knowledge by interacting with an environment, conducting experiments and by observing the effects of these experiments’ (Ainsworth & VanLabeke, 2004, p. 243), in ways that are sometimes impossible in real situations. A review study, conducted by De Jong et al. (2013) to compare the value of physical and virtual investigations, shows that research does not support the belief that students learn best from working with physical equipment. Instead, the review study has found that, in much of the current literature, students working with simulations perform as well, or better, on measures of conceptual understanding than their peers who perform similar experiments with physical equipment. A defining characteristic of virtual laboratories is that the reality can be modified to make the interpretation of the studied phenomena easier. For example, important information in a digital representation can be accentuated while confusing details can be concealed. Digital representations often provide simultaneous access to the formal representations of physical concepts and phenomena that would not be perceivable in the same way as they would be while using physical equipment. In addition, virtual experiments enable students to conduct multiple experiments in a short amount of time. The scholars also note that several studies emphasise the benefit of teacher guidance, particularly when students are asked to predict the outcome of an experiment and to later compare the outcome with their own ideas. Students may, however, still encounter difficulties interpreting outcomes despite using carefully designed simulations. For example, a study conducted by López and Pintó (2017) revealed that 14- to 16-year-old students encountered several types of reading difficulties when interacting with a simulation of friction. The simulation displays a chemistry book sliding on top of a physics book with a different colour. Some students were confused about the simulation, as they associated the titles and colours of the books to particular physical vs. chemical properties of friction. López and Pintó argue that the observed interpretation difficulties are caused by an overload of information, ambiguous information and unnecessary, decorative elements and visual inaccuracies.

Some simulation environments, such as Physics Education Technology (PhET) simulations, are fixed environments in which students learn by exploring the implications of manipulating or modifying parameters (Wieman et al., 2008). Other platforms, such as Algodoo (Gregorcic & Bodin, 2017), allow users to create and modify their own simulations. Recently, GeoGebra (Hohenwarter et al., 2009), an open source software originally designed for mathematics education, has received some attention in physics education (Milner-Bolotin, 2016; Walsh, 2017), as it enables teachers without in-depth programming knowledge to create their own computer simulations. Different representations of the same mathematical object, e.g. formulas, tables and graphs, are connected in a dynamic and simultaneous manner. GeoGebra allows teachers to create engaging learning environments which enhance students’ cooperative learning (Wassie & Zergaw, 2018) or to create mathematical models of physical phenomena and hence to achieve their pedagogical goals (Milner-Bolotin, 2016). GeoGebra can also be used by students to create mathematical models of physical phenomena which helps them understand in depth the process of modelling and the modelled physical phenomena (Marciuc et al., 2016). In this way, research suggests that digital simulations can be effectively used in physics education. Moreover, GeoGebra allows users to adjust simulations to avoid unnecessary information or to insert desired elements.

Methods

Participants and Data Collection

The study was conducted in a class of 19 first-year upper-secondary school students (16 years old) specialising in natural science in Sweden. They took part in a 1-h lesson following the written instructions provided by the authors. This lesson was not part of the students’ standard curriculum, but was given during school time by the researchers as replacement for the regular teacher. Written informed consent was obtained from all students, and six of the students (three pairs) consented to the video recording of their participation. Both authors (Solvang:A1, Haglund:A2) remained present throughout the entire lesson, offering advice on how to use the software and how to interpret the questions of the task when the students were stuck. The authors also encouraged the students to dig further into interesting observations. The video cameras were placed behind each of the three pairs to record the students’ voices and gestures and their computer screens. In addition, students’ unprompted representations in the form of drawings of the phenomena were collected and analysed.

The Task

Because the students were at the very beginning of their first upper-secondary physics course, we chose a topic usually included in secondary school physics, namely, friction between two solid objects. We used the example of making an object slide across a plane surface by applying an external force on the object. The frictional force has two forms: static friction (when the external force is applied but the object does not move) and dynamic friction (when the object is in motion). The dynamic friction is independent of the contact area between the object and the surface, but is proportional to the normal force, and hence to the mass of the sliding object. The friction is caused by the adherence between the surface molecules of the two objects and the microscopic structure, which explains the dependence of friction on the materials of the surfaces. However, textbooks only briefly mention microscopic aspects, such as the role of surface roughness on frictional force. According to Besson et al. (2007), the standard representation of friction, which depicts a rectangle moving on a plane, hinders students from imagining the underlying microscopic phenomena. Since microscopic observations of surface topography are not easily accessed through direct observation, Trundle and Bell (2010) recommend using computer simulations. For the present study, we chose a readily available GeoGebra simulation of friction which is a dynamic representation of a block sliding over a surface. A hand pulls the block with a dynamometer, which shows the value of the pulling force. Simultaneously, a force–time diagram is displayed as shown in Fig. 1.

Fig. 1
figure 1

GeoGebra simulation of friction used in the study. Overview of picture of block and hand, sliders and graph after the block has started to move (freely available at https://www.geogebra.org/m/F6GAjhpw)

Full size image

By dragging the sliders (the top left corner in Fig. 1), students can choose the materials of the block and the surface, and the value of the block’s weight and base area. Using the buttons at bottom left, students can start and pause the simulation, or reset the graph.

Around this simulation, we developed a set of instructions for students to use together with the GeoGebra simulation. The set of instructions was designed, according to the Predict–Observe–Explain (POE) teaching strategy (White & Gunstone, 1992), to encourage students to explore, justify and eventually revise their own ideas, and enable our analysis of the range of students’ ideas. The instructions prompt students to change one parameter at a time, to predict what the graph would look like, to run the simulation and finally, to discuss the differences between their predicted graph and the one displayed in the GeoGebra simulation. The full version of the classroom instructions is available as an electronic supplementary material (Appendix 2).

The purpose of this task was to enable students to observe and explain:

  • that the weight and the material, but not the base area, influence the frictional force,

  • the decrease in force when the block begins to move, due to the lower dynamic friction compared to the static friction (a ‘shark tooth’, see Fig. 2a), and

  • ripples in the graph (Fig. 2b), due to unevenness of surfaces obstructing the motion of the block.

Fig. 2
figure 2

Force–time graph from the GeoGebra simulation of friction, displaying the ‘shark tooth’ (a) and ripples (b)

Full size image

Analytic Approach

The entire video collection was scrutinised several times by the authors both separately and together. Specific events were then selected for further analysis. The methodological approach for selection is consistent with the guidance on video analysis in educational research provided by Derry et al. (2010) that researchers ‘may look across a vast array of video records to find the most representative instance or instances—perhaps the most salient video chunks that best illustrate and represent one day or month of data collection’ (Derry et al., 2010, p. 12). Our first consideration was about the degree of explicitness of the participants’ verbal and gestural contributions (Linell, 2009; Mercer, 2007). One pair of students was excluded from the analysis, due to a low level of cooperation and poor sound quality of the recording. From the other two pairs, we selected events where the participants were highly involved in verbal and gestural interactions, that is, a high degree of explicitness. The second consideration was about the role of the microscopic aspects in understanding the concept of friction (Cheong et al., 2019). We have selected for further analysis episodes where students engaged in explorations of the ripples and the ‘shark tooth’ displayed on the graph. The discontinuous graph has an additional significance for our study as it is not a common way of presenting the relationship between the applied force and time. Because of the ripple on the graph and the dynamics of the simulation, we considered this GeoGebra simulation as a non-sanctioned representation of friction (diSessa, 2004).

To bring insight into how students understand the representation, we relied on previous research on students’ representational competence which shows that students’ interaction with a provided representation and with each other often include gesture and enactment (Steier et al., 2019). Using McNeill’s (1992) categorisation of different types of gestures, we examined depictive gestures (gestures where hands are used to produce pictorial representations of an object or a phenomenon) and deictic gestures (pointing). Here, students primarily pointed to features of the simulation on the screen, and used depictive gestures in representing friction as a phenomenon. Because these bodily actions are essential in our analysis, we include images of students’ gestures and enactment taken as screen shots of the video material. We chose to use graphic vignettes (Laurier, 2014) because the narrative structure of a comic strip enables depiction of speech, in conjunction with sound, gestures, motion and objects. To connect the images with the corresponding verbal transcript, we employed a transcript convention based on Jefferson (2004) as shown in Appendix 1.

Results

Reviewing the total corpus of video data, we identified three events where two pairs of students explored the microscopic level of friction by means of gestural representations and enactment of the studied phenomenon, in conjunction with spoken language.

Anna and Bella: Observing the Ripples

This event takes place during the part of the activity where Anna, sitting on the left side, and Bella, sitting on the right side, explore the role of the material (all students have been given fictitious names). The students are discussing the differences between the graph they predicted and the one in the simulation. They notice the ripples on the graph, approaching in this way the microscopic level of friction (Fig. 3).

Fig. 3
figure 3

Anna sitting on the left and Bella sitting on the right side observe the ripples in the graph

Full size image

Anna and Bella interact with the interface by pointing at the ripples, directing attention to that specific feature using deictic gestures. Bella moves her attention away for a while; then, she goes back to the ripples, pointing at another region of the graph where the ripples were visible. At this point, Anna seems puzzled about the graph’s ripples, as the surface is seemingly plane. To reinforce her statement, Anna gestures a flat surface, moving her hand horizontally back and forth. Here, Anna begins to realise the relation between the digital representation (the graph’s ripples) and the real world (the material’s topography). The physical properties of materials have become the focus of Bella’s attention as well. When Bella wonders whether the materials’ topography is connected to the graph’s shape in some way, A1 encourages the pair to explore the effect of different materials.

Next, Bella restarts the simulation and rejects the idea of the graph being ripply for other materials. She changes the material to wood on wood without resetting the graph and the new graph appears instantaneously. By switching to a static representation, she enables exploration of fewer aspects of the representation in more detail. The shape of the graph remains the same, but the slope increases and the ripples become slightly more visible. Bella still does not distinguish the ripples, but Anna adds that there are ‘some (ripples)’ and gestures a sine wave, indicating that the graph is nonlinear. Anna’s new representational strategy of depicting the graph with a schematic gesture helps Bella grasp the association between friction and ripples. When she sees Anna’s hand moving up and down, Bella says that ‘(When) the friction is getting higher, then you get more ripples’ taking a step towards a microscopic understanding of friction.

Anna and Bella: Explaining the Ripples

After exploring the role played by the material and the mass, Anna and Bella explore the influence of the block’s base area on the frictional force. The students begin by predicting that an increased base area will result in an increased force. They run the simulation several times, and are surprised to notice that their hypothesis is wrong. They first agree that it must be something else which makes all the graphs look the same, and they change first the weight of the block, and then the materials without resetting the previous graphs. Anna and Bella study carefully the graphs pointing all the time at the features they talk about. They present several hypotheses and conclude that a larger base area should increase friction. Then, A1 asks them to reason about how friction occurs and why the friction on ice was lower than that on other materials. Anna’s answer is that ‘(The ice) is flat’ and she accompanies her answer with depictive gestures, as shown in Fig. 4.

Fig. 4
figure 4

Anna sitting on the left and Bella sitting on the right side represent the topography of materials

Full size image

Anna moves her palms parallel to each other to show that the ice is flat, using this time both hands as opposed to the first event where she only used one hand. By doing so, Anna embodies both of the objects that take part in friction (the block and the platform) by using a depictive gesture. As soon as both of the objects become part of the representation, Bella generates a representation of a counter-example, also using both hands. She gestures rough materials getting stuck on each other by intertwining her fingers and refers to the irregularities of wood’s surface as ‘chops’. Then, both students almost simultaneously say that ‘they get stuck on each other’. To expand the explanation, Bella takes a pen that happened to lie on the table and moves it along the table saying ‘So, if you pull it like this’. It seems that this enactment is not entirely satisfying, so Bella continues by using her other hand enacting an intermittent movement, something like a jumping frog accompanied by the sound of a choo-choo train. In their written answer, the students made yet another representation (Fig. 5) of the microscopic properties of materials, drawing wood (‘trä’ in Swedish) which has a rougher surface than ice (‘is’ in Swedish).

Fig. 5
figure 5

Drawing the microscopic structure of two pairs of objects made of wood (left) and ice (right)

Full size image

By using yet another representation, the students communicate their understanding in multiple ways, recognising the advantages of displaying the information in different modes.

Diana and Mark: Explaining the Ripples

Towards the end of the task, the students were asked to write comments on possible unexplained observations. We here follow another pair of students, Diana and Mark. First, Diana changes the materials several times and then she mentions again the ripples, and that they are more visible on specific materials. Then, the students take turns to handle the computer, changing the parameters without resetting the previous graph (Fig. 6).

Fig. 6
figure 6

Diana, sitting on the right side, represents the topography of materials

Full size image

Similar to Anna and Bella, Diana and Mark employ the same strategy of using depictive gestures to represent only the objects that take part in friction (the block and the platform). With less details to examine, the students can now focus on the relationships materials-friction, weight-friction and base area-friction. Both Diana and Mark struggle for a while to formulate their thoughts, and when Diana wants to say that the graph is less ripply on a smooth surface like the ice’s surface, she uses depictive gestures instead of words. She then uses both hands to give a counterexample of rougher surfaces, clenching her hands to represent the protuberances on a rough material. Again, we can see the similarity with the first pair, who also goes from one-hand representation to two-hand representation of the surfaces. When Diana then represents the rough materials using half-clenched hands, she seems to realise that it is more difficult to get the hands to move in opposite directions and she is now able to express her ideas using an analogy with teeth, stating that ‘every time they [the surfaces] fold on each other, it looks like some kind of teeth… that make them [the surfaces] get stuck’. The conversation between Diana and Mark continues for a while around the image of Diana’s half-clenched hands. Mark describes the microscopic structure of the material as being made of ‘peaks and valleys’, that those valleys act as obstructions for the motion and that more force is needed to get ‘the peaks’ out from this obstructing ‘valleys’. Diana adds at the end that ‘The force goes in waves, more or less, depending on the material’. The students’ explanation that a higher force is required to get the objects to continue the motion, when it is restricted by the surfaces’ asperities, shows that they understand the mechanism that causes friction.

Discussion

To summarise our findings and discuss them in relation to prior research on students’ representational competence, we revisit our research questions.

How Do Pairs of Students Use and Explore a Simulation to Make Sense of Microscopic Aspects of Friction?

The students quickly came to understand how to use and interpret the features of the simulation, such as sliders and buttons. One of the reasons may be the absence of overloading or ambiguous information, decorative elements or visual inaccuracies (López & Pintó, 2017).

Like many other simulation software, GeoGebra allows users to explore the implications of manipulating or modifying parameters in ways that could not have been studied empirically using traditional tools, such as hans-on laboratory work or static images (Milner-Bolotin, 2016). In line with Milner-Bolotin (2016), we found that by using the dynamic software which simultaneously displayed the force–time graph as the parameters were changed, the students could examine several alternatives in a short period of time. In addition, the students came up with a new strategy, that of changing the parameters without resetting the previous graph. This innovative strategy proved to be an effective way of exploring and identifying the relationship between friction and the influencing parameters by allowing the students to focus only on the aspects of the representation they assumed to be important. They came up with it by themselves, showing that learning about a representation can be effectively used to learn with it (diSessa, 2004). This seemed to have prompted the students to ask and answer several and more sophisticated what-if and why questions that would have remained hidden or memorised using traditional learning environments (Milner-Bolotin, 2014). For example, Anna and Bella wondered how time influences the frictional force. After exploring the graphs for a few minutes, they figured out that the time it takes to get the block moving remains unchanged in this simulation.

Neither of the video-recorded pairs predicted the ripples on the graph, a feature associated with microscopic aspects of friction, and they put considerable effort into interpreting this feature. The ripply appearance, as well as the fact that the base area did not influence friction, was, for students, unexpected elements of the digital representation. The conflict with their earlier prediction prompted students to revise their initial ideas (Tao & Gunstone, 1997), a process during which students did further investigation to answer the why questions. Both pairs of students were rewarded with the experience of ‘Aha’! moments (Milner-Bolotin, 2016), as they were able to explain that friction occurs due to interactions between the opposing protuberances of sliding surfaces. In this respect, this GeoGebra simulation provided a causal mechanism for friction (Cheong et al., 2019). However, none of the pairs came with explanations of why dynamic friction is lower than the static friction (the shark tooth).

What Triggers Students to Come Up with Their Own Representations when Working in Pairs to Make Sense of a Computer Simulation of Friction?

In situations where the simulation did not play out as the students had predicted, or the graph displayed surprising features in different ways, this often served as a trigger for the students to generate their own representations of the phenomena. Our results support Scherr’s (2008) view that students’ gestures may serve as an indication of their pre-articulated ideas and facilitate their construction of new ideas. In this regard, Steier et al. (2019) emphasise the collaborative setting in encouraging students to make their understanding of physical phenomena publicly available. As students experience challenges in conveying their ideas in formal language, they resort to whatever means available, such as enactment of involved phenomena. An example of this in our study is how Bella exaggerates the intermittent movement of the block by comparing it to a jumping frog and making choo-choo train sound effects.

In making sense of the ripples, the students made use of interaction with the interface of the simulation and generated their own representations in the form of gestures, enactment and drawings. Exploring shifts across modalities, we observed shifting not only between physical and digital representations but also between gestural and verbal modes of communication where gestures, spoken language and enactment are sometimes replaced by each other and at other times combined with each other to form static or dynamic representations. For instance, in the second event, Bella communicates that a surface is rough first by gesturing and then verbally. Then, she enacts the macroscopic motion by moving a pen and the microscopic features of the surface in motion with a jumping hand and sounds. After that, Bella switches to verbal communication and at the end to drawing. In line with Steier et al. (2019), we found the use of depictive gestures and enactment to be a common practice amongst the observed pairs.

Similarly, we found support of Euler et al. (2019) that gestures and enactment can serve as shared embodied images around which students coordinate their exploration, with the examples of Anna rubbing her hands together and Diana clenching her hands to represent friction between two surfaces. These embodied images persisted long enough for the students to refer back to them in their dialogue, and stimulated creation of more permanent representations (Volkwyn et al., 2019), such as drawings of the microscopic structure.

Conclusions

In this study, we have shown that two pairs of students use unprompted representations in the form of gestures, enactments and drawings, as they engage with a simulation of friction. In particular, these representations are recruited in the students’ interpretation of links between microscopic aspects of the phenomenon and the simulation. It thereby contributes to research on students’ collaborative use of unprompted gestures and enactments in science learning (Steier et al., 2019), by showing how they can be triggered by use of dynamic, interactive digital resources. In addition to using their own bodies in enacting the phenomenon (Euler et al., 2019), the students recruit nearby objects in showing friction and make drawings of microscopic aspects of the phenomenon. In this way, the students show representational competence, in the sense of constructing and interpreting links between the phenomenon and representations of it (Lemke, 2003).

The study also contributes to showing the potential of using simulations in physics teaching (De Jong et al., 2013). If the students had conducted physical experiments of friction, corresponding to what was modelled in the simulation, they would not have been able to perform as many trials with different parameter values during the same time. In addition, they could have interpreted ripples in a graph as measurement errors, which would not invite to microscopic reasoning. In line with Stahl (2006), the two pairs of students spent a considerable amount of time trying to understand why they obtained different results than expected (the area does not affect the friction) or unexpected aspects of the provided representation (the ripples on the graph). In this regard, the obstacles triggered the students to discuss and rationalise microscopic aspects of friction. By analogy with the lesson’s topic, friction was introduced in the learning activity in order to stimulate students to reflect on their own understanding of the phenomenon.