1 Introduction

To adapt ones seeing towards the socially situated and historically constituted practices of a profession is a fundamental aspect of what it means to learn a profession (Goodwin 1994). What it means for a learner to develop such professional vision has been a reoccurring topic in the learning sciences since Goodwin (1994) coined the term. In this study the development of the sort of knowledge that has been described as bodily skills is in focus (e.g. Hindmarsh et al. 2014). The argument is that this sort of bodily knowledge is, similar to professional vision, part of the socially situated and historically constituted practices of a profession. For example, in Hutchins and Nomura (2011), it is highlighted that flying is an embodied activity, bringing forward the pilot’s need to know the plane’s roll and pitch, yaw and thrust. Their findings included a wide array of interactions where the pilot’s in simulator-based training used talk, gestures and enactments of the plane to jointly construct multimodal representations of the objects, events and actions that constitute flying an airplane. Similar expertise is needed also in shiphandling, although manoeuvring a modern size vessel has significantly different movement characteristics than the aircraft. Modern ships are massive objects – a medium size tanker weighs approximately 60,000 dead weight tonnage (dwt) – and the size and weight of vessels makes them slow to respond to changes in speed or direction (Bailey et al. 2006). Moreover, different vessels have different manoeuvring characteristics due to variances in ship design. As a result, there are differences in acceleration, stability and inertia between vessels. Amongst maritime professionals, the notion of ship sense is known as getting a feeling for your ship’s behaviour. Ship sense can be attributed to the ability to sense body-related input from the vessel, as well as visual outlooks of the environment and has been described as striving for harmony between the ship and its environment (Prison et al. 2013). Hence, ship sense relies on several different factors: the environmental prerequisites such as weather and contextual factors, the vessel’s specific prerequisites of the ship’s design, and the personal prerequisites of the seafarer or the ship’s crew. The personal prerequisites rely on experience, theoretical knowledge as well as spatial awareness. It is described as a result of both time on board ships and education (Prison et al. 2009; Prison et al. 2013).

In maritime education, the use of simulators in training has replaced parts of the on-board training during recent decades (Barsan 2004). There are several pedagogical advantages with simulator-based training: experiences can be made and lessons learned in a risk-free manner, as simulators offer opportunities to train skills that are time-consuming and costly to practice on board real ships (see, e.g., Dahlström et al. 2009). The controlled environment of the simulator also has pedagogical advantages as exercises can be designed to train and assess specific learning outcomes. In the simulator, layers of complexity in different situations can be added or reduced by adjusting to the students’ prior experience and knowledge (e.g., Maran and Glavin 2003). Moreover, simulator-based training offers possibilities to make changes during the scenario to adjust to the student’s performance or even pause the scenario for feedback and discussion. However, besides the clear potential for learning, there may be limitations to what can be experienced during simulation. Hontvedt and Arnseth (2013) suggest that maritime work practices rely heavily on aspects of space and temporality in an intricate way that hardly can be simulated in an educational setting. This claim seems mostly based on Goodwin’s (1995) study on work practices on board an oceanographic research vessel, rather than in their own analysis of simulator-based maritime training. Before establishing this view, it needs to be empirically grounded in analyses of simulator-based training in educational settings. In his following work, Hontvedt (2015) analysis on maritime pilot training on two different bridge simulators, one full-mission simulator and one simulator that lacks kinaesthetic feedback to its users, reached the conclusion that the lack of photorealism and fidelity in the simulator can negatively affect the dynamics of the exercises. A lack of simulator fidelity may cause the learners to simply manipulate the simulated model instead of training towards a professional work environment. Hontvedt (2015) argued that training sessions have to be carefully configured in order to avoid such pitfalls. As highlighted by Hindmarsh et al. (2014) in a study on dental students in training, that the simulator offers little in terms of learning. It also does not provide instructions on how to perform a manual skill or contribute with any explanations of why a certain conduct is right or wrong. What Hindmarsh et al. (2014) emphasize as important in instruction is instead to explore the seams between the simulation and the real work practice that the students are training for, which were found to occasion debate and discussion that was important for their development of professional expertise. In this way the inconsistencies between the simulator environment and the real world work practice were actually an instructional recourse rather than a deficiency of the simulator.

In relation to this, the current study explores how the embodied activity of shiphandling is trained in simulators that are lacking kinaesthetic and proprioceptive feedback of movement in the world. The bridge operations simulator in our study is a high-fidelity navigation simulator that mimics many of the features of the bridge of a real ship, such as the technologies used for navigating and manoeuvring the vessel as well as projections of the marine environment as seen through the front window of the bridge. While the bridge operation simulator is not a full-mission simulator with a motion platform that mimics the ship’s movements, the sense of moving in the world is rather simulated through visual input rather than kinaesthetic or proprioceptive input from the simulator. When replacing parts of the on-board apprenticeship with simulator-based training, there might be some types of knowledge that cannot be transferred from one context to another. In this study the analysis draws on video-recorded data from a navigation course to conduct detailed analysis on the following questions: How is body and talk coordinated with the environment to create and coordinate representations of the missing aspect of the simulator during instructions? What is the role of these representations in developing the students’ understanding of the ship’s movements in manoeuvring? The overall aim is to contribute with empirically grounded knowledge about handling the built-in inconsistencies in the simulator environment with results that have the potential to further the instructional practices in simulator-based training. The study uses a Distributed cognition (DCog) framework for analysis, taking the transformation and propagation of representations in a functional system as the unit-for-analysis (Hutchins 1995). Two instructional episodes are analysed with emphasis on how a bodily action becomes a representation, and how a representation accurate it’s meaning by analysing the coordination of talk and bodily conduct in instructions that draw on the material context on the bridge operations simulator. Following the DCog view, understanding, knowledge or expertise is not an internal property of either an artefact or a person: it is the act of interacting or coordinating with an artefact or a person that exhibits understanding knowledge or expertise (Hutchins 1995). Hence, the analysis revolves around how the students exhibited understanding of the instructions that take place in the simulator. However, since this is a microanalysis of local productions of instructions, no claims of if the students actually learn or develop shipsense will be made since that would require an analyse on a different time-scale. The interest is, as highlighted in the research aim, to explore how such instructions might facilitate learning.

The paper is structured as follows: The first section provides a background on simulators in training that focuses on aspects of the concepts of fidelity, realism and relevance of simulator-based training. Following this, there is a section on the theoretical framework of DCog (Hutchins 1995). The methods section provides an insight into the research design in terms of setting under study, the methods for collecting data, and the analytical process. The analysis sections present two episodes of video-recorded instructions with focus on analysing the events in relation to the research questions posed. The last section concludes the results and discusses the implications for simulator-based training.

2 Developing professional skills through simulator-based training

Shiphandling and the professionals’ knowledge known as ship sense have previously been explored by collecting shiphandlers’ experiences of the concepts through questionnaires and interviews (Prison et al. 2009; Prison et al. 2013). Results from these studies show that the shiphandler strives for harmony between the ship and the environment when manoeuvring the vessel. In order to do this, the shiphandler needs to develop ship sense, which is a skill accomplished by a combination of spatial awareness, knowledge and experience to handle environmental factors of context, situation and vessel specific factors of inertia, as well as the use of navigational instruments. Theoretical knowledge includes an understanding of the physical forces at work due to design parameters, while experience is mostly gained through sea time on board real ships (Prison et al. 2013). However, they also briefly mention the possibility to train specific scenarios in simulators. Simulator-based training is well established in the maritime education system, training students on a number of tasks related to manoeuvring and navigation. These tasks include the use of radar technologies, collision avoidance, and bridge resource management. Since simulators are used to practice skills in safe settings before training in real and potentially dangerous situations, simulation-based training is often discussed in terms of realism. In maritime education, the International Convention on Standards of Training, Certification and Watchkeeping for Seafarers (STCW) regulates the use of simulators for training and assessment, highlighting the need for using simulators that are suitable for selected objectives. The performance standards formulated put emphasis on the level of physical realism of the shipboard equipment concerned and the behavioural realism to allow a trainee to acquire the skills appropriate to the training and assessment objectives, as well as psychological factors involving instruction and debriefing. Also, the degree of fidelity is a central question in the literature on simulation in other domains with high standards for safety, such as healthcare and aviation. Since the nature of situations encountered in high-risk domains is complex and dynamic, it is considered important for the simulation to resemble the conditions of real world tasks (see, e.g., Dahlström et al. 2009). The prevailing idea is that if the simulation is close to the real world task, the greater the likelihood is that skill transfer occurs from one context to another. In the literature, a distinction is usually made between low-fidelity simulators that simulate aspects of the physical work setting in an abstract way, to high-fidelity simulators that are designed to match the appearance and behaviour of the setting to a high degree (see, e.g., Maran and Glavin 2003). For example, desktop-based simulators with simplified representations of visual aspects of the ship and the environment are considered to be low-fidelity. A simulator that simulates the ship’s visual, auditory, and motions cues in a realistic way would be considered as high-fidelity. While low-fidelity simulators are considered to be well suited for what is labelled “cognitive tasks”, high-fidelity simulators are more appropriate when training motor skills according to Maran and Glavin (2003). Beaubien and Baker (2004) suggest that this distinction is one-dimensional and too simplistic, putting too much emphasis on the technology rather than the educational goals, content, and design. They adapt a typology of simulation fidelity that has been used in aviation for their research on fidelity in healthcare team training, based on three interrelated aspects. First, the simulator’s equipment fidelity concerns the appearance and feel of the real technical system, i.e., if the simulator mimics the layout of the ship’s bridge in a realistic way. Second, environmental fidelity concerns to what extent the simulator represents visual and motion cues. Third, psychological fidelity concerns the degree to which the trainee perceives the training as a realistic task.

Without making a substantive review of an on-going debate on how to describe matters of learning settings and activities as authentic representations, a relevant concept for this study is to view this as a matter of realism and relevance for learning the real world work practice (see, e.g., Rystedt and Sjöblom 2012; Hindmarsh et al. 2014). However, it is important to recognize that realism and relevance are complex questions that, like the concept of psychological fidelity, go beyond technical features of the simulator. What the term realism and relevance does in contrast to the concept of psychological fidelity is to go beyond the individual, to highlight also the social and cultural aspects of a work profession. In a study on simulator-based training in healthcare, simulation as a realistic and relevant learning activity emerges and is maintained in and through interaction between the participants and the context (Rystedt and Sjöblom 2012). Even though the learners have been told to treat the scenario as if it was their work practice, this notion will not tell them how to act in the situations that unfold during a training session. Instead, the participants seem to continuously display to each other how the situations should be understood in terms of work practice, “as being objects of that sort” (p. 795). Moreover, their results show that both similarities and differences with clinical practice are important to grasp and understand in terms of work practice in the simulator environment. Hindmarsh et al. (2014) argue that these kinds of moments should be seen as instructional resources rather than a deficiency of the simulator. In their study on the use of simulators in dental education, inconsistencies between the simulator environment and the real world were used to highlight aspects of the curriculum for the students. Hence, realism of simulation-based training should be understood as a continuous social achievement that draws on the cultural practice that the students are training for. In this view, realism is an instructional concern rather than a matter of technical fidelity. Since improvements in technical fidelity are costly and have been found to provide rather small improvements for learning, an approach that considers the instructional work to facilitate realistic and relevant learning events might be more fruitful for improving simulator-based training (cf. Maran and Glavin 2003).

3 The role of gestures in distributed cognition

Using navigation on board a military vessel as the unit of analysis, Hutchins (1995) explored how cognition is socially and culturally distributed through time and space. While his emphasis was how information flows and propagates between different representational states in the system, DCog became a theory of cognition as embedded in socio-material practices between members of a social group and the tools they use. In this view, learning is seen as adaptive reorganization in a complex system, a socio-technical system (Hutchins 1995). The DCog’s focus of learning becomes one of the learners’ adaption towards the socio-technical system, by internalizing knowledge through interactions with mediating structures. This means that the boundaries between what is considered internal, mental processes and external, socio-material structures, are blurred and plastic; What happens inside a learner is the propagation of some kinds of structures of a system to another through mediation. Mediating structures can be both immaterial systems such as ideas, norms and rules, as well as culturally constituted objects, artefacts and tools, or the behaviour of others in the social group such as instructors or co-learners. Moreover, rather than viewing the relation between the individual and the environment as moving coded information across boundaries, Hutchins (1995) looked for processes of coordination and resonance inside the functional systems as well as processes of entrainment towards external systems.

However, as pointed out in Lindblom (2015), Hutchins’ work in Cognition in the wild was leaning more towards the social-material realm of cognition, being “oddly disembodied” to use the words of Hutchins himself. Instead, he proposes that all thinking and imagination has their origin in the interaction of body, talk and cultural world (Hutchins 2006). In Hutchins’ studies that followed his 1995 work gestures as representations is a prominent part of the DCog analysis, are clearly stated as influenced by the work of Goodwin. In Goodwin (1994, 2000, 2003), the notion of environmentally coupled gestures and symbiotic gestures, i.e. gestures that are used together with verbal articulations to highlight and provide meaning to elements of the physical world at the same time as these gestures acquire their meaning because of the coupling towards the physical world. This analytical focus can be seen in Hutchins’ studies on simulator-based training of airplane pilots. In Hutchins and Palen (1997) the analysis revealed how the participants are using crossmodal representations of space, gesture and talk in relation to objects and artefacts, constructing meaning through complex and multi-layered representations. What a DCog analysis revealed was new relations in what was firstly described as a single persons gestures to highlight and articulate aspects of the physical world. Instead, Hutchins and Palen (1997) delineated how this takes place when the process is distributed when involving multiple participants, and thus multiple streams of talk and gestures that articulate and highlight a shared physical world. In Hutchins and Nomura (2011), the mutual elaboration of multimodal utterances, talk and bodily conduct, are analysed: exploring how these multimodal utterances are distributed and co-constructed between participants and their environment. The results revealed how the pilots during simulator-based training use both their speech and their bodies to collaboratively represent the objects, events and actions that constitute the bodily skills involved in flying an airplane. What was particularly interesting is that this environment is not only material, it is also an imagined world that becomes manifested through the participants’ co-constructed multimodal utterances. Similar findings is seen in Goodwin (2000) as “embodied participant frameworks”. Such utterances are not orienting towards relevant aspects of the elements of the particular physical world such as environmentally coupled gestures does, they are a pointing towards the world that is the focus of the activity in a more general sense, but are still being symbiotic with the environment in which they occur.

Why the work of Goodwin was alluring for a DCog analyst is clear when considered in relation to Hutchins commitment to situate cognition and learning in the socio-material world of culturally constituted activities. Hence, the particular movements of the individual’s gesture and speech is not viewed in isolation in DCog, but also considers the distribution of representations in contextual surrounding in which the bodily action is carried out as important for meaning-making. Williams (2013, p. 240) explains the role of bodily gestures in DCog in the following way:

Gesture does more than externalize thought; it is often part of the cognitive process itself. Gestures create representations in the air; enact representational states on and over other media, and bring states in different media into coordination to produce functional outcomes. Gestures often plays a central role in propagating functional systems – associated cultural practices, cognitive models, and forms of coordination – across generations, while adopting them to the particulars of new problem situation.

As outlined in the quotation, gestures create temporal representations, they enact representational states, and they bring together representational states in different media to produce desired outcomes. The prevailing argument is that gestures have representational properties and should be included in the functional system. The relationship between representational states and enactions deserves attention here. In Hutchins (2010) it is described as a question of what it is that makes a material pattern into a representation, but moreover, what it is that makes it into the particular representation it is? The answer to these questions is in Hutchins’ view one of engaging in specific culturally shaped perceptual processes of action, a sort of high-level cognitive processes that require meaning from movement. The view is supported by recent work that has been measuring to what extent contextual cues can make a movement into a gesture, and a gesture appear more meaningful, and subsequently can encourage onlookers to give representational responses for bodily actions (Novak et al. 2016). The results from Novak et al. (2016) revealed the impact of external cues to the interpretation of bodily action, where the meaning can be enacted from the environment within which the embodied action occurred. They also suggested that the onlooker’s prior knowledge of a particular object might affect the ability to use the particular object to interpret a movement as representational as well as cues that indicate that the embodied action is part of a communicative act (e.g., the specific content of the speech, facial expressions and eye-gaze). To conclude, Novak et al. (2016) revealed that the richer the context, the more likely an embodied action will be interpreted as meaningful, and thus as representational, rather than as a movement for the sake of movement. In other world, it is the context in which a movement is embedded that makes the movement a gesture that has representational properties of being an enactment.

However, what is of greater importance in a DCog analysis is how the information continues to flow and propagate between different actors and representational states in the functional socio-technical system. That means how the student treats the instruction as meaningful, and what kind of understanding, knowledge or expertise is exhibited through that representational state (cf. Hutchins 1995). Drawing on an analysis of clinical work during dental education as an example, Hindmarsh et al. (2011) found that the bodily conduct of the students provides an exhibit of understanding that can either support or undermine a verbal claim of understanding. “In verbally claiming understanding the student simultaneously looks and, maybe more importantly, shows movement to look more closely into the patient’s mouth. So the claims are accompanied by visual indication of leaning in to ‘look with’ the demonstrator.” (p. 495). As outlined in the quotation, during demonstration aiming towards learning to see dental phenomena, the supervisors actively seek evidence from the students’ verbal claims in relation to their visual orientation. Hence, such representational states can be either verbal exhibits or claims of understanding or manifestations of bodily conduct, or a mutual elaboration of both.

To sum up, the studies outlines show how bodily actions and gestures have fundamental cognitive and communicative functions. As Lindblom (2015) formulates it: they lend a helping hand in expressing and sharing thoughts and are enabling us to talk about dynamic associations between different matters. Moreover, they acquire their meaning from the socio-material environment in which they are embedded.

4 Research design

The research questions posed in this study on how body and talk is coordinated with the simulator environment and the role of these representations in developing the students’ understanding of the ship’s movements situated the study out in the field: at a maritime simulator centre. The bridge operations simulator that the students are training on in this study mimics the bridge of a technologically equipped modern vessel. In the simulator, there are photorealistic projections of the marine environment and auditory feedback from sound, signals, and the constant background noise from the engine room. However, the simulator lacks a motion platform and thus lacks the kinaesthetic feedback of a real ship sailing at sea. The simulated vessel is modelled after a so called handy-size chemical/oil products tanker, 144 m long with a beam of 23 m, weighing 16,613 dwt. During simulator-based training in the navigation courses, the students are practicing shiphandling, the use of radar technologies, and the appliance of anti-collision regulations in bridge teams of two or three students in different scenarios. A typical scenario can be, for example, to manoeuvre and navigate through narrow or shallow waters, to handle different types of traffic situations, and/or to practice under different weather conditions.

In technical learning environments, such as the simulator under study, there has been an increased use of ethnographic fieldwork and video recordings as primary methods for investigation of learning activities (Heath et al. 2011). These so called workplace studies are driven by a practical concern to gain in-depth understanding of how people use technologies in their day-to-day activities (Luff et al. 2000). The aim is to explore the complexity of human-technology interactions “in the wild”, as they naturally occur in the setting (Heath et al. 2011). While ethnographic fieldwork is essential for developing an understanding of the educational content in the navigation courses that enables an analysis of maritime simulator-based training, video creates stable records for detailed, repeated and collaborative analyses of the verbal, visual and material learning practices under study (Heath et al. 2011). Following this approach, ethnographic fieldwork was conducted over a period of three years, in order to develop the kind of competence needed to analyse maritime training practices. Moreover, video-recorded data was collected from two different navigation courses that took place during the master mariner students’ second year of training. Approximately 75 h of simulation-based training were filmed using multiple cameras on the five different bridge operations simulators that were used simultaneously during training sessions. All participants in the study (n = 65) have signed an informed consent, agreeing to be filmed for research purposes. When the analytical phase started, a first selection of video data was made. Based on observations, the instructions during on-going training scenarios appeared as interesting. Moreover, two of the training sessions were singled out as representative for the simulator-based training design of students training in teams on the different bridge operations simulators. This selection narrowed the analysis down to 44 identified instructional episodes that was transcribed with focus on what was being said as recommended by Heath et al. (2011). For this study, the focus was to identify and select a class of instructions that uses bodily conduct or gestures to represent the movement of the ship during exercises in the simulator. When such an episode was identified in the larger data corpus, it was added to the collection. In this stage of analysis, the transcriptions were developed to also delineate how things were said, with focus on both the deliverance of utterances and bodily conduct (cf. Heath et al. 2011). In all, the collection consists of five episodes where the body was used to illustrate movement in different ways. For the article, two episodes were singled out to demonstrate different aspects that emerged as interesting in these kinds of instructions. These two episodes were analysed further, scrutinized in a frame-by-frame analysis with focus on the details of the created representations and the coordination of representational states between the instructor and the students (cf. Lindblom 2015). In the different episodes there are two different, well-experienced instructors and three, respectively two, students on the bridge operation simulator (n = 5). The video material was also the subject of collaborative analyses in different stages of the research process, involving several educational researchers and maritime instructors in data sessions (cf. Heath et al. 2011).

5 Analysis of two instructional episodes

The analysis in this study is built on two brief episodes of video recorded data to conduct detailed analysis on the following questions: How is body and talk coordinated with the environment to create and coordinate representations of the missing aspect of the simulator during instructions? What is the role of these representations in developing the students’ understanding of the ship’s movements in manoeuvring? Episode 1 embodies the ship’s movements during a turn, and Episode 2 embodies the speed of the vessel. Both episodes took place in a navigation course during the second year of a master mariner programme, training the students on manoeuvring, radar navigation and collision avoidance.

5.1 Representing direction and enacting sway

In the first episode three students in a bridge team are training where they are located while moving on open waters in a scenario. During the exercise the students are practicing the different work roles of a bridge team: officer-of-the-watch and lookouts. The role of officer-of-the-watch is taken by Student 1, who has the overall responsibility on the bridge, as well as the task to answer the instructor’s questions. The pedagogical idea behind this organization is that the commanding officer should exhibit an elevated awareness of the situation at hand, training the students into the working demands of their future profession. Student 2 is taking the role of lookout in this exercise, a lower rank on a ship’s bridge, mainly supporting the officer in command in keeping a close eye on the marine environment. At sea the commanding officer is someone of greater professional training and experience than the lookout, but in the educational context both students are classmates and have a similar level of knowledge and experience. Even if the simulated exercises are designed as role-playing bridge teamwork and work roles, they also function as a learning experience. We are going to see that there are moments where the students are able to step out of the role-play and into a student role where anyone can answer the instructor’s questions (Fig. 1).

Fig. 1
figure 1

A gesture that is representing the rudder of the vessel

Full size image

During the scenario, the students encounter another vessel and have to take evasive actions to avoid a close quarters situation, i.e., a situation where there is risk of collision with another vessel. When the episode starts, the instructor (referred to as “ins” in the excerpts) has seen the events unfold from the instructor’s room and now has entered the bridge operations simulator to coach the students (referred to as “st1” and “st2” in the excerpts) through the situation. In the first part of the transcript, information flows back and forth between the instructor and Student 1 who is taking the role of officer-of-the-watch in this scenario.

Excerpt 1: Highlighting aspects of the curriculum

figure a

The instructor’s first two sentences (lines 01 and 03) are interesting in relation to the notion of ship sense. It puts forward the need to know your ship, as well as the need to strive for harmony in manoeuvring if there are passengers on board, which is well in line with how ship sense is described in the literature. Moreover, the talk is framing the instruction as being an object “of that sort” (cf. Rystedt and Sjöblom 2012), ensuring that the information in the functional simulator system propagates in line with the discourses that constitute the real world work practice. In lines 05–10 we can see how the instruction is oriented towards the manoeuvring technicalities of using the rudder. In line 05 the instructor is commenting on the student’s settings on the rudders and thrusters on the bridge panel, correcting the settings with the instruction “it’s always left there”, making a hand movement towards the equipment and highlights the need to activate it. The indexical nature of the utterance “there” in coordination with the hand movement toward a specific device on the manoeuvring panel makes the hand movement into an indexical gesture, showing the students exactly where “there” means in this instance. The student responds to this information with an action; he activates the right setting in the manoeuvring panel. After the settings are aligned in a way that enables the activity to be carried out, the instruction continues with the verbal utterance in line 07, “then you’ll start with just a few degrees rudder”, which connects back to the previous talk on not knowing your ship, or the possibility of sailing a passenger ship, emphasizing the need for caution in these particular situations. Just as the instructor says “few degrees rudder”, he is putting both his hands in front of him, aligned vertically towards each other, at a height with his shoulders. In this way, he is making his hands the centre of attention, tilting them slightly from left to right and then left again. The coordination of the two different media, the talk on a few degrees rudder and the hand movements, creates a representation of the rudder of an imagined real vessel. Moreover, the subtle hand movements from left to right and back to left, when seen in coordination towards the simulator environment, become an enactment of the imagined rudders’ movements from portside of the imaginary vessel to starboard and then back again. Student 1’s question in line 09 is following the information provided by this instruction and transforms it into an actual manoeuvring action, filling in a few degrees rudder on the manoeuvring panel. The instructor oversees the student’s actions and confirms the starboard turn as a correct sort of turn. Also, a clarification of what a few degrees rudder means 2–3 degrees is made, instructing the student’s use of the instruments, ensuring that the information is transformed into the exact correct action.

The design and placement of the rudders are one of the many things that affect the manoeuvring characteristics of the ship in a fundamental way (Prison et al. 2013). As can be seen in the beginning of the excerpt, the instructor is framing the encountered event as being an “object of that sort”, i.e., an event where ship sense is of importance (cf. Rystedt and Sjöblom 2012, p. 795). To address such professional matters during simulator-based training is known to enhance the learning experience, connecting the simulated experience with the real world work practice. It is argued that both similarities and differences with the real world are important in order for the students to grasp and understand the simulation in terms of the work practice. In line with their results, Hindmarsh et al. (2014) argue that the pondering of questions about relevance and irrelevance during training is an instructional resource rather than a performance gap or shortage of the simulator. Inconsistencies between the simulator environment and the real world are a resource that can be used to highlight aspects of the curriculum, providing insights on the work practice for the students. Furthermore, the instructors’ work in this episode of treating the simulator as if it was a real vessel are important to avoid falling into a situation where the students are adapting to the simulated model instead of the real world practice. Similar mistakes have been found in simulator studies across domains. For example, Hindmarsh et al. (2014) found that dental students training to use the dental drill on a simulated patient made mistakes that would cause serious distress on a living patient and, in fact, even break a real tooth. Related mistakes were found in Hontvedt’s (2015) study where even professional maritime pilots in training on a full-mission simulator started to rely on the electronic chart instead of using a visual lookout of the environment, which was preferred. The reason for this behaviour was that the simulator lacked in photorealism and navigating by the chart became a convenient workaround. Because the simulator in the current study lacks real world movement, it would be an easy mistake to make for the second year students to just turn their simulated vessel towards the planned course or to overlook the need to increase speed, not paying attention towards the need to strive for harmony in manoeuvring. In this way, the class of embodied instructions identified in the material does important instructional work in preventing negative skill transfers that can occur when the students are merely learning how to master a simulated model (cf. Hontvedt 2015).

Excerpt 2: Failing to deliver the preferred answer

figure b

When the instructor asks how the ship reacts to rudder he is merely asking a control question to see if the students understand the non-verbal instruction, the subtle cues he gives that at this time lacks a declarative account. At this time he seeking eye contact with st1 that are taking the role of officer-of-the watch in the exercise and are in charge of the bridge team. However, the answer that follows from st1 that “hopefully, it responds?” and how the instructor treats the answer suggest that this is not the confirmation of understanding the instructor hopes for. Instead, the instructor is reformulation the question in lines 14–15, addressing it as something with a different preferred answer. Although the answer from st1 is correct in a trivial sense, it does not display the sort of understanding the instructor is looking for. At this time, the instructor is also orienting his gaze towards the other students on the bridge, inviting them to provide an answer (cf. Goodwin 1981). Using a DCog lens on the situation this can be described as an instance of information flow breakdown, raising questions on where the information is in the overall system or if the information has been communicated effectively (cf. Galliers et al. 2007). The reformulation of the question could provide a clearer request for information, asking about the linear lateral movement from side to side during a turn known as sway. The instructor’s head movements in this instance of talk is also interesting, tilting his head from left to right and the left again as he is looking at the students, or as we say on the bridge of a vessel, from portside to starboard and back to portside. Although subtle, the instructor’s head movements in this instance are used to represent the sway during the turn. When the head movements is delivered together with a shift in facial expression, squinting his eyes, and an emphasis on lean in his verbal utterance the different semiotic resources are elaborating each other, giving the students a cue of what the instructor is asking for. Also, the instructor’s talk together with his head movements and his gaze orienting towards the other students on the bridge functions as an invitation to answer his raised question. While st1 remains rather passive st2 starts to answer the question instead, picking up on the invitation to respond. Hence, the information flow breakdown of st1 failing to deliver the preferred answer opens up for a situation where the roles on the bridge goes from role-playing in the professional realm towards taking on student roles in an educational setting. The following sequence of talk shows a sequence of coordination between the student’s answer, using both hands and talk in his explanation, and the instructors response to his displayed understanding of the situation (Fig. 2):

Fig. 2
figure 2

Gestures that enable instructions on the ship’s movements

Full size image

Excerpt 3: Representing and enacting sway

figure c

When starting to explain that the vessel “lean inwards”, Student 2’s utterance is paired with a hand gesture, starting a movement of the right hand from the portside of the vessel towards starboard (see Frames 2 and 3). Student 1 continues the explanation with the utterance “n’ then it leans out”, which is paired with a movement from starboard towards the portside (see Frames 4 and 5). The instructor’s utterance is delivered right after the student’s, as well as his hand movements that are following the student’s gesture, again confirming the student’s explanation as being the correct answer to the question. The instruction that follows goes beyond confirming the student’s correct answer. The instructor’s utterance “n’ then it starts to cut” is paired with a hand movement showing what that means, simulating how the vessel is straightening up and again leans slightly towards the starboard side again (Frames 6 and 7). For the instructor, the student’s explanation becomes an exhibit of understanding (cf. Hindmarsh et al. 2011). Bodily conduct can either support or undermine a verbal claim of understanding, and in this case the student’s hand gesture shows the direction and exhibits that he has at least a fundamental understanding on how the ship would move at sea following this particular manoeuvre. The hand gesture represents the movement the student is trying to express also verbally. The student’s use of different representations, talk coupled with hand gestures, complement and enhance each other and allows him to express his idea on the dynamics of the ship’s movements. The instructor responds to the student’s account of the ship’s movement by confirming that the ship will lean inwards, also pairing his talk with hand gesturing that shadows the hand gestures of the student. However, the instructor is not using gestures in the exact same way as the student. Instead of using one hand to show the direction, he is using both hands together, shaping them like the bow of the vessel. The differences between the student’s hand movements and the instructor’s hands are interesting here, revealing their different perspectives on the situation. The student’s gestures indicate that he views himself standing on the bridge of the vessel, while the instructor’s shaping his hands as the ship’s bow displays an observer perspective of the situation. Although this is not communicated verbally in the episode, it might not need a declarative account. The instructor’s hand gesture goes beyond an explanation of direction, as it becomes an enactment of the ship’s sway. At this time, the enactment becomes a stand-in representation of the information that is missing in the simulator environment. Also, the instructor is adding information that is missing in the student’s explanation, providing talk and gestures to create a representation that shows the students how the vessel will start to cut through the waves, enacting the ship’s movements in relation to the environment, to ensure that the students’ understanding of the ship’s movements are in line with the real world practice. Of interest to observe in the video is also how the student and the instructor mirror each other’s body posture, leaning from one side to the other simultaneously, representing the leaning motion of an imagined ship at sea. In this way the movement of the ship is not just represented and enacted with hand movements, their whole bodily postures are used to enact standing on the bridge of a vessel during a sharp turn. Hutchins and Nomura (2011) made similar findings, suggesting that the participants imagined the whole situation of the body on the flight deck based on the embodied enactments that occurred. In this way, these enactments provide meaning-making activities that “bring forth the imagined environment and are coupled to elements of that imagined environment” (Hutchins and Nomura 2011, p. 28).

The last part of the episode is aimed at explaining what action the student can do after exploring how the vessel reacts to 2–3 degrees rudder:

Excerpt 4: Returning to the role-play

figure d

Here, the instructor highlights it is in this exact time, when the vessel starts to cut through the waves, one can start to increase the turn ratio. At this time, 15 degrees rudder is suggested, accompanied by a nodding towards the manoeuvring panel that is paired with the indexical utterance “there” in line 23. The instruction is followed by a confirmation from Student 1, returning to his role as officer-of-the-watch and the role-play, answering back in the closed-loop format that is used in bridge teamwork.

5.2 Representing speed and enacting inertia

In the next episode, the students are training collision-avoidance in the heavily trafficked Strait of Dover, crossing the Traffic Separation Scheme (TSS), where two lanes run through the strait for inward and outward-bound traffic. Making a crossing in this area is a high-risk operation. Large vessels can take several minutes to respond to changes in speed and direction, and speed changes at sea can be hard to notice. The main reason for this difficulty is that ships move at a considerably slower pace than most traffic. That means that speed changes can be hard to detect with the bare eye, especially in areas where there are few landmarks or because visibility is poor due to weather conditions. In this the case, the exercise has been created where the students have to navigate in restricted visibility caused by heavy fog. When both visual and kinaesthetic feedback is missing, the main source of information about speed for the students is the feedback from the navigational instruments. Hence, rather than seeing or sensing speed, the students have to grasp the notion of speed on a theoretical and abstract level during this exercise. In the beginning of Excerpt 5, the instructor follows up on an instruction of making the crossing by a certain degree of turn. At this time, he has moved towards the door of the simulator, standing in front of the doorway while addressing what would be a suitable speed in this situation (Fig. 3):

Fig. 3
figure 3

Representing the simulated increase of speed with a hand gesture

Full size image

Excerpt 5: Representing speed and enacting inertia

figure e

Since the students have slowed down their vessel in order to time their crossing over the TSS lane, adapting to the surrounding traffic, the instructor highlights the need to now increase the speed (line 01). His talk on increasing the speed, formulated as “if we have zero seven then we’re going up to fourteen knots” is paired with a hand gesture. At the time he says “zero seven”, his hand is in a position in front of him, holding a laser pointer (Frame 8), but moving in a direction upwards and forward, using his hand as a representation of “going up” (Frame 9), locating his hand in a position further away from his body as he says “fourteen knots” (Frame 10). The student in command, Student 1 agrees with this by making a humming sound while following the instructor’s hand gestures with his gaze, nodding his head. In a similar vein as in Episode 1, the instructor’s use of two different representational states, hand gestures paired with verbal instructions, are used to share and express his knowledge of the need to accelerate. In this way, the hand gestures represent the increasing speed by going upward and moving forward function, creating a metaphor for speed, his gestures are lending a helping hand in sharing and expressing thoughts about something that is rather abstract in the situation (cf. Lindblom 2015). Although speed is a concept that might not be considered abstract in most cases, it is abstract in the non-moving simulator where the information of speed is obtained mainly through the abstract and discreet numerical values displayed on the navigational instruments. Following this, the creation of bodily representations in instruction to represent abstract concepts has the potential to ensure that information propagates in the functional system, avoiding information flow breakdowns, i.e., that the students are missing relevant information for maintaining the activity due to the discreetness of the representational state. It is also interesting to notice the difference between these hand gestures as representations in comparison to hand movements as enactments. Here, the metaphorical nature of the gesture makes them into a representation of an abstract concept rather than an enactment of speed in the real world.

The instructor continues his instruction to explain the need for a plan, a common instruction in the bridge operations simulator. Since the vessels that are trafficking the seas today are massive and slow to respond to manoeuvring actions, the manoeuvring characteristics of large vessels influence the cognitive demands of the bridge team. The team has to be able to think ahead, to understand how the traffic situation will unfold over time. The instructor follows up on this reminder of having a plan with a verbal instruction on what one should avoid: “not to be crawling at eleven knots”. At the moment when the instructor says eleven he starts moving from his position by the door taking three steps towards the bridge panel. The body position while moving forward is leaning backwards with his left arm stretched before him, taking small steps in a trudging way, twitching his body back and forth in small movements (Frame 11). In this way the instructor is using his whole body as a representation to simulate the inertness of the ship’s movements at a low speed (Fig. 4).

Fig. 4
figure 4

The instructor is enacting inertia by moving his whole body in the simulator

Full size image

The bodily conduct here is interesting for two reasons; being an action representation that lacks an explicit verbal articulation, but at the same time is symbiotic with the simulator environment (cf. Goodwin 2003). The staggering walk enacts the inertia of the ship, another of the different manoeuvring characteristics a seafarer would attribute to shipsense (Prison et al. 2013). An increase of speed would reduce inertia, enhancing the harmony between the ship and its environment. Similar to Episode 1, the instructor is enacting the whole situation of the body on the bridge of a real vessel, and the embodied enactments are coupled to elements of that imagined environment (cf. Hutchins and Nomura 2011). These moments of entrainment in instruction can be explained as adding a layer of functionality in the simulator where functionality is missing, even if that layer is imagined, using the body to fill the missing information from the simulator environment. When movements are viewed as representations or enactments in gesture, Novak et al. (2016) are arguing that these representational actions have an impact on communication, learning, and cognition in significant ways. Their results show that when interpreting another individual’s movements, onlookers can attend to contextual cues arising from a variety of sources. If these properties are similar to properties of a movement that might actually be performed on an object, the movement is more possible to be seen as a meaningful bodily action, and therefore interpreted as representational. It is the both the fact that the instructor is moving in this way in the bridge simulator, in combination with the topic of speed in his verbal instructions and his use of the Swedish saying “ligga å harva” (which was translated to “crawling” in the simplified transcript) that provides meaning to the movement even if there is no explicit verbal account on this being a moment of inertness. For the students that have been spending time on board real ships during their training such a moment of inertness can be connected to the ships propulsion system.

At this time, both students respond to this performance, Student 1 with the talk “No no no!” and Student 2 (in the role as lookout) with a mumbling sound. The instructor then reminds the students not to forget that they have slowed down, highlighting the need for treating the simulated event as a real situation. Student 1 responds to this remark with a glance at the instruments, gathering information from the representation of speed on the bridge panel, reporting back that the speed is set at 15 knots. The transformation of a representational state, from the digital representation of speed towards the student’s verbal response, makes the information a public display of his understanding of the need for speed in this situation. Hence, the instruction can be seen as addressing something that the students already understood, merely checking that they did not forget to accelerate after slowing down, instead of developing their knowledge. However, the instructor’s enacting of inertia aims to ensure that the correct information propagates in the functional system, i.e., that the students grasp and understand the need for reducing speed as striving for harmony in manoeuvring, sharing their experienced knowledge in order to develop their professional understanding of ship sense.

6 Conclusion and discussion

This study explored the following questions: How is body and talk coordinated with the environment to create and coordinate representations of the missing aspect of the simulator during instructions? What is the role of these representations in developing the students’ understanding of the ship’s movements in manoeuvring? When conducting a detailed analysis show that the use and coordination of different representations, talk and gestures that are coupled towards either the simulator environment or an imagined real world, are used in multiple ways during exercises. These representations are used to highlight aspects of professional knowledge, and they have the potential to prevent negative skill transfer caused by a lack of realism in the simulator. Moreover, as in all types of learning settings, the coordination of different representations enables the participants to grasp and communicate dynamic relations between elements in the world. Following this, the results suggest that the coordination of representational states, bodily conduct and talk, are used to fill in missing aspects of the real world in the simulator environment. This includes representations of the objects involved in shiphandling, i.e., the rudders or the vessel itself, showing and enacting their movements during an activity. Moreover, the events and activities that constitute shiphandling are addressed in instruction, enacting dynamic aspects such as direction and the ship’s interactions with the marine environment. The instructional work of an experienced seafarer that is accomplished in the simulator environment connects the simulated events with the students’ experiences of the real world work practice encountered during on-board training, as well as showing the relevance of theoretical and abstract knowledge in practical situations.

The second part of the research question concerns the role of these representations in developing the students’ understanding of the ships’ movements in manoeuvring or, as Hutchins (1995) might put it, the propagation of representational states in the functional system across generations. The instructor’s concern in simulator-based environments is one that teachers everywhere regularly ponder: questions on whether the students understand the instructions or not. Hence, the instructor looks for displays of understanding in the routine instructional work that takes place in the simulator environment (cf. Hindmarsh et al. 2011). While it might be difficult to make any strong statements on how the students learn from the verbal instruction and the enactments of the ship’s movements, there are interesting aspects of synchronizing body position and movements between Student 2 and the instructor in the first episode. Although the cues about the right answer are in the detailed nuances of the instructor’s talk and environmentally coupled gestures, Student 2 is able to pick up on them and answer the question, using a combination of hands, bodily posture and talk to express his idea of the ship’s movements. In Episode 2, the instructed embodiment of enacting inertia aims to ensure that the students grasp and understand the need for reducing speed as striving for harmony in manoeuvring, developing their professional understanding of ship sense. However, if the students are picking up on this with increased understanding, it is not put on display in the analysed video material. In Hindmarsh et al. (2014, p. 256), the corrections during simulator-based training are described as instructional work to “help shape the body of the student to the body of a dentist, even though the socio-material apparatus at hand does not demand it”. Although it would be too strong to claim that the students develop actual ship sense during simulator-based exercises from our material or that the analysed instructions are shaping the students’ bodies to the body of a seafarer, the conclusion of the analysis is that these instances of instructions show potential for developing these types of skills. In the episodes, the instructor’s coordination of talk and gestures towards the simulator and an imagined real world during training sessions is being used to develop their understanding of the ship’s movements are in line with the discourses that constitute the practice of a seafarer. In an analysis on different time scales, one might start to see a progress in terms of learning the skills involved in ship sense. This in turn brings us to the next question: would that be a development of ship sense or a development of “simulator sense”? This study might not provide the answer to this complex question, but it contributes with added knowledge on the instructional practices in the simulator and their potential for learning the profession.

While it might be tempting to solve the problem of training motor and bodily skills with more advanced simulators, it is important to recognize that a motion platform could not facilitate learning in the ways that are seen in these episodes. The simulator does not provide information on the importance for a seafarer to know one’s ship or contributes with any explanations of why a certain conduct should be carried out in exactly that way and not in another (cf. Hindmarsh et al. 2014). This might be one explanation on why costly improvements in technical fidelity have been found to provide rather small improvements for learning (cf. Maran and Glavin 2003). We might actually be fooling ourselves if we start to believe that technologies are self-instructing if they are advanced enough to be perceived as realistic. Instead, inconsistencies between the simulator and the real world should be seen as instructional recourses, but they must be addressed in instruction in order to avoid pitfalls in training (Rystedt and Sjöblom 2012; Hindmarsh et al. 2014; Hontvedt and Arnseth 2013; Hontvedt 2015). In line with these results, this study contributes to a corpus of work that considers how instructors and students attend to matters of realism in the course of instructional sequences, highlighting the need for competent and fastidious instructors to closely monitor and facilitate the students through exercises in the simulator. Although the video-recorded episodes of instructions in the simulator are brief, the first episode is 60 s long and the duration of the second episode is merely 12 s, they are packed with meaning. In research traditions concerned with these kinds of micro-analyses: scrutinizing small samples of data to analyse even the finest details of interactions the quality of analysis lies in the analytical rigour of the analyst rather in the generalizability of results (Heath et al. 2011). Still, similar results can be seen also in other simulator settings, as for example in the results from Hutchins and Nomura (2011) from simulator-based training in aviation, or in Hindmarsh et al. (2014) analysis on simulators in dental training. Hence, the results might be transferrable also to other contexts. The implications of this and mentioned studies call for an allocation of the educational resources spent on simulator-based training. Alongside developing the technologies used for instruction, significant effects towards better and more meaningful learning activities would be achieved if the striving for fidelity was matched with efforts of developing the instructor’s pedagogical competence and supporting his/hers everyday work.