1 Introduction

Simulators have been used for training and certification in Maritime Education and Training (MET) since they first appeared in the 1950s. Hanzu-Pazara et al. (2008) describe how simulator-based training was introduced in MET with the primary intent to train navigation skills such as passage planning and the master/pilot relationship. Today, simulators are used in several parts of the maritime industry, from offshore operation training on vessels and oil rigs, involving bridge operations, cargo handling, engine control, crane operations, towing and anchor handling. Simulators are also used in ship-to-shore training, training for crane operations and vessel traffic services (VTS). Furthermore, the use of simulators in MET is regulated in the Standard of Training, Certification and Watchkeeping for Seafarers (STCW) convention that provides regulations for training in simulators as well as performance-based competency tests (Section A-I/6; Section A-I/12). The latest update of the STCW code, the Manila amendments 2010, has a greater focus on technical proficiency and the non-technical skills of team management and resource management on the bridge than previous conventions. Although the practice of using simulators is well regulated and widespread in MET, it seems few studies address the pedagogical use and benefits of simulator-based training in this domain. The overall aim of the current systematic review is to create an overview of the research field, common themes of interest, conclusions made to date and to synthesise the collective knowledge of the field at large.

While the systematic review methodology has been commonly used in educational practice in healthcare over the last two decades, and had a great impact of what is known as an “evidence-based practice”, the use outside of medicine has been limited (Bearman et al. 2012). The benefits of doing systematic reviews are several: ideally a well-executed literature review should make studies assessable and guide the reader towards the literature. It should provide trustworthiness and accountability of the review process, opening up for the readers to make their own judgement of the quality and meaning of the evidence. Moreover, a structured review reveals conceptual and value positions that can otherwise remain hidden within the discourse of different disciplines. The qualitative synthesis approach was chosen since research in the field is cross-disciplinary and consists of both quantitative and qualitative research methods. It is regularly used within systematic reviews as a way of pooling different sets of data and thus gathering collective wisdom from a range of different research methodologies (Bearman and Dawson, 2013).

2 Method of the systematic review

The systematic review methodology draws on the Cochrane handbook, and the key features of conducting a systematic review in an explicit, reproducible and methodological way as presented in Bearman et al. (2012):

  • A clearly defined set of objectives with pre-defined inclusion criteria for studies

  • A systematic search that attempts to identify all studies that would meet the eligibility criteria

  • An assessment of the validity of the findings of the included studies

  • A systematic presentation and synthesis of the characteristics and finding of the included studies

The criteria for inclusion in this review are that articles should study the use of simulators for training and assessing bridge operations in MET. Furthermore, the studies should be peer-reviewed journal articles, searchable in major academic databases, available in English and published between the years 2000 and 2016.

The keywords that were used were grouped into three categories and combined during the search, following the strategy of Bearman et al. (2015). The first category included words in relation to the maritime domain: “navigation”, “bridge”, “ship”, “marine” and “maritime”. The second category contained words related to the use of simulators, but to cover all possible variations the search used “simula*”. The third category of words used in the search is connected to MET: “training”, “assessment” and “education”. A first search was conducted by using Google Scholar, resulting in 71 found articles on simulator-based maritime training and assessment. While Google Scholar is known to include searches from student work, organisational and technical reports, journals lacking in academic credibility and so on, it was necessary to go over the results from the Google Scholar search to ensure the quality of the studies found (cf. Gehanno et al. 2013). After an initial screening, based on whether the articles meet the criterion of being published in a peer-reviewed academic journal, 41 articles remained. After reading through the abstracts and the results sections, ten more papers were excluded from the review, mainly because the scope of the paper was outside the use of simulators for training purposes (e.g., addressing simulator design questions on an algorithmic level rather than their usefulness for training). In all, 31 peer-reviewed journal articles were included in the review after the Google Scholar search.

Although Gehanno et al. (2013) concluded that the coverage of a Google Scholar search was 100 % in comparison to searches in so-called “gold standard” databases in medicine, others state that Google Scholar should never be used in isolation (e.g. Giustini, 2005). In order to ensure that all relevant studies were included in the review, the next step of the literature search involved two different libraries and two different librarians, one with focus on educational literature and the other from a technological university were guiding the search through the chosen databases: ERIC, Education Research Complete, ProQuest, Scopus, Marine Technology Abstracts and IEEE Explore. Six more items were found of which three was included in the review after screening, showing that a Google Scholar Search alone does not suffice. In the last stage of the article search, items found through recommendations were included (n = 1). A flow chart of the systematic search is presented in Appendix 1, and the result of the systematic search is presented with a detailed overview of the articles in Appendix 2. Table 1 is a summary of the field at large, containing information on the research domain, the methods used, geographical location, as well as a list of academic journals where the studies were published.

Table 1 Summary of the reviewed articles
Full size table

3 The qualitative synthesis

As the studies included in the review are both quantitative and qualitative, the systematic review is followed by a qualitative synthesis to draw conclusions on topics and themes that are recurrent in the articles and the conclusions made to date. A narrative summary approach to qualitative synthesis that aims to produce a selective account of the evidence was chosen, since it allows for going beyond mere description towards interpretations and reflections on a higher level of abstraction than for example thematic analyses (Dixon-Woods et al. 2005). The synthesis is organised in accordance to each of the disciplines with a section providing an insight into the nature of the studies, the major themes and foci identified in the articles, as well as reflections on their contributions towards the MET research field at large.

3.1 The knowledge and experience of maritime professionals

As can be seen in Tables 1 and 2, a large amount of the found articles come from what was categorised as “maritime professionals”. These are most often well-experienced seafarers and maritime instructors at leading positions in maritime universities, and they come from MET institutions from all over the world. It is also notable that these articles are based on knowledge of the field and experiences rather than empirical data. At times, surveys are used, but mainly in order to collect the experience and opinions of maritime professionals in a wider sense. While some of these articles provide expert advice to inform policymakers in the industry (e.g. Barsan, 2004; Barsan et al. 2007), their main contribution is something other than empirical results to inform an “evidence-based practice”. They offer an insight into MET, its history and challenges, and formulates important questions that are connected both to Human factors and Education research.

Maritime professionals are most often positive towards simulators in training, yet remain concerned about the challenge that MET faces when introducing new technologies and replacing old traditions. Some of the main concerns are the upgrades of the IMO and STCW conventions and the practical impact it will have on MET. In order to ensure that future mariners can act properly and safely in practice, the conventions stress that simulators should be used for training and certification of proficiency and non-technical skills. This in turn raises a number of questions amongst maritime professionals, most commonly, if simulator-based training work in terms of improving safety at sea and reduce human error (e.g. Hanzu-Pazara et al., 2008). Another type of questions is more didactic: What would be the optimum training to ensure non-technical skills transfer? How can non-technical skills be effectively assessed on both individual and group level? What cultural factors need to be identified and addressed in training? (Pekcan et al. 2005). Also, a number of papers call for simulator technologies that are more advanced, as well as better-trained instructors, to meet these new demands on MET institutions (e.g. Ali, 2008; Hanzu-Pazara et al. 2010).

3.2 Human factors research and the usefulness of simulators in training

Human factors is a field of research that is well established in the maritime domain according to Grech et al. (2008). They describe the main interest of the field to be the human and their use of the elements in a technical system: the well-being of the human and the overall performance of the human-technology system. Hence, one central interest in human factors is to develop technologies that better fit the cognitive abilities of the user, creating safer systems for different aspects of the maritime domain. In the review, it can be seen as an interest in user testing how simulators can be designed to support different aspects of navigation training, such as ice navigation (Cemal and Burak, 2012) or precision navigation on rivers (Demchenkov, 2011). Of interest is also the relationship between simulator fidelity and the quality and transferability of training. The prevailing idea in most domains that use simulators in training is that if the simulation is close to the real-world task, the more likely it is that skills transfer from one context to another. In a case study on simulators used both in aviation and shipping, Dahlström et al. (2009) concluded that transferability of training should not rely on high-fidelity simulators alone. The primary reason for this is that is economical; it is costly to train in high-fidelity environments, and the training sessions are often designed to follow rehearsed roles, duties and procedures, and supports the development of procedural skills. Low-fidelity simulators on the other hand offer opportunities to train repeatedly for unanticipated and escalating situations, which is highlighted as important for developing resilience. Hence, based on Dahlström et al. (2009), it can be suggested that training programmes should combine the use of high-fidelity and low-fidelity simulators in their curriculum.

Another interest in Human factors is if simulator-based training works efficiently; for example, if the development of different so-called non-technical skills such as situation awareness (SA) or decision making can be trained in simulators (Saus et al., 2010, 2012; Chauvin et al. 2009). These studies use experimental and mixed method approaches to isolate and measure SA during different training conditions. The results show that the trainees subjective SA as well as the perceived realism of the training event has a positive effect on the perceived learning outcome of the trainees. Drawing on the results of Saus et al. (2010), it is important to design training procedures and scenarios that enable students to perceive the simulation as a realistic training event, and at the same time design scenarios that are well adjusted to their level of competence in order for them to benefit from the learning experience. Also, Chauvin et al. (2009) found that students that were practising SA and decision making in simulators improved in analysing complex situations. However, it is notable that students that were part of an on-board-training programme showed significantly better results that the simulator-based training group. These results are important because they point to the value of on-board-training, and call for caution before replacing on-board-time with simulator-time in MET.

3.3 Educational research on learning and learning activities

Education is a diverse scientific field with a broad interest in learning. It has been described as the “nurturing if the mental capacities through witch the learners come to know, understand, judge, reflect and behave intelligently” (Pring, 2005, p. 32). Educational research generally takes an interest in the educational activity, i.e. the transactions between teachers and learners with a focus on how the learners develop their seeing and understanding towards the object of learning. Although the search found few studies in Education (n = 6), the interactional approach to study learning and learning activities is dominant in the included studies that draw on ethnographic approaches to study simulator-based training and assessment in a naturalistic way. These studies highlight the complexities of considering questions of choosing the right level of fidelity for different training purposes or facilitating simulator training. When designing simulator-based training, the degree of fidelity must meet the requirements of the work tasks and learning objectives, while also attending to the specific nature of the learners’ level of knowledge or expertise (Hontvedt 2015). Also, the work practices that take place on the bridge are heavily reliant on time, space and temporality in an intricate way that can hardly be simulated in an educational setting (Hontvent and Arnseth 2013). At the same time, they argue that the practice of simulator training is closely intertwined with the maritime profession’s way of organising and perceiving the world. Their results support research on simulations in the healthcare domain viewing realism as an interactional matter as well as an instructional concern. Hence, simulators showed clear potential for learning, but these studies have shown that how the training is organised and carried out far exceeds the simulator.

Kobayashi (2005) studied maritime policy and documents to draw conclusions about simulators and their use for training and assessment competences involved in safe navigation and came to the conclusion that simulators are well suited for training and assessing the competencies involved in safe navigation if they are used properly. Studies of the actual application of simulator-based assessments point towards serious problems regarding the use of competency tests in the MET system. Emad and Roth (2008) comes to the conclusion that not only is the learning objectives not fulfilled in the MET system, the assessment system has changed the learning objectives. Instead of striving to learn the skills and knowledge required on board ships, the objective in the current MET system is to pass competency tests. In fact, Gekara et al. (2011) are warning that the haphazard ways in which assessments are currently being implemented pose a possible safety hazard for the shipping industry. Moreover, Sampson et al. (2011) discovered that maritime instructors lack knowledge and are very uncertain of how to make assessments of competency in the simulator. The results of these studies are not surprising. It is far from evident how to conduct assessment based on observation of actions in the simulated environments and the instructors’ concern connects to longstanding pedagogical debates about the character of knowledge in action. Consequently, more studies are needed in order to provide guidelines for simulator-based assessments of competency to ensure validity and reliability of the assessment methods or models.

4 Conclusions and further directions

The aim of the current systematic literature review is to give an overview of the field at large, its main interests and an overview of some important findings. It should be considered a guide to the literature for its reader rather than to provide any deeper analysis on different issues, as for example the questions regarding STCW identified in the synthesis (cf. Bearman et al. 2012). Following this approach, the current systematic literature review found 34 articles published in a wide range of academic journals, displaying a global field of research consisting of three main disciplines: Maritime professionals (n = 15), Human factors (n = 13) and Education (n = 6). Hence, simulator-based maritime training seems to be a rather small and quite diverse field of research, and several of the studies found lack empirical data as a basis for analysis (n = 13). The main result of the systematic literature review is that there are more questions than answers regarding the use of simulators in bridge operation training. While initial results point towards simulators as useful for training both procedural and non-technical skills, results also point towards the need for skilled instruction during training since “the simulation far exceeds the simulator” to lend the words of Hontvedt and Arnseth (2013). It is also important to take seriously the empirical results from Emad and Roth (2008) and Gekara et al. (2011) that shows that there is potential advantages of simulator-based training and assessment, but that they are currently being poorly implemented, which poses possible safety hazards for the shipping industry. Since MET institutions train their students for one of the most safety-critical industries in the world, there is a need for empirical studies on these questions to enhance the quality of training and assessment and lay the foundation for an evidence-based practice for simulation-based training of seafarers.