Introduction

In the previous chapter, we have described virtual reality (VR) and augmented reality (AR) simulators of endoscopic sinus surgery (ESS). An ideal model for a training or fellowship programme would score highly in all standard measures of validity whilst maintaining an attainable cost. This is a challenging goal that is worthy of pursuit given that simulation training has been shown to be cost-effective option in other domains [1]. Non-AR or VR models are attractive ways to fill gaps in simulation training whilst reducing costs given the overheads associated with computer-based models [1].

This is now a maturing field that shows reasonable evidence for a number of models and the incorporation of these models into real training programmes requires study to ensure that training opportunities are being maximised. In previous chapter, we described behaviours of ideal ESS simulator, ways of assessment and comparison using standard measures of validity.

In this chapter, physical (i.e. non-computer based) models of ESS will be evaluated in order to clarify their utility in training ENT surgeons.

Cadaver Dissection

Cadaver dissection was the cornerstone of gross anatomy teaching for centuries and constitutes a fundamental component of surgical education. There is no doubt about its significance to the understanding of the 3D position and relationship of all anatomical structures [2, 3]. A thorough knowledge of this 3D anatomy is for surgeons essential.

To conquer all this anatomical knowledge, cadaveric dissection yields a realistic visual representation of the identification of anatomical structures; detailed demonstration of anatomical variation; allows trainees to experience tissue handling; and provides an approximation of the operating theatre setting [4]. Dissection can also allow the practice of different approaches to the same area and different techniques to solve a problem that can improve trainee confidence.

There was a reduction in the number of hours assigned to dissection in the anatomy curriculum during recent years. The reasons are multiple: a reduction in availability of cadavers, increased costs of running dissection laboratories and a change in emphasis in favour of skills such as communication and teamwork among medical students [5, 6•]. Despite this, it is clear that cadaveric endoscopic sinus dissection courses are an excellent anatomical learning and surgical confidence-building tool [7].

In terms of limitations, cadaveric dissection has clear impediments that limit its utility in training. Cadaveric tissue is precious and increasingly difficult to obtain due to economical and legislative limitations [8]. Each structure can only be dissected once, limiting the repeatability of practice—particularly when anatomical variation is key to the task. Finally, cadaveric tissue handling will vary depending on the preservation process and cannot simulate the bleeding encountered in even the most straightforward sinonasal operation [9]. In response to the above challenges, a number of alternative simulation strategies have been developed. These are described below in increasing order of complexity.

Physical Models

Though electronic models of ESS have, from the outset, been far-reaching in their goals and quality, they carried significant limitations. The most important of these is their cost, which limited their use to well-funded centres. As a reaction to this, animal models and physical simulators (also known as bench or box simulators) were developed in order to try and obtain the complexity of virtual reality simulators whilst providing lower cost, more realistic tissue handling and better sensory feedback [10, 11]. The physical simulators tend to self-categorise into a spectrum whereby their aesthetic and haptic realism segregates them into low-, medium- and high-fidelity groups. Table 1 summarises below described models with references to validation studies.

Table 1 Validation studies
Full size table

Low-Fidelity ESS Models

Low-fidelity models lack identifiable anatomy and realistic tissue handling. They have, therefore, been proposed to teach basic instrument handling and familiarise surgeons with the unique challenges of endoscopic surgery. These draw origin from general surgical laparoscopic box trainers and the most basic of two models in this category simply attempts to teach endoscope navigation and object manipulation skills. It is, perhaps, due to this simplicity that the model fails to demonstrate construct validity when comparing junior trainees to experienced trainees [12]. This was despite a well-constructed study which validated a computer-based video assessment programme against expert human assessment on cadavers and then used the computer assessor on videos of trainees performing tasks on the trainer [12]. This computer-based assessment model was abandoned for a subsequent study that looked at performance on a cadaveric global rating scale and task-specific checklist between groups that had been randomised to simulator training with standard teaching or standard teaching alone. Counter-intuitively, senior residents performed better on the cadaveric task following simulation training whereas junior residents did not derive this benefit. In any case, the small sample size of nine in the subgroup that demonstrated a significant difference suggests that the difference could have been due to chance and the randomisation method used in the trial was not described [22]. Though not carried through to the operating theatre, this is the only trial of physical models that demonstrates any form of predictive validity, in this case from the model to the cadaver.

In contrast to the above simple model, a more elaborate gelatin-embedded model has demonstrated reasonable face and content validity following global assessment by a small sample of attending surgeons [13, 24]. This model also has construct validity for the majority of tasks assessed via global rating scales [13]. Perhaps its most useful role can be in its validated utility in training novices (medical students) [14], though it would be useful to repeat this work in junior trainees and attempt to demonstrate whether this paradigm has predictive validity—i.e. that improved performance on the task correlates with improved operative performance.

Medium-Fidelity ESS Models

Medium-fidelity models in this context attempt to approximate intranasal anatomy without high-fidelity modelling or realistic tasks. Such models were developed after the advent of high-fidelity virtual reality models of ESS as a response to the cost and access issues of such models. These seem to demonstrate that more inexperienced operators (medical students) gain skills more rapidly than medium-skilled candidates (residents) and that improvements gained in training are sustainable—for the most part—for at least 2 weeks. As with other models, the assessment in this model was not carried through to look at intra-operative performance [15].

The development of such trainers has continued over the last decade and construction approaches have tended toward 3D printing as this technology has become more ubiquitous and affordable. As such, medium-fidelity ESS models are now constructible whilst adhering to a low-fidelity model budget and reducing the active construction time of the model. Though limited by cost and technical construction issues, one such model has subsequently demonstrated good face and content validity with tasks that relate to ESS [17••]. A recent study by Yoshiyasu et al. focused on construct validity which has been successfully demonstrated [18].

It is interesting to note that, despite increasing affordability of 3D printing, not all new models utilise this technology. Furthermore, despite using more traditional methods to build their simulator, Harbison et al. were able to demonstrate face, content and construct validity for their model on a global rating skill score [16]. As described above, this means that their model was judged to measure the correct factors, represent the steps of the desired procedure and discriminate between candidates of varying experience. A univariate regression model was used to demonstrate an association between level of training and cumulative global rating scale score in their population of 26 subjects (12 medical students/interns and 14 more senior doctors including attending levels). This is in keeping with the increased rigour that has been demanded for the publication of training models in recent years [16]. However, performance correlated with grade which included medical students. As with the low-fidelity models above, it is arguable that the medical student comparison group is not of particular use in this context. In fact, based on the raw data discernible from jitter plots provided in this study, it is likely that the construct validity of this model would disappear if medical students were not included.

Despite this, the utility of transferable fundamental skills such as camera technique is likely to be useful for the medical student population as they may perform endoscopies early in their medical career. However, interventional ESS skills are not likely to be relevant to the medical student population. Therefore, in the future, it is likely that even low-medium-fidelity ESS models should be able to demonstrate construct validity in a population that will actually perform the procedure—i.e. resident grades and above—prior to implementation.

High-Fidelity ESS Models

Earlier high-fidelity ESS training models required taking a mould of sectioned formalin-fixed cadaveric tissue and then using this as a negative cast for the formation of polyurethane models. Very basic assessment of such models was performed in terms of face and content validity [19]. Such early models, in some cases, were extremely elaborate, encompassing the entire nasal cavity and paranasal sinuses up to the pituitary fossa. However, unfortunately, the quality of the model does not seem matched by the quality of the validation. It is, therefore, difficult to appreciate how useful such models are for training purposes [25, 26]. A subsequent paper describes the facility of sinus catheterisation though this does not compare performance to live ESS [27]. Finally, participants trained on this model during a course still rate the model highly 6 months later in terms of the ESS confidence that it had given them 6 months later. It is worth noting that this was a self-reported study with no objective verification of participants’ abilities following the course.

Moving into the 3D-printing era, it is clear that such models—which can even be targeted on a case-by-case basis—can attain high anatomical accuracy [28]. Unfortunately, the more basic high-fidelity models do not include soft tissue simulation as this requires more complex 3D printers. Therefore, a two-material model has been created and demonstrated content and construct validity for some—but not all—of the ESS-related tasks created for it [29]. However, this model costs more than ten times than its medium-fidelity peers and investment in this model would, therefore, be harder to justify. The same group have also created an even more elaborate model that combines multiple materials with different strengths and elasticities in order to reproduce the tissue handling characteristics of bone, skin, cartilage and mucosa. This latter model was targeted at septoplasty training rather than sinus surgery though it seems to demonstrate face and content validity. However, the construct validity is questionable for the lack of significant results in relevant metrics and lack of statistical transparency in reporting [20].

As with this septoplasty model, the remit of ESS training models is extending further and starting to focus on the skull base. This includes 3D printing and the modelling of bony pathology. However, as the focus is on the skull base and approached from a joint ENT-Neurosurgical perspective, it is possible that such models may not encompass the breadth of ENT-led procedures. Of the factors addressed in the assessment of one such model, the nasal structures were found to be the worst reproduced area of the model [30].

Summary of Physical Models/Box Trainers/ESS Models

In general, it seems that the majority of models on which material is published have achieved at least face validity and, possibly, content validity. However, a systematic review that encompassed the validation-relevant papers on the above simulators only found four papers that demonstrated the ability to discriminate between users of differing experience (construct validity) and it is therefore difficult to recommend which of the above models is best suited to training at any particular grade [26]. Ideally, head-to-head data would be available for the above models as well as predictive validity data (i.e. whether performance on the model predicts theatre performance) prior to being able to determine which—if any—model is the best.

Animal Models

Though significant effort has been put into trying to create models of the human nasal cavity and paranasal sinuses, both physical simulators and virtual reality simulators struggle to obtain a realistic tissue feel. Therefore, fresh frozen animal tissue has been utilised as this inherently has a tissue feel that is comparable with human tissue. The ovine model has the attraction of bearing some resemblance to its human equivalent following some modifications and being extremely cost-effective. The cadaveric ovine model involves shortening the muzzle of a domestic sheep head prior to the performance of a number of tasks [31]. This model subsequently demonstrated face, content and construct validity using global and task-related measures [21, 23].

Though the cadaveric ovine model has advantages in terms of tissue feel, it still does not surmount a major limitation of physical models in that it does not provide a simulation of bleeding. Some of this challenge can be surmounted by the use of live sheep to provide a high-fidelity experience whilst also mimicking the haemodynamic changes one would expect during major haemorrhage. Furthermore, this model incorporates the animal’s innate haemostatic mechanisms to increase the reality of the simulation [32, 33]. The results of the latest study by Jukes et al. suggest that participants find the course realistic, that it is able to induce a stress response and that it provides participants with an increased level of confidence in their ability to deal with major vessel bleeding in an endoscopic environment [34]. However, it is important to note that this model is a hybrid that utilises the sinonasal anatomy of a physical model coupled with the vasculature of the sheep neck following neck dissection.

The Role of e-Learning in Simulation Training

Web learning in medical education has become increasingly popular due to its advantages over traditional teaching methods such as flexible scheduling, reduced costs and individualised instruction [6]. In surgical specialties, e-learning includes the use of virtual patient cases, digital modelling and online tutorials [35].

Several papers reported on the effectiveness of e-learning methods which were implemented into a training curriculum [36]. Mendez et al. found that the use of educational video modules was associated with fewer surgical errors and less attending take over events [37].

An increasing number of platforms host learning content free of charge [38••]. Hughes et al. provide a brief synopsis of user-generated, ENT-specific educational channels on the popular website YouTube (Google, Mountain View, CA) [39]. A recent review of mobile application stores found 75 mobile apps (as of September 2014) related to resident, student and patient education within otolaryngology, head and neck surgery [40, 41]. Among these, apps like “ENT for Students”, “ENT Surgery Handbook” and “LearnENT” were highly rated among users and available on multiple platforms [42]. The content in most cases varies and was not evaluated for educational value or efficacy [43]. However, it is important to differentiate between e-learning for knowledge versus technical skill.

To this end, the efficacy of these novel educational tools was examined in a comprehensive systematic review which shows e-learning to be a powerful alternative to standard teaching techniques within otolaryngology education, for both residents and medical students. Of the 12 studies included within this review, nine reported either improved objective performance in academic or clinical measures, or no difference in performance but higher satisfaction with use of e-learning [6, 44]. However, technical skill was not assessed in these studies and, though this type of improvement has been demonstrated in other specialties, such a benefit has not been demonstrated in ENT [44].

Conclusions

Given the ongoing need for simulation training in order to compensate for limited surgical experience in training, simulation is here to stay. Different models address different aspects of the complex physical and mental skills required for individuals to perform well in the operative environment. Such training experiences should prepare the surgeon for the operation in question and should benefit the patient by reducing both operation time and complications. Unfortunately, a relatively limited body of work supports the validity of a number of the models described herein in this context.

In an era of improved computer technology and improved 3D printing, it will be beneficial for the focus in the field to shift away from the manufacture process. Future studies on ESS simulation should focus on robust demonstration of the validity of each model with particular focus on the predictive validity—the ability of a model to predict performance in the operative environment. One area, however, that would benefit from further technological advancement would be the creation of models that simulate bleeding as this would obviate the need for animal models as part of an ongoing scientific enterprise to reduce and replace animal involvement in such practices.