Skip to main content

Simulator Evaluation

  • Chapter
  • First Online:
Effective Training of Arthroscopic Skills

Abstract

Validation is a very important, but not the sole criterion based upon which simulators should be evaluated. In this chapter, we propose three sets of evaluation criteria to assess the appropriateness of simulators to train arthroscopic skills: wishes from the arthroscopic community (Chap. 2), general requirements for medical simulators and validation (Chap. 8). The first two sets of criteria are elucidated in the remainder of this section; the latter is fully covered in Chap. 8. For the simulators presented in Chaps. 5, 6, and 7, we evaluate to what extent they fulfil these three sets of criteria. This will be done using a 3-point Likert scale: + implies the simulator completely fulfils a requirement, ~ implies that the simulator fulfils a requirement to some extent and – implies that the simulator does not fulfil a requirement. Whenever possible, the evaluation is performed per type of simulator, for example, high-fidelity virtual reality simulators or box trainers (see classification Chap. 6).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We have interpreted this requirement as the simulator offering a high usability that is being ‘user friendly’, which implies that no manual is required to handle the simulator, and ‘easy to use’, which implies that no preparation time is required to start training.

  2. 2.

    As indicated, simulator validation will be discussed in a separate section, where the literature has been reported most extensively on this requirement.

Bibliography

  • Andersen C, Winding TN, Vesterby MS (2011) Development of simulated arthroscopic skills. Acta Orthop 82(1):90–95, available from: PM:21281257

    Article  PubMed Central  PubMed  Google Scholar 

  • Bayona S, Fernandez-Arroyo JM, Martin I, Bayona P (2008) Assessment study of insightARTHRO VR arthroscopy virtual training simulator: face, content, and construct validities. J Robotic Surg 2(3):151–158

    Article  Google Scholar 

  • Bliss JP, Hanner-Bailey HS, Scerbo MW (2005) Determining the efficacy of an immersive trainer for arthroscopy skills. Stud Health Technol Inform 111:54–56, available from: PM:15718698

    PubMed  Google Scholar 

  • Butler A, Olson T, Koehler R, Nicandri G (2013) Do the skills acquired by novice surgeons using anatomic dry models transfer effectively to the task of diagnostic knee arthroscopy performed on cadaveric specimens? J Bone Joint Surg Am 95(3):e15–e18, available from: PM:23389795

    Article  PubMed  Google Scholar 

  • Buzink SN, Christie LS, Goossens RH, de Ridder H, Jakimowicz JJ (2010) Influence of anatomic landmarks in the virtual environment on simulated angled laparoscope navigation. Surg Endosc 24(12):2993–3001, available from: PM:20419318

    Article  PubMed Central  PubMed  Google Scholar 

  • Cannon WD, Nicandri GT, Reinig K, Mevis H, Wittstein J (2014) Evaluation of skill level between trainees and community orthopaedic surgeons using a virtual reality arthroscopic knee simulator. J Bone Joint Surg Am 96(7):e57, available from: PM:24695934

    Article  PubMed  Google Scholar 

  • Ceponis PJ, Chan D, Boorman RS, Hutchison C, Mohtadi NG (2007) A randomized pilot validation of educational measures in teaching shoulder arthroscopy to surgical residents. Can J Surg 50(5):387–393, available from: PM:18031640

    PubMed Central  PubMed  Google Scholar 

  • Cheetham M, Jancke L (2013) Perceptual and category processing of the Uncanny Valley hypothesis’ dimension of human likeness: some methodological issues. J Vis Exp (76), available from: PM:23770728

    Google Scholar 

  • Escoto A, Le BF, Trejos AL, Naish MD, Patel RV, Lebel ME (2013) A knee arthroscopy simulator: design and validation. Conf Proc IEEE Eng Med Biol Soc 2013:5715–5718, available from: PM:24111035

    PubMed  Google Scholar 

  • Frank RM, Erickson B, Frank JM, Bush-Joseph CA, Bach BR Jr, Cole BJ, Romeo AA, Provencher MT, Verma NN (2014) Utility of modern arthroscopic simulator training models. Arthroscopy 30(1):121–133, available from: PM:24290789

    Article  PubMed  Google Scholar 

  • Gomoll AH, O'Toole RV, Czarnecki J, Warner JJ (2007) Surgical experience correlates with performance on a virtual reality simulator for shoulder arthroscopy. Am J Sports Med 35(6):883–888, available from: PM:17261572

    Article  PubMed  Google Scholar 

  • Gomoll AH, Pappas G, Forsythe B, Warner JJ (2008) Individual skill progression on a virtual reality simulator for shoulder arthroscopy: a 3-year follow-up study. Am J Sports Med 36(6):1139–1142, available from: PM:18326032

    Article  PubMed  Google Scholar 

  • Henn RF III, Shah N, Warner JJ, Gomoll AH (2013) Shoulder arthroscopy simulator training improves shoulder arthroscopy performance in a cadaveric model. Arthroscopy 29(6):982–985, available from: PM:23591380

    Article  PubMed Central  PubMed  Google Scholar 

  • Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65–70

    Google Scholar 

  • Howells NR, Auplish S, Hand GC, Gill HS, Carr AJ, Rees JL (2009) Retention of arthroscopic shoulder skills learned with use of a simulator. Demonstration of a learning curve and loss of performance level after a time delay. J Bone Joint Surg Am 91(5):1207–1213, available from: PM:19411470

    Article  CAS  PubMed  Google Scholar 

  • Howells NR, Brinsden MD, Gill RS, Carr AJ, Rees JL (2008a) Motion analysis: a validated method for showing skill levels in arthroscopy. Arthroscopy 24(3):335–342, available from: PM:18308187

    Article  PubMed  Google Scholar 

  • Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL (2008b) Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. J Bone Joint Surg Br 90(4):494–499, available from: PM:18378926

    Article  CAS  PubMed  Google Scholar 

  • Hurmusiadis V, Rhode K, Schaeffter T, Sherman K (2011) Virtual arthroscopy trainer for minimally invasive surgery. Stud Health Technol Inform 163:236–238, available from: PM:21335795

    PubMed  Google Scholar 

  • Issenberg SB, McGaghie WC, Petrusa ER, Lee GD, Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 27(1):10–28, available from: PM:16147767

    Article  PubMed  Google Scholar 

  • Jackson WF, Khan T, Alvand A, Al-Ali S, Gill HS, Price AJ, Rees JL (2012) Learning and retaining simulated arthroscopic meniscal repair skills. J Bone Joint Surg Am 94(17):e132, available from: PM:22992861

    Article  CAS  PubMed  Google Scholar 

  • MacDorman K. Androids as an experimental apparatus: why is there an uncanny valley and can we exploit it? In: Toward Social Mechanisms of Android Science: A CogSci 2005 Workshop, pp 106–118

    Google Scholar 

  • Martin KD, Belmont PJ, Schoenfeld AJ, Todd M, Cameron KL, Owens BD (2011) Arthroscopic basic task performance in shoulder simulator model correlates with similar task performance in cadavers. J Bone Joint Surg Am 93(21):e1271–e1275, available from: PM:22048106

    Article  PubMed  Google Scholar 

  • Martin KD, Cameron K, Belmont PJ, Schoenfeld A, Owens BD (2012) Shoulder arthroscopy simulator performance correlates with resident and shoulder arthroscopy experience. J Bone Joint Surg Am 94(21):e160, available from: PM:23138247

    Article  PubMed  Google Scholar 

  • McCarthy AD, Hollands RJ (1998) A commercially viable virtual reality knee arthroscopy training system. Stud Health Technol Inform 50:302–308, available from: PM:10180558

    CAS  PubMed  Google Scholar 

  • McCarthy AD, Moody L, Waterworth AR, Bickerstaff DR (2006) Passive haptics in a knee arthroscopy simulator: is it valid for core skills training? Clin Orthop Relat Res 442:13–20, available from: PM:16394733

    Article  PubMed  Google Scholar 

  • McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ (2010) A critical review of simulation-based medical education research: 2003-2009. Med Educ 44(1):50–63, available from: PM:20078756

    Article  PubMed  Google Scholar 

  • Megali G, Tonet O, Mazzoni M, Dario P, Vascellari A, Marcacci M (2002) A new tool for surgical training in knee arthroscopy. In: Dohi T, Kikinis R (eds), Tokio, pp 170–177

    Google Scholar 

  • Megali G, Tonet O, Dario P, Vascellari A, Marcacci M (2005) Computer-assisted training system for knee arthroscopy. Int J Med Robot 1(3):57–66, available from: PM:17518391

    Article  CAS  PubMed  Google Scholar 

  • Modi CS, Morris G, Mukherjee R (2010) Computer-simulation training for knee and shoulder arthroscopic surgery. Arthroscopy 26(6):832–840, available from: PM:20511043

    Article  PubMed  Google Scholar 

  • Moody L, Waterworth A, McCarthy AD, Harley P, Smallwood R (2008) The feasibility of a mixed reality surgical training environment. Virtual Real 12:77–86

    Article  Google Scholar 

  • O'Neill PJ, Cosgarea AJ, Freedman JA, Queale WS, McFarland EG (2002) Arthroscopic proficiency: a survey of orthopaedic sports medicine fellowship directors and orthopaedic surgery department chairs. Arthroscopy 18(7):795–800, available from: PM:12209439

    Article  PubMed  Google Scholar 

  • Olson T, Koehler R, Butler A, Amsdell S, Nicandri G (2013) Is there a valid and reliable assessment of diagnostic knee arthroscopy skill? Clin Orthop Relat Res 471(5):1670–1676, available from: PM:23254692

    Article  PubMed Central  PubMed  Google Scholar 

  • Pedowitz RA, Esch J, Snyder S (2002) Evaluation of a virtual reality simulator for arthroscopy skills development. Arthroscopy 18(6):E29, available from: PM:12098111

    Article  PubMed  Google Scholar 

  • Pollard TC, Khan T, Price AJ, Gill HS, Glyn-Jones S, Rees JL (2012) Simulated hip arthroscopy skills: learning curves with the lateral and supine patient positions: a randomized trial. J Bone Joint Surg Am 94(10):e68, available from: PM:22617934

    Article  PubMed  Google Scholar 

  • Safir O, Dubrowski A, Mirsky L, Lin C, Backstein D, Carnahan A (2008) What skills should simulation training in arthroscopy teach residents? Int J Comp Assist Radiol Surg 3(5):433–437

    Article  Google Scholar 

  • Sherman KP, Ward JW, Wills DP, Sherman VJ, Mohsen AM (2001) Surgical trainee assessment using a VE knee arthroscopy training system (VE-KATS): experimental results. Stud Health Technol Inform 81:465–470, available from: PM:11317792

    CAS  PubMed  Google Scholar 

  • Slade Shantz JA, Leiter JR, Gottschalk T, MacDonald PB (2014) The internal validity of arthroscopic simulators and their effectiveness in arthroscopic education. Knee Surg SportsTraumatol Arthrosc 22(1):33–40, available from: PM:23052120

    Article  Google Scholar 

  • Smith S, Wan A, Taffinder N, Read S, Emery R, Darzi A (1999) Early experience and validation work with Procedicus V--he Prosolvia virtual reality shoulder arthroscopy trainer. Stud HealthTechnol Inform 62:337–343, available from: PM:10538383

    Google Scholar 

  • Srivastava S, Youngblood PL, Rawn C, Hariri S, Heinrichs WL, Ladd AL (2004) Initial evaluation of a shoulder arthroscopy simulator: establishing construct validity. J Shoulder Elbow Surg 13(2):196–205, available from: PM:14997099

    Article  PubMed  Google Scholar 

  • Tashiro Y, Miura H, Nakanishi Y, Okazaki K, Iwamoto Y (2009) Evaluation of skills in arthroscopic training based on trajectory and force data. Clin Orthop Relat Res 467((2):546–552, available from: PM:18791774

    Article  PubMed Central  PubMed  Google Scholar 

  • Tuijthof GJ, van Sterkenburg MN, Sierevelt IN, Van OJ, van Dijk CN, Kerkhoffs GM (2010a) First validation of the PASSPORT training environment for arthroscopic skills. Knee Surg Sports Traumatol Arthrosc 18(2)):218–224, available from: PM:19629441

    Article  PubMed  Google Scholar 

  • Tuijthof GJ, Visser P, Sierevelt IN, van Dijk CN, Kerkhoffs GM (2011) Does perception of usefulness of arthroscopic simulators differ with levels of experience? Clin Orthop Relat Res, available from: PM:21290203

    Google Scholar 

  • Tuijthof G, de Wit M, Horeman T, Kerkhoffs G. Evaluation of PASSPORT v2, 22th Annual meeting of the Dutch Arthroscopy Association, ed., Den Bosch

    Google Scholar 

  • Tuijthof G, Sierevelt IN, Dijk C, Kerkhoffs G (2010b) Is the presence of a dummy leg essential in the SIMENDO arthroscopy trainer? Knee Surg Sport Tr A 18(1):16

    Google Scholar 

Ref Type: Abstract

  • Vitale MA, Kleweno CP, Jacir AM, Levine WN, Bigliani LU, Ahmad CS (2007) Training resources in arthroscopic rotator cuff repair. J Bone Joint Surg Am 89(6):1393–1398, available from: PM:17545443

    Article  PubMed  Google Scholar 

  • Zivanovic A, Dibble E, Davies B, Moody L, Waterworth A (2003) Engineering requirements for a haptic simulator for knee arthroscopy training. Stud Health Technol Inform 94:413–418, available from: PM:15455938

    PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gabriëlle J. M. Tuijthof .

Editor information

Editors and Affiliations

Appendix 9.A Questionnaire Face Validity and Usability

Appendix 9.A Questionnaire Face Validity and Usability

The questionnaire will remain anonymous! Please fill in all the questions by encircling one number ranging from 0 to 10, much as you would score an exam. Encircling N/A if the question does not apply to you. Encircling one of the options that applies to you and filling in the boxes if needed.

figure afigure afigure afigure afigure afigure afigure afigure a

Rights and permissions

Reprints and permissions

Copyright information

© 2015 ESSKA

About this chapter

Cite this chapter

Tuijthof, G.J.M., Rees, J.L. (2015). Simulator Evaluation. In: Karahan, M., Kerkhoffs, G., Randelli, P., Tuijthof, G. (eds) Effective Training of Arthroscopic Skills. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44943-1_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-44943-1_9

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-44942-4

  • Online ISBN: 978-3-662-44943-1

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics