Skip to main content
SpringerLink
Log in

The internal validity of arthroscopic simulators and their effectiveness in arthroscopic education

Knee Surgery, Sports Traumatology, Arthroscopy Aims and scope Submit manuscript

Cite this article

Abstract

Purpose

The purpose of this systematic review was to identify standard procedures for the validation of arthroscopic simulators and determine whether simulators improve the surgical skills of users.

Methods

Arthroscopic simulator validation studies and randomized trials assessing the effectiveness of arthroscopic simulators in education were identified from online databases, as well as, grey literature and reference lists. Only validation studies and randomized trials were included for review. Study heterogeneity was calculated and where appropriate, study results were combined employing a random effects model.

Results

Four hundred and thirteen studies were reviewed. Thirteen studies met the inclusion criteria assessing the construct validity of simulators. A pooled analysis of internal validation studies determined that simulators could discriminate between novice and experts, but not between novice and intermediate trainees on time of completion of a simulated task. Only one study assessed the utility of a knee simulator in training arthroscopic skills directly and demonstrated that the skill level of simulator-trained residents was greater than non-simulator-trained residents.

Conclusions

Excessive heterogeneity exists in the literature to determine the internal and transfer validity of arthroscopic simulators currently available. Evidence suggests that simulators can discriminate between novice and expert users, but discrimination between novice and intermediate trainees in surgical education should be paramount. International standards for the assessment of arthroscopic simulator validity should be developed to increase the use and effectiveness of simulators in orthopedic surgery.

Level of evidence

Diagnostic study, Level III.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA III, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193(6):797–804

    Article  PubMed  Google Scholar 

  2. Atesok K, Mabrey JD, Jazrawi LM, Egol KA (2012) Surgical simulation in orthopaedic skills training. J Am Acad Orthop Surg 20(7):410–422

    Article  PubMed  Google Scholar 

  3. Babineau TJ, Becker J, Gibbons G, Sentovich S, Hess D, Robertson S, Stone M (2004) The “cost” of operative training for surgical residents. Arch Surg 139(4):366–369

    Article  PubMed  Google Scholar 

  4. Barnes RW (1987) Surgical handicraft: teaching and learning surgical skills. Am J Surg 153(5):422–427

    Article  CAS  PubMed  Google Scholar 

  5. Berwick DM, Leape LL (1999) Reducing errors in medicine. BMJ 319(7203):136–137

    Article  CAS  PubMed  Google Scholar 

  6. Bridges M, Diamond DL (1999) The financial impact of teaching surgical residents in the operating room. Am J Surg 177(1):28–32

    Article  CAS  PubMed  Google Scholar 

  7. Cannon WD, Eckhoff DG, Garrett WE Jr, Hunter RE, Sweeney HJ (2006) Report of a group developing a virtual reality simulator for arthroscopic surgery of the knee joint. Clin Orthop Relat Res 442:21–29

    Article  PubMed  Google Scholar 

  8. Ceponis PJ, Chan D, Boorman RS, Hutchison C, Mohtadi NG (2007) A randomized pilot validation of educational measures in teaching shoulder arthroscopy to surgical residents. Can J Surg 50(5):387–393

    PubMed Central  PubMed  Google Scholar 

  9. Deziel DJ, Millikan KW, Economou SG, Doolas A, Ko ST, Airan MC (1993) Complications of laparoscopic cholecystectomy: a national survey of 4,292 hospitals and an analysis of 77,604 cases. Am J Surg 165(1):9–14

    Article  CAS  PubMed  Google Scholar 

  10. Farnworth LR, Lemay DE, Wooldridge T, Mabrey JD, Blaschak MJ, DeCoster TA, Wascher DC, Schenck RC Jr (2001) A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: the financial impact of orthopaedic surgical training in the operating room. Iowa Orthop J 21:31–35

    CAS  PubMed Central  PubMed  Google Scholar 

  11. Gomoll AH, O’Toole RV, Czarnecki J, Warner JJ (2007) Surgical experience correlates with performance on a virtual reality simulator for shoulder arthroscopy. Am J Sports Med 35(6):883–888

    Article  PubMed  Google Scholar 

  12. Gomoll AH, Pappas G, Forsythe B, Warner JJ (2008) Individual skill progression on a virtual reality simulator for shoulder arthroscopy: a 3-year follow-up study. Am J Sports Med 36(6):1139–1142

    Article  PubMed  Google Scholar 

  13. Hariri S, Rawn C, Srivastava S, Youngblood P, Ladd A (2004) Evaluation of a surgical simulator for learning clinical anatomy. Med Educ 38(8):896–902

    Article  CAS  PubMed  Google Scholar 

  14. Howells NR, Brinsden MD, Gill RS, Carr AJ, Rees JL (2008) Motion analysis: a validated method for showing skill levels in arthroscopy. Arthroscopy 24(3):335–342

    Article  PubMed  Google Scholar 

  15. Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL (2008) Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. J Bone Joint Surg Br 90(4):494–499

    Article  CAS  PubMed  Google Scholar 

  16. Insel A, Carofino B, Leger R, Arciero R, Mazzocca AD (2009) The development of an objective model to assess arthroscopic performance. J Bone Joint Surg Am 91(9):2287–2295

    Article  PubMed  Google Scholar 

  17. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ (1996) Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 17(1):1–12

    Article  CAS  PubMed  Google Scholar 

  18. James PB, Hope SH-B, Mark WS (2005) Determining the efficacy of an immersive trainer for arthroscopy skills. In: MMVR13, Amsterdam, Netherlands, 1999. Medicine meets virtual reality. The magical next becomes the medical now. IOS press, pp 54–56

  19. Kauvar DS, Braswell A, Brown BD, Harnisch M (2006) Influence of resident and attending surgeon seniority on operative performance in laparoscopic cholecystectomy. J Surg Res 132(2):159–163

    Article  PubMed  Google Scholar 

  20. Mabrey JD, Reinig KD, Cannon WD (2010) Virtual reality in orthopaedics: is it a reality? Clin Orthop Relat Res 468:2586–2591

    Article  PubMed  Google Scholar 

  21. McCarthy A, Harley P, Smallwood R Virtual arthroscopy training: do the virtual skills developed match the real skills required? In: MMVR 7, Amsterdam, Netherlands, 1999. Medicine meets virtual reality. Convergence of physical and informational technologies: Options for a New Era in Healthcare. IOS Press, pp 221–227

  22. McCarthy AD, Moody L, Waterworth AR, Bickerstaff DR (2006) Passive haptics in a knee arthroscopy simulator: is it valid for core skills training? Clin Orthop Relat Res 442:13–20

    Article  PubMed  Google Scholar 

  23. Megali G, Tonet O, Mazzoni M, Dario P, Vascellari A, Marcacci M (2002) A new tool for surgical training in knee arthroscopy. In: Proceedings of the part II, Berlin, Germany, 2002. Medical image computing and computer-assisted intervention—MICCAI 2002. 5th international conference. Springer, pp 170–177

  24. Modi CS, Morris G, Mukherjee R (2010) Computer-simulation training for knee and shoulder arthroscopic surgery. Arthroscopy 26(6):832–840

    Article  PubMed  Google Scholar 

  25. Moorthy K, Munz Y, Sarker SK, Darzi A (2003) Objective assessment of technical skills in surgery. BMJ 327(7422):1032–1037

    Article  PubMed  Google Scholar 

  26. Pedowitz RA, Esch J, Snyder S (2002) Evaluation of a virtual reality simulator for arthroscopy skills development. Arthroscopy 18(6):E29

    Article  PubMed  Google Scholar 

  27. Review Manager (RevMan) [Computer program] (2011) Version 5.1. Copenhagen: The Nordic Cochrane Centre, The cochrane collaboration

  28. Schulz KF, Altman DG, Moher D (2010) CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ 340:c332

    Article  PubMed Central  PubMed  Google Scholar 

  29. Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236(4):458–463

    Article  PubMed  Google Scholar 

  30. Sherman KP, Ward JW, Wills DP, Sherman VJ, Mohsen AM (2001) Surgical trainee assessment using a VE knee arthroscopy training system (VE-KATS): experimental results. Stud Health Technol Inform. 81:465–470

    CAS  PubMed  Google Scholar 

  31. Sidhu RS, Park J, Brydges R, MacRae HM, Dubrowski A (2007) Laboratory-based vascular anastomosis training: a randomized controlled trial evaluating the effects of bench model fidelity and level of training on skill acquisition. J Vasc Surg 45(2):343–349

    Article  PubMed  Google Scholar 

  32. Smith S, Wan A, Taffinder N, Read S, Emery R, Darzi A (1999) Early experience and validation work with Procedicus VA—the Prosolvia virtual reality shoulder arthroscopy trainer. Studies In Health Technology And Informatics 62:337–343

    CAS  PubMed  Google Scholar 

  33. Srivastava S, Youngblood PL, Rawn C, Hariri S, Heinrichs WL, Ladd AL (2004) Initial evaluation of a shoulder arthroscopy simulator: establishing construct validity. J Shoulder Elbow Surg 13(2):196–205

    Article  PubMed  Google Scholar 

  34. Strom P, Kjellin A, Hedman L, Johnson E, Wredmark T, Fellander-Tsai L (2003) Validation and learning in the Procedicus KSA virtual reality surgical simulator. Surg Endosc 17(2):227–231

    Article  CAS  PubMed  Google Scholar 

  35. Strom P, Kjellin A, Hedman L, Wredmark T, Fellander-Tsai L (2004) Training in tasks with different visual-spatial components does not improve virtual arthroscopy performance. Surg Endosc 18(1):115–120

    Article  CAS  PubMed  Google Scholar 

  36. Tashiro Y, Miura H, Nakanishi Y, Okazaki K, Iwamoto Y (2009) Evaluation of skills in arthroscopic training based on trajectory and force data. Clin Orthop Relat Res 467(2):546–552

    Article  PubMed  Google Scholar 

  37. Tuijthof GJM, van Sterkenburg MN, Sierevelt IN, van Oldenrijk J, Van Dijk CN, Kerkhoffs GMMJ (2010) First validation of the PASSPORT training environment for arthroscopic skills. Knee Surg Sports Traumatol Arthrosc 18(2):218–224

    Article  PubMed  Google Scholar 

Download references

Conflict of interest

None.

Author information

Authors and Affiliations

  1. Department of Orthopaedic Surgery, University of California, San Francisco, CA, USA

    Jesse Alan Slade Shantz

  2. Department of Surgery, Section of Orthopedics, University of Manitoba, Winnipeg, Canada

    Jeff R. S. Leiter & Peter Benjamin MacDonald

  3. Department of Human Anatomy and Cell Science, University of Manitoba, Winnipeg, Canada

    Jeff R. S. Leiter

  4. Neil John Maclean Health Sciences Library, University of Manitoba, 239 Brodie Centre, 750 McDermot Ave, Winnipeg, MB, R3T 2N2, Canada

    Tania Gottschalk

  5. Pan Am Clinic, 75 Poseidon Bay, Winnipeg, MB, R3M 3E4, Canada

    Jesse Alan Slade Shantz, Jeff R. S. Leiter & Peter Benjamin MacDonald

  6. San Francisco General Hospital, Orthopaedic Trauma Institute, 2550 23rd Street, San Francisco, CA, 94110, USA

    Jesse Alan Slade Shantz

Authors
  1. Jesse Alan Slade Shantz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeff R. S. Leiter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tania Gottschalk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Benjamin MacDonald
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jesse Alan Slade Shantz.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 124 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Slade Shantz, J.A., Leiter, J.R.S., Gottschalk, T. et al. The internal validity of arthroscopic simulators and their effectiveness in arthroscopic education. Knee Surg Sports Traumatol Arthrosc 22, 33–40 (2014). https://doi.org/10.1007/s00167-012-2228-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00167-012-2228-7

Keywords

search

Navigation