The purpose of this systematic review was to identify standard procedures for the validation of arthroscopic simulators and determine whether simulators improve the surgical skills of users.
Arthroscopic simulator validation studies and randomized trials assessing the effectiveness of arthroscopic simulators in education were identified from online databases, as well as, grey literature and reference lists. Only validation studies and randomized trials were included for review. Study heterogeneity was calculated and where appropriate, study results were combined employing a random effects model.
Four hundred and thirteen studies were reviewed. Thirteen studies met the inclusion criteria assessing the construct validity of simulators. A pooled analysis of internal validation studies determined that simulators could discriminate between novice and experts, but not between novice and intermediate trainees on time of completion of a simulated task. Only one study assessed the utility of a knee simulator in training arthroscopic skills directly and demonstrated that the skill level of simulator-trained residents was greater than non-simulator-trained residents.
Excessive heterogeneity exists in the literature to determine the internal and transfer validity of arthroscopic simulators currently available. Evidence suggests that simulators can discriminate between novice and expert users, but discrimination between novice and intermediate trainees in surgical education should be paramount. International standards for the assessment of arthroscopic simulator validity should be developed to increase the use and effectiveness of simulators in orthopedic surgery.
Level of evidence
Diagnostic study, Level III.
This is a preview of subscription content,to check access.
Access this article
Similar content being viewed by others
Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA III, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193(6):797–804
Atesok K, Mabrey JD, Jazrawi LM, Egol KA (2012) Surgical simulation in orthopaedic skills training. J Am Acad Orthop Surg 20(7):410–422
Babineau TJ, Becker J, Gibbons G, Sentovich S, Hess D, Robertson S, Stone M (2004) The “cost” of operative training for surgical residents. Arch Surg 139(4):366–369
Barnes RW (1987) Surgical handicraft: teaching and learning surgical skills. Am J Surg 153(5):422–427
Berwick DM, Leape LL (1999) Reducing errors in medicine. BMJ 319(7203):136–137
Bridges M, Diamond DL (1999) The financial impact of teaching surgical residents in the operating room. Am J Surg 177(1):28–32
Cannon WD, Eckhoff DG, Garrett WE Jr, Hunter RE, Sweeney HJ (2006) Report of a group developing a virtual reality simulator for arthroscopic surgery of the knee joint. Clin Orthop Relat Res 442:21–29
Ceponis PJ, Chan D, Boorman RS, Hutchison C, Mohtadi NG (2007) A randomized pilot validation of educational measures in teaching shoulder arthroscopy to surgical residents. Can J Surg 50(5):387–393
Deziel DJ, Millikan KW, Economou SG, Doolas A, Ko ST, Airan MC (1993) Complications of laparoscopic cholecystectomy: a national survey of 4,292 hospitals and an analysis of 77,604 cases. Am J Surg 165(1):9–14
Farnworth LR, Lemay DE, Wooldridge T, Mabrey JD, Blaschak MJ, DeCoster TA, Wascher DC, Schenck RC Jr (2001) A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: the financial impact of orthopaedic surgical training in the operating room. Iowa Orthop J 21:31–35
Gomoll AH, O’Toole RV, Czarnecki J, Warner JJ (2007) Surgical experience correlates with performance on a virtual reality simulator for shoulder arthroscopy. Am J Sports Med 35(6):883–888
Gomoll AH, Pappas G, Forsythe B, Warner JJ (2008) Individual skill progression on a virtual reality simulator for shoulder arthroscopy: a 3-year follow-up study. Am J Sports Med 36(6):1139–1142
Hariri S, Rawn C, Srivastava S, Youngblood P, Ladd A (2004) Evaluation of a surgical simulator for learning clinical anatomy. Med Educ 38(8):896–902
Howells NR, Brinsden MD, Gill RS, Carr AJ, Rees JL (2008) Motion analysis: a validated method for showing skill levels in arthroscopy. Arthroscopy 24(3):335–342
Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL (2008) Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study. J Bone Joint Surg Br 90(4):494–499
Insel A, Carofino B, Leger R, Arciero R, Mazzocca AD (2009) The development of an objective model to assess arthroscopic performance. J Bone Joint Surg Am 91(9):2287–2295
Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ (1996) Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 17(1):1–12
James PB, Hope SH-B, Mark WS (2005) Determining the efficacy of an immersive trainer for arthroscopy skills. In: MMVR13, Amsterdam, Netherlands, 1999. Medicine meets virtual reality. The magical next becomes the medical now. IOS press, pp 54–56
Kauvar DS, Braswell A, Brown BD, Harnisch M (2006) Influence of resident and attending surgeon seniority on operative performance in laparoscopic cholecystectomy. J Surg Res 132(2):159–163
Mabrey JD, Reinig KD, Cannon WD (2010) Virtual reality in orthopaedics: is it a reality? Clin Orthop Relat Res 468:2586–2591
McCarthy A, Harley P, Smallwood R Virtual arthroscopy training: do the virtual skills developed match the real skills required? In: MMVR 7, Amsterdam, Netherlands, 1999. Medicine meets virtual reality. Convergence of physical and informational technologies: Options for a New Era in Healthcare. IOS Press, pp 221–227
McCarthy AD, Moody L, Waterworth AR, Bickerstaff DR (2006) Passive haptics in a knee arthroscopy simulator: is it valid for core skills training? Clin Orthop Relat Res 442:13–20
Megali G, Tonet O, Mazzoni M, Dario P, Vascellari A, Marcacci M (2002) A new tool for surgical training in knee arthroscopy. In: Proceedings of the part II, Berlin, Germany, 2002. Medical image computing and computer-assisted intervention—MICCAI 2002. 5th international conference. Springer, pp 170–177
Modi CS, Morris G, Mukherjee R (2010) Computer-simulation training for knee and shoulder arthroscopic surgery. Arthroscopy 26(6):832–840
Moorthy K, Munz Y, Sarker SK, Darzi A (2003) Objective assessment of technical skills in surgery. BMJ 327(7422):1032–1037
Pedowitz RA, Esch J, Snyder S (2002) Evaluation of a virtual reality simulator for arthroscopy skills development. Arthroscopy 18(6):E29
Review Manager (RevMan) [Computer program] (2011) Version 5.1. Copenhagen: The Nordic Cochrane Centre, The cochrane collaboration
Schulz KF, Altman DG, Moher D (2010) CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ 340:c332
Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236(4):458–463
Sherman KP, Ward JW, Wills DP, Sherman VJ, Mohsen AM (2001) Surgical trainee assessment using a VE knee arthroscopy training system (VE-KATS): experimental results. Stud Health Technol Inform. 81:465–470
Sidhu RS, Park J, Brydges R, MacRae HM, Dubrowski A (2007) Laboratory-based vascular anastomosis training: a randomized controlled trial evaluating the effects of bench model fidelity and level of training on skill acquisition. J Vasc Surg 45(2):343–349
Smith S, Wan A, Taffinder N, Read S, Emery R, Darzi A (1999) Early experience and validation work with Procedicus VA—the Prosolvia virtual reality shoulder arthroscopy trainer. Studies In Health Technology And Informatics 62:337–343
Srivastava S, Youngblood PL, Rawn C, Hariri S, Heinrichs WL, Ladd AL (2004) Initial evaluation of a shoulder arthroscopy simulator: establishing construct validity. J Shoulder Elbow Surg 13(2):196–205
Strom P, Kjellin A, Hedman L, Johnson E, Wredmark T, Fellander-Tsai L (2003) Validation and learning in the Procedicus KSA virtual reality surgical simulator. Surg Endosc 17(2):227–231
Strom P, Kjellin A, Hedman L, Wredmark T, Fellander-Tsai L (2004) Training in tasks with different visual-spatial components does not improve virtual arthroscopy performance. Surg Endosc 18(1):115–120
Tashiro Y, Miura H, Nakanishi Y, Okazaki K, Iwamoto Y (2009) Evaluation of skills in arthroscopic training based on trajectory and force data. Clin Orthop Relat Res 467(2):546–552
Tuijthof GJM, van Sterkenburg MN, Sierevelt IN, van Oldenrijk J, Van Dijk CN, Kerkhoffs GMMJ (2010) First validation of the PASSPORT training environment for arthroscopic skills. Knee Surg Sports Traumatol Arthrosc 18(2):218–224
Conflict of interest
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Slade Shantz, J.A., Leiter, J.R.S., Gottschalk, T. et al. The internal validity of arthroscopic simulators and their effectiveness in arthroscopic education. Knee Surg Sports Traumatol Arthrosc 22, 33–40 (2014). https://doi.org/10.1007/s00167-012-2228-7