Surgical Endoscopy

, Volume 23, Issue 1, pp 130–139

Construct validity of the ProMIS laparoscopic simulator

  • Michael G. C. Pellen
  • Liam F. Horgan
  • J. Roger Barton
  • Stephen E. Attwood
Article

Abstract

Background

Laparoscopic surgery challenges both the surgical novice and experienced open surgeon with unique psychomotor adaptations. Surgical skills assessment has historically relied on subjective opinion and case experience. Objective performance metrics have stimulated much interest in surgical education over the last decade and proficiency-based simulation has been proposed as a paradigm shift in surgical skills training. New assessment tools must be subjected to scientific validation. This study examined the construct validity of a hybrid laparoscopic simulator with in-built motion tracking technology.

Methods

Volunteers were recruited from four experience groups (consultant surgeon, senior trainee, junior trainee, medical student). All subjects completed questionnaires and three tasks on the ProMIS laparoscopic simulator (laparoscope orientation, object positioning, sharp dissection). Motion analysis data was obtained via optical tracking of instrument movements. Objective metrics included time, path length (economy of movement), smoothness (controlled handling) and observer-recorded penalty scores.

Results

One hundred and sixty subjects completed at least one of the three tasks. Significant group differences were confirmed for number of years qualified, age and case experience. Significant differences were found between experts and novices in all three tasks. Sharp dissection was the strongest discriminator of four recognised laparoscopic skill groups: consultants outperformed students and juniors in all three performance metrics and objective penalty score (p < 0.05), and only accuracy of dissection did not distinguish them from senior trainees (p = 0.261). Seniors dissected faster, more efficiently and more accurately than juniors and students (p < 0.05).

Conclusions

ProMIS provides a construct valid laparoscopic simulator and is a feasible tool to assess skills in a cross-section of surgical experience groups. ProMIS has the potential to objectively measure pre-theatre dexterity practice until an agreed proficiency level of dexterity is achieved. Future work should now examine whether training to expert criterion levels on ProMIS correlates with actual operative performance.

Keywords

Laparoscopy Simulation Training Validity Assessment 

References

  1. 1.
    Carlin AM, Gasevic E, Shepard AD (2007) Effect of the 80-hour work week on resident operative experience in general surgery. Am J Surg 193:326–330PubMedCrossRefGoogle Scholar
  2. 2.
    Chikwe J, de Souza A (2004) Pepper JR (2004) No time to train surgeons. BMJ 328:418–419PubMedCrossRefGoogle Scholar
  3. 3.
    Heath RM, Gate T, Halloran C, Callaghan M, Paraoan M, Blair SD (2005) The European Working Time Directive ‘triple whammy’: hitting surgical trainees where it hurts. Br J Surg 92(Supp 1):113Google Scholar
  4. 4.
    Modernising Medical Careers: the response of the four UK Health Ministers to the consultation on Unfinished Business: Proposals for Reform of the Senior House Officer Grade. 26-2-0003Google Scholar
  5. 5.
    UK Health Departments. Modernising Medical Careers: the next steps. 15-4-2004. London, Department of HealthGoogle Scholar
  6. 6.
  7. 7.
    Paisley AM, Baldwin PJ, Patterson-Brown S (2001) Validity of surgical simulation for the assessment of operative skill. Br J Surg 88:1525–1532PubMedCrossRefGoogle Scholar
  8. 8.
    Darzi A, Mackay S (2001) Assessment of surgical competence. Qual Health Care 10:ii64–ii69PubMedCrossRefGoogle Scholar
  9. 9.
    Figert P, Park A, Witkze D, Schwartz R (2001) Transfer of training in acquiring laparoscopic skills. J Am Coll Surg 193:534–537CrossRefGoogle Scholar
  10. 10.
    Satava RM (1993) Virtual reality surgical simulator: the first steps. Surg Endosc 7:203–205PubMedCrossRefGoogle Scholar
  11. 11.
    Gallagher AG, Ritter EM, Champion HR, Higgins G, Fried GM, Moses G, Smith CD, Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241:364–372PubMedCrossRefGoogle Scholar
  12. 12.
    Gallagher AG, Ritter EM, Satava RM (2003) Fundamental principles of validation and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 17:1525–1529PubMedCrossRefGoogle Scholar
  13. 13.
    Streiner D, Norman G (1989) Health measurement scales. New York, Oxford University PressGoogle Scholar
  14. 14.
    Derossis AM, Fried GM, Abrahamowicz MA, Sigman HH, Barkun JS, Meakins JL (1998) Development of a model for training and evaluation of laparoscopic skills. Am J Surg 175:482–487PubMedCrossRefGoogle Scholar
  15. 15.
    Gallagher AG, Lederman AB, McGlade K, Satava RM, Smith CD (2004) Discriminitive validity of the Minimally Invasive Surgical Trainer in Virtual Reality (MIST VR) using criteria levels based on expert performance. Surg Endosc 18:660–665PubMedCrossRefGoogle Scholar
  16. 16.
    Schijven MP, Jakimowicz J (2003) Construct validity: experts and novices performing on the Xitact LS500 laparoscopy simulator. Surg Endosc 17:803–810PubMedCrossRefGoogle Scholar
  17. 17.
    Royal College of Surgeons of England (2006) Inquiry into independent sector treatment centres. Ann R Coll Surg 88(Suppl):116–119Google Scholar
  18. 18.
    Cruelty to Animals Act. 39 and 40 Vict, 1–8. 1876. Ch77Google Scholar
  19. 19.
    Mahadevan V (2006) The Human Tissue Act 2004: implications for anatomical work at the college. Bulletin of the Royal College of Surgeons of England 88:264–265CrossRefGoogle Scholar
  20. 20.
    Reznick R, MacRae H (2006) Teaching surgical skills—changes in the wind. N Engl J Med 355:2664–2669PubMedCrossRefGoogle Scholar
  21. 21.
    Gaba D (2004) The future vision of simulation in healthcare. Qual Safety Healthcare 13:2–10CrossRefGoogle Scholar
  22. 22.
    Kneebone R (2003) Simulation in surgical training: educational issues and practical implications. Med Educ 37:267–277PubMedCrossRefGoogle Scholar
  23. 23.
    Scott D, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, Euhus D, Jeyarajah R, Thompson W (2000) Jones DB (2000) Laparoscopic training on bench models: better and more cost effective than operating room experience. J Am Coll Surg 191:272–283PubMedCrossRefGoogle Scholar
  24. 24.
    Rosser JC, Rosser LE, Savalgi RS (1997) Skill acquisition and assessment for laparoscopic surgery. Arch Surg 132:200–204PubMedGoogle Scholar
  25. 25.
    Smith CD, Tung P (1998) Assessing laparoscopic manipulative skill: beyond the stopwatch. Surg Endosc 12:492Google Scholar
  26. 26.
    Schmidt R, Wrisberg C (2000) Motor Learning and Performance. Champaign, IL, Human KineticsGoogle Scholar
  27. 27.
    Moynihan BGA (1920) Ritual of a surgical operation. Br J Surg 8:27–35CrossRefGoogle Scholar
  28. 28.
    Adrales G, Donnelly M, Chu U, Witkze D, Hoskins J, Mastrangelo M Jr, Gandsas A, Park A (2004) Determinants of competency judgments by experienced laparoscopic surgeons. Surg Endosc 18:323–327PubMedCrossRefGoogle Scholar
  29. 29.
    Martin J, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278PubMedCrossRefGoogle Scholar
  30. 30.
    Moorthy K, Munz Y, Dosis A, Bello F, Chang A, Darzi A (2004) Bimodal assessment of laparoscopic suturing skills: construct and concurrent validity. Surg Endosc 18:1608–1612PubMedGoogle Scholar
  31. 31.
    Broe D, Ridgway P, Johnson S, Tierney S, Conlon C (2006) Construct validation of a novel hybrid surgical simulator. Surg Endosc 20:900–904PubMedCrossRefGoogle Scholar
  32. 32.
    Gallagher AG, Satava RM (2002) Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Surg Endosc 16:1746–1752PubMedCrossRefGoogle Scholar
  33. 33.
    Gallagher AG, Smith CD, Bowers SP, Seymour NE, Pearson A, McNatt S, Hananel D, Satava RM (2003) Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures. J Am Coll Surg 197:479–488PubMedCrossRefGoogle Scholar
  34. 34.
    Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2003) Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am J Surg 185:146–149PubMedCrossRefGoogle Scholar
  35. 35.
    Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen P (2001) Virtual reality computer simulation: an objective method for the evaluation of laparoscopic surgical skills. Surg Endosc 15:242–244PubMedCrossRefGoogle Scholar
  36. 36.
    Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150PubMedCrossRefGoogle Scholar
  37. 37.
    Pearson A, Gallagher AG, Rosser JC, Satava RM (2002) Evaluation of structured and quantitative training methods for teaching intracorporeal knot tying. Surg Endosc 16:130–137PubMedCrossRefGoogle Scholar
  38. 38.
    Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Anderson DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–464PubMedCrossRefGoogle Scholar
  39. 39.
    Chaudhry A, Sutton C, Wood J, Stone R, McCloy R (1999) Learning rate for laparoscopic surgical skills on MIST VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg 81:281–286Google Scholar
  40. 40.
    Schijven MP, Klaassen R, Jakimowicz J, Terpstra OT (2003) The intercollegiate basic surgical skills course. Surg Endosc 17:1978–1984PubMedCrossRefGoogle Scholar
  41. 41.
    Woodrum D, Andreatta P, Yellamanchilla R, Feryus L, Gauger P, Minter R (2006) Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 191:28–32PubMedCrossRefGoogle Scholar
  42. 42.
    McNatt S, Smith CD (2001) A computer-based laparoscopic skills assessment device differentiates experienced from novice laparoscopic surgeons. Surg Endosc 15:1085–1089PubMedCrossRefGoogle Scholar
  43. 43.
    Datta V, Chang A, Mackay S, Darzi A (2002) The relationship between motion analysis and surgical technical assessments. Am J Surg 184:70–73PubMedCrossRefGoogle Scholar
  44. 44.
    Scott DJ, Rege RV, Bergen PC, Guo WA, Laycock R, Tesfay ST, Valentine RJ, Jones DB (2000) Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J LaparoendoscAdv Surg Tech 10:183–190CrossRefGoogle Scholar
  45. 45.
    Thomas W (2006) Teaching and assessing surgical competence. Ann R Coll Surg 88:429–432CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Michael G. C. Pellen
    • 1
    • 2
    • 3
  • Liam F. Horgan
    • 1
    • 2
  • J. Roger Barton
    • 1
    • 2
  • Stephen E. Attwood
    • 1
    • 2
  1. 1.Northumbria Healthcare NHS Trust, North Tyneside General HospitalNorth ShieldsUK
  2. 2.Northumbria Upper Gastrointestinal Team of SurgeonsUniversity of Newcastle upon Tyne, The Medical SchoolNewcastle upon TyneUK
  3. 3.Department of General SurgeryNorth Tyneside General HospitalNorth ShieldsUK

Personalised recommendations