Skip to main content
SpringerLink
Log in

Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills

  • Original Article
  • Published:
Journal of Robotic Surgery Aims and scope Submit manuscript

Abstract

Increasing focus on patient safety makes it important to ensure surgical competency among surgeons before operating on patients. The objective was to gather validity evidence for a virtual-reality simulator test for robotic surgical skills and evaluate its potential as a training tool. Surgeons with varying experience in robotic surgery were recruited: novices (zero procedures), intermediates (1–50), experienced (> 50). Five experienced surgeons rated five exercises on the da Vinci Skills Simulator. Participants were tested using the five exercises. Participants were invited back 3 times and completed a total of 10 attempts per exercise. The outcome was the average simulator performance score for the 5 exercises. 32 participants from 5 surgical specialties were included. 38 participants completed all 4 sessions. A moderate correlation between the average total score and robotic experience was identified for the first attempt (Spearman r = 0.58; p = 0.0004). A difference in average total score was observed between novices and intermediates [median score 61% (IQR 52–66) vs. 83% (IQR 75–91), adjusted p < 0.0001], as well as novices and experienced [median score 61% (IQR 52–66) vs. 80 (IQR 69–85), adjusted p = 0.002]. All three groups improved their performance between the 1st and 10th attempts (p < 0.00). This study describes validity evidence for a virtual-reality simulator for basic robotic surgical skills, which can be used for assessment of basic competency and as a training tool. However, more validity evidence is needed before it can be used for certification or high-stakes assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Aggarwal R, Mytton OT, Derbrew M et al (2010) Training and simulation for patient safety. Qual Saf Health Care 19 Suppl 2:i34–i43. https://doi.org/10.1136/qshc.2009.038562

  2. Carreyrou J (2010) Botched operation using Da Vinci robot spurs lawsuit. Wall Street J. https://www.wsj.com/articles/SB10001424052748703341904575266952674277806. Accessed 30 Apr 2018

  3. Schreuder HWR, Wolswijk R, Zweemer RP et al (2012) Training and learning robotic surgery, time for a more structured approach: a systematic review. BJOG 119:137–149. https://doi.org/10.1111/j.1471-0528.2011.03139.x

    Article  PubMed  CAS  Google Scholar 

  4. Larsen CR, Oestergaard J, Ottesen BS, Sorensen JL (2012) The efficacy of virtual reality simulation training in laparoscopy: a systematic review of randomized trials. Acta Obstet Gynecol Scand 91:1015–1028. https://doi.org/10.1111/j.1600-0412.2012.01482.x

    Article  PubMed  Google Scholar 

  5. Ahlberg G, Heikkinen T, Iselius L et al (2002) Does training in a virtual reality simulator improve surgical performance? Surg Endosc 16:126–129. https://doi.org/10.1007/s00464-001-9025-6

    Article  PubMed  CAS  Google Scholar 

  6. Zendejas B, Cook DA, Hernández-Irizarry R et al (2012) Mastery learning simulation-based curriculum for laparoscopic TEP inguinal hernia repair. J Surg Educ 69:208–214. https://doi.org/10.1016/j.jsurg.2011.08.008

    Article  PubMed  Google Scholar 

  7. Bric JD, Lumbard DC, Frelich MJ, Gould JC (2016) Current state of virtual reality simulation in robotic surgery training: a review. Surg Endosc 30:2169–2178. https://doi.org/10.1007/s00464-015-4517-y

    Article  PubMed  Google Scholar 

  8. Aydin A, Shafi AMA, Shamim Khan M et al (2016) Current status of simulation and training models in urological surgery: a systematic review. J Urol 196:312–320. https://doi.org/10.1016/j.juro.2016.01.131

    Article  PubMed  Google Scholar 

  9. Moglia A, Ferrari V, Morelli L et al (2016) A systematic review of virtual reality simulators for robot-assisted surgery. Eur Urol 69:1065–1080. https://doi.org/10.1016/j.eururo.2015.09.021

    Article  PubMed  Google Scholar 

  10. Downing SM, Yudkowsky R (2009) Assessment in health professions education. Routledge, New York

    Book  Google Scholar 

  11. Thinggaard E, Bjerrum F, Strandbygaard J et al (2016) Ensuring competency of novice laparoscopic surgeons-exploring standard setting methods and their consequences. J Surg Educ 73:986–991. https://doi.org/10.1016/j.jsurg.2016.05.008

    Article  PubMed  Google Scholar 

  12. Smith R, Patel V, Satava R (2014) Fundamentals of robotic surgery: a course of basic robotic surgery skills based upon a 14-society consensus template of outcomes measures and curriculum development. Int J Med Robot 10:379–384. https://doi.org/10.1002/rcs.1559

    Article  PubMed  Google Scholar 

  13. Foote JR, Valea FA (2016) Robotic surgical training: where are we? Gynecol Oncol 143:179–183. https://doi.org/10.1016/j.ygyno.2016.05.031

    Article  PubMed  Google Scholar 

  14. Downing SM (2003) Validity: on meaningful interpretation of assessment data. Med Educ 37:830–837

    Article  PubMed  Google Scholar 

  15. Zijlstra F (1993) Efficiency in work behaviour: a design approach for modern tools. Delft University Press

  16. Bharathan R, Vali S, Setchell T et al (2013) Psychomotor skills and cognitive load training on a virtual reality laparoscopic simulator for tubal surgery is effective. Eur J Obstet Gynecol Reprod Biol 169:347–352. https://doi.org/10.1016/j.ejogrb.2013.03.017

    Article  PubMed  Google Scholar 

  17. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate—a practical and powerful approach to multiple testing. J R Stat Soc Ser B Methodol 57:289–300

    Google Scholar 

  18. Yang K, Zhen H, Hubert N et al (2017) From dV-trainer to real robotic console: the limitations of robotic skill training. J Surg Educ 74:1074–1080. https://doi.org/10.1016/j.jsurg.2017.03.006

    Article  PubMed  Google Scholar 

  19. Cook DA (2015) Much ado about differences: why expert-novice comparisons add little to the validity argument. Adv Health Sci Educ Theory Pract 20:829–834. https://doi.org/10.1007/s10459-014-9551-3

    Article  PubMed  Google Scholar 

  20. Julian D, Tanaka A, Mattingly P et al (2018) A comparative analysis and guide to virtual reality robotic surgical simulators. Int J Med Robot 14:e1874. https://doi.org/10.1002/rcs.1874

    Article  Google Scholar 

  21. Ahmed K, Khan R, Mottrie A et al (2015) Development of a standardised training curriculum for robotic surgery: a consensus statement from an international multidisciplinary group of experts. BJU Int 116:93–101. https://doi.org/10.1111/bju.12974

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all the surgeons from the eastern part of Denmark who volunteered and participated in this study.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Gynaecology, Juliane Marie Centre for Children, Women and Reproduction, Rigshospitalet-Glostrup University Hospital, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen, Denmark

    Maria Cecilie Havemann, Torur Dalsgaard, Jette Led Sørensen, Kristin Røssaak, Berit Jul Mosgaard & Claus Høgdall

  2. Department of Surgery, Zealand University Hospital, University of Copenhagen, Copenhagen, Denmark

    Steffen Brisling

  3. Department of Surgery, Herlev Gentofte Hospital, University of Copenhagen, Copenhagen, Denmark

    Flemming Bjerrum

Authors
  1. Maria Cecilie Havemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Torur Dalsgaard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jette Led Sørensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kristin Røssaak
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steffen Brisling
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Berit Jul Mosgaard
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Claus Høgdall
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Flemming Bjerrum
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria Cecilie Havemann.

Ethics declarations

Conflict of interest

Drs. Havemann, Dalsgaard, Sorensen, Røssaak, Brisling, Moesgaard, Høgdall and Bjerrum declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Havemann, M.C., Dalsgaard, T., Sørensen, J.L. et al. Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills. J Robotic Surg 13, 99–106 (2019). https://doi.org/10.1007/s11701-018-0811-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11701-018-0811-8

Keywords

Search

Navigation