Skip to main content
SpringerLink
Log in

Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS)

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Objective

Current skills assessment in the Fundamentals of Laparoscopic Surgery (FLS) program is labor intensive, requiring one proctor for every 1–2 subjects. The ProMIS Augmented Reality (AR) simulator (Haptica, Dublin IR) allows for objective assessment of physical tasks through instrument tracking technology. We hypothesized that the ProMIS metrics could differentiate between ability groups as well as standard FLS scoring with fewer personnel requirements

Methods

We recruited 60 volunteer subjects. Subjects were stratified based on their laparoscopic surgical experience. Those who had performed more than 100 laparoscopic procedures were considered experienced (n = 8). Those with fewer than 10 laparoscopic procedures were considered novices (n = 44). The rest were intermediates (n = 8). All subjects performed up to five trials of the peg transfer task from FLS in the ProMIS simulator. The FLS score, instrument path length, and instrument smoothness assessment were generated for each trial.

Results

For each of the five trials, experienced surgeons outperformed intermediates, who in turn outperformed novices. Statistically significant differences were seen between the groups across all trials for FLS score (p < 0.001), ProMIS path length (p < 0.001), and ProMIS smoothness (p < 0.001). When the FLS score was compared to the path length and smoothness metrics, a strong relationship between the scores was apparent for novices (r = 0.78, r = 0.94, p < 0.001) respectively), intermediates (r = 0.5, p = 0.2, r = 0.98, p < 0.001), and experienced surgeons (r = 0.86, p = 0.006, r = 0.99, p < 0.001).

Conclusions

The construct that the standard scoring of the FLS peg transfer task can discriminate between experienced, intermediate, and novice surgeons is validated. The same construct is valid when the task is assessed using the metrics of the ProMIS. The high correlation between these scores establishes the concurrent validity of the ProMIS metrics. The use of AR for objective assessment of FLS tasks could reduce the personnel requirements of assessing these skills while maintaining the objectivity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

Similar content being viewed by others

References

  1. Derossis AM, Antoniuk M, Fried GM (1999) Evaluation of laparoscopic skills: a 2-year follow up during residency training. Can J Surg 42:293–296

    PubMed  CAS  Google Scholar 

  2. Fraser SA, Klassen DR, Feldman LS, Ghitulescu GA, Stanbridge D, Fried GM (2003) Evaluating laparoscopic skills: setting the pass/fail score for the MISTELS System. Surg Endosc 17:964–967

    Article  PubMed  CAS  Google Scholar 

  3. Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004). Proving the value of simulation in laparoscopic surgery. Ann Surg 240:518–528

    Article  PubMed  Google Scholar 

  4. Gallagher AG, Richie K, McClure N, McGuigan J (2001). Objective psychomotor skills assessment of experienced, junior, and novice laparoscopists with virtual reality. World J Surg 25:1478–1483

    Article  PubMed  CAS  Google Scholar 

  5. Gallagher A, Ritter E, Champion H, Higgins G, Fried M, Moses G, Smith CD, Satava RM (2005) Virtual reality for the operating room: proficiency based training as a paradigm shift in surgical skill training. Ann Surg 241:364–372

    Article  PubMed  Google Scholar 

  6. Grantcharov T, Kristianson V, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2003) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150

    Article  Google Scholar 

  7. McClusky D, Gallagher A, Ritter E, Lederman A, Van Sickle K, Baghai M, Smith C (2004) Virtual reality training improves junior residents’ operating room performance: results of a prospective randomized, double-blinded study of the complete laparoscopic cholecystectomy. Paper presented at American College of Surgeons, Clinical Congress, New Orleans. J Am Coll Surg 199:73

    Article  Google Scholar 

  8. Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002). Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–464

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NCA Medical Simulation Center, Norman M. Rich Department of Surgery, Uniformed Services University, 4301 Jones Bridge Road, Bethesda, Maryland, 20814

    E. M. Ritter, C. Michael, E. A. Pimentel & M. W. Bowyer

  2. Division of Surgery, National Naval Medical Center, 8901 Wisconsin Avenue, Bethesda, Maryland, 20889

    T. W. Kindelan

Authors
  1. E. M. Ritter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. W. Kindelan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. Michael
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. E. A. Pimentel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. M. W. Bowyer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. M. Ritter.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ritter, E.M., Kindelan, T.W., Michael, C. et al. Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS). Surg Endosc 21, 1441–1445 (2007). https://doi.org/10.1007/s00464-007-9261-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-007-9261-5

Keywords

Search

Navigation