Skip to main content
SpringerLink
Log in

Development and validation of an endoscopic submucosal dissection video assessment tool

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Despite a need for assessment of endoscopic submucosal dissection (ESD) skills in order to track progress and determine competence, there is no structured measure of assessing competency in ESD performance. The present study aims to develop and examine validity evidence for an assessment tool to evaluate the recorded performance of ESD for gastric neoplasms.

Methods

The ESD video assessment tool (EVAT) was systematically developed by ESD experienced endoscopists. The EVAT consists of a 25-item global rating scale and 3-item checklist to assess competencies required to perform ESD. Five unedited videos were each evaluated by 2-blinded experienced ESD endoscopists to assess inter-rater reliability using intraclass correlation coefficients (ICC). Seventeen unedited videos in total were rated by 3 blinded experienced ESD endoscopists. Validity evidence for relationship to other variables was examined by comparing scores of inexperienced (fellows) and experienced endoscopists (attending staff), and by evaluating the relationship between the EVAT scores and ESD case experience. Internal consistency was evaluated using Cronbach’s alpha.

Results

The inter-rater reliability for the total score was high at 0.87 (95% confidence interval 0.11 to 0.99). The total score [median, interquartile range (IQR)] was significantly different between the inexperienced (71, 63–77) and experienced group (95, 91–97) (P = 0.005). The total scores demonstrated high correlation with the number of ESD cases (Spearman’s ρ = 0.79, P < 0.01). The internal consistency was 0.97.

Conclusions

This study provides preliminary validity evidence for the assessment of video-recorded ESD performances for gastric neoplasms using EVAT.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Ono H, Kondo H, Gotoda T, Shirao K, Yamaguchi H, Saito D, Hosokawa K, Shimoda T, Yoshida S (2001) Endoscopic mucosal resection for treatment of early gastric cancer. Gut 48:225–229

    Article  CAS  Google Scholar 

  2. Gotoda T, Kondo H, Ono H, Saito Y, Yamaguchi H, Saito D, Yokota T (1999) A new endoscopic mucosal resection procedure using an insulation-tipped electrosurgical knife for rectal flat lesions: report of two cases. Gastrointest Endosc 50:560–563

    Article  CAS  Google Scholar 

  3. Pimentel-Nunes P, Dinis-Ribeiro M, Ponchon T, Repici A, Vieth M, De Ceglie A, Amato A, Berr F, Bhandari P, Bialek A, Conio M, Haringsma J, Langner C, Meisner S, Messmann H, Morino M, Neuhaus H, Piessevaux H, Rugge M, Saunders BP, Robaszkiewicz M, Seewald S, Kashin S, Dumonceau JM, Hassan C, Deprez PH (2015) Endoscopic submucosal dissection: European society of gastrointestinal endoscopy (ESGE) guideline. Endoscopy 47:829–854

    Article  Google Scholar 

  4. Cao Y, Liao C, Tan A, Gao Y, Mo Z, Gao F (2009) Meta-analysis of endoscopic submucosal dissection versus endoscopic mucosal resection for tumors of the gastrointestinal tract. Endoscopy 41:751–757

    Article  CAS  Google Scholar 

  5. Maple JT, Abu Dayyeh BK, Chauhan SS, Hwang JH, Komanduri S, Manfredi M, Konda V, Murad FM, Siddiqui UD, Banerjee S (2015) Endoscopic submucosal dissection. Gastrointest Endosc 81:1311–1325

    Article  Google Scholar 

  6. Fukami N (2014) ESD around the world: United States. Gastrointest Endosc Clin N Am 24:313–320

    Article  Google Scholar 

  7. Dath D, Regehr G, Birch D, Schlachta C, Poulin E, Mamazza J, Reznick R, MacRae HM (2004) Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills. Surg Endosc 18:1800–1804

    Article  CAS  Google Scholar 

  8. Bhatt A, Abe S, Kumaravel A, Parsi MA, Stevens T, Jang S, Lopez R, Oda I, Vargo JJ, Saito Y (2016) Video-based supervision for training of endoscopic submucosal dissection. Endoscopy 48:711–716

    Article  Google Scholar 

  9. Downing SM (2003) Validity: on meaningful interpretation of assessment data. Med Educ 37:830–837

    Article  Google Scholar 

  10. Takao M, Bilgic E, Waschke K, Kaneva P, Endo S, Nakano Y, Tanaka S, Morita Y, Toyonaga T, Umegaki E, Kodama Y, Fried GM (2019) Defining competencies for endoscopic submucosal dissection (ESD) for gastric neoplasms. Surg Endosc 33:1206–1215

    Article  Google Scholar 

  11. Japanese gastric cancer treatment guidelines (2014) (ver. 4). Gastric Cancer 20:1–19

    Article  Google Scholar 

  12. Chung IK, Lee JH, Lee SH, Kim SJ, Cho JY, Cho WY, Hwangbo Y, Keum BR, Park JJ, Chun HJ, Kim HJ, Kim JJ, Ji SR, Seol SY (2009) Therapeutic outcomes in 1000 cases of endoscopic submucosal dissection for early gastric neoplasms: Korean ESD Study Group multicenter study. Gastrointest Endosc 69:1228–1235

    Article  Google Scholar 

  13. Epstein RM (2007) Assessment in medical education. N Engl J Med 356:387–439

    Article  CAS  Google Scholar 

  14. Madani A, Watanabe Y, Vassiliou M, Feldman LS, Duh QY, Singer MC, Ruan DT, Tabah R, Mitmaker E (2016) Defining competencies for safe thyroidectomy: an international Delphi consensus. Surgery 159(86–94):96–101

    Google Scholar 

  15. Vassiliou MC, Kaneva PA, Poulose BK, Dunkin BJ, Marks JM, Sadik R, Sroka G, Anvari M, Thaler K, Adrales GL, Hazey JW, Lightdale JR, Velanovich V, Swanstrom LL, Mellinger JD, Fried GM (2010) Global assessment of gastrointestinal endoscopic skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc 24:1834–1841

    Article  Google Scholar 

  16. Sedlack RE (2011) Training to competency in colonoscopy: assessing and defining competency standards. Gastrointest Endosc 74:355–366 (e351–352)

    Article  Google Scholar 

  17. Gupta S, Anderson J, Bhandari P, McKaig B, Rupert P, Rembacken B, Riley S, Rutter M, Valori R, Vance M, van der Vleuten CP, Saunders BP, Thomas-Gibson S (2011) Development and validation of a novel method for assessing competency in polypectomy: direct observation of polypectomy skills. Gastrointest Endosc 73:1232–1239 (e1232)

    Article  Google Scholar 

  18. Sedlack RE (2010) The Mayo colonoscopy skills assessment tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc 72:1125–1133 (1133.e1121–1123)

    Article  Google Scholar 

  19. Walsh CM, Ling SC, Khanna N, Grover SC, Yu JJ, Cooper MA, Yong E, Nguyen GC, May G, Walters TD, Reznick R, Rabeneck L, Carnahan H (2015) Gastrointestinal endoscopy competency assessment tool: reliability and validity evidence. Gastrointest Endosc 81:1417–1424 (e1412)

    Article  Google Scholar 

  20. Vogt VY, Givens VM, Keathley CA, Lipscomb GH, Summitt RL Jr (2003) Is a resident's score on a videotaped objective structured assessment of technical skills affected by revealing the resident's identity? Am J Obstet Gynecol 189:688–691

    Article  Google Scholar 

  21. Scott DJ, Rege RV, Bergen PC, Guo WA, Laycock R, Tesfay ST, Valentine RJ, Jones DB (2000) Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J Laparoendosc Adv Surg Technol A 10:183–190

    Article  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Surgery and Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University, Montreal, QC, Canada

    Madoka Takao, Elif Bilgic, Pepa Kaneva, Satoshi Endo & Gerald M. Fried

  2. Division of Gastroenterology, Department of Internal Medicine, Graduate School of Medicine, Kobe University, Kobe, Hyogo, Japan

    Madoka Takao, Yoshiko Nakano, Fumiaki Kawara, Shinwa Tanaka, Tsukasa Ishida, Yoshinori Morita, Eiji Umegaki & Yuzo Kodama

  3. Division of Gastroenterology, McGill University, Montreal, QC, Canada

    Kevin Waschke

  4. Department of Endoscopy, Kobe University Hospital, Kobe, Hyogo, Japan

    Takashi Toyonaga

  5. Faculty of Medicine, McGill University, L9.313, 1650 Cedar Avenue, Montreal, QC, H3G 1A4, Canada

    Gerald M. Fried

Authors
  1. Madoka Takao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elif Bilgic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pepa Kaneva
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kevin Waschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Satoshi Endo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yoshiko Nakano
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Fumiaki Kawara
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shinwa Tanaka
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Tsukasa Ishida
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Yoshinori Morita
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Takashi Toyonaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Eiji Umegaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Yuzo Kodama
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Gerald M. Fried
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gerald M. Fried.

Ethics declarations

Disclosures

Drs. Madoka Takao, Elif Bilgic, Pepa Kaneva, Kevin Waschke, Satoshi Endo, Yoshiko Nakano, Fumiaki Kawara, Shinwa Tanaka, Tsukasa Ishida, Yoshinori Morita, Takashi Toyonaga, Eiji Umegaki, Yuzo Kodama, and Gerald M. Fried have no conflicts of interest or financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 36 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Takao, M., Bilgic, E., Kaneva, P. et al. Development and validation of an endoscopic submucosal dissection video assessment tool. Surg Endosc 35, 2671–2678 (2021). https://doi.org/10.1007/s00464-020-07688-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-020-07688-0

Keywords

Search

Navigation