Development of a novel tool to assess skills in laparoscopic gastrectomy using the Delphi method: the Japanese operative rating scale for laparoscopic distal gastrectomy (JORS-LDG)

  • Yo KurashimaEmail author
  • Yusuke Watanabe
  • Naoki Hiki
  • Saseem Poudel
  • Hidehiko Kitagami
  • Yuma Ebihara
  • Soichi Murakami
  • Toshiaki Shichinohe
  • Satoshi Hirano



Assessment of the performance of laparoscopic gastrectomy is yet unreported, likely because of the complexity of the procedure. We aimed to develop a tool to assess the skills required for laparoscopic distal gastrectomy (LDG) through cognitive task analysis (CTA) and expert consensus using the Delphi method.


CTA involved the listing of the knowledge and criteria required for completing each step of LDG as subtasks based on interviews by experts and novices and text book, instructional video, and procedural review. The Delphi evaluation method involved anonymous online surveys, conducted to merge the opinions of experts in laparoscopic gastrectomy. The experts were asked to rate (from 1 to 5) the importance of subtasks for skill evaluation and training for LDG using a Likert scale. Consensus among expert viewpoints was determined by the internal consistency of each item using Cronbach’s approach.


Essential subtasks drafted for the assessment of LDG performance were determined based on the CTA. Thirty-one LDG experts participated in the online-survey with a response rate over 90%. A consensus was achieved after 2 rounds of surveys with a Cronbach alpha of 0.86, and 34 subtasks of LDG were selected. We finally created the Japanese Operative Rating Scale for Laparoscopic Distal Gastrectomy (JORS-LDG) based on the 34 subtasks.


We developed the JORS-LDG using CTA and the Delphi method.


Laparoscopy Gastrectomy Assessment Delphi Surgical education 



The author would like to thank the 31 laparoscopic surgeons who participated in this study. We could not complete this study without their kind support.

Compliance with ethical standards


Dr. Yo Kurashima is a consultant at Ethicon. Drs. Yusuke Watanabe, Naoki Hiki, Saseem Poudel, Hidehiko Kitagami, Yuma Ebihara, Soichi Murakami, Toshiaki Shichinohe, and Satoshi Hirano have no conflicts of interest or financial ties to disclose.


  1. 1.
    Derevianko AY, Schwaitzberg SD, Tsuda S, Barrios L, Brooks DC, Callery MP, Fobert D, Irias N, Rattner DW, Jones DB (2010) Malpractice carrier underwrites fundamentals of laparoscopic surgery training and testing: a benchmark for patient safety. Surg Endosc 24:616–623CrossRefGoogle Scholar
  2. 2.
    Hafford ML, Van Sickle KR, Willis RE, Wilson TD, Gugliuzza K, Brown KM, Scott DJ (2013) Ensuring competency: are fundamentals of laparoscopic surgery training and certification necessary for practicing surgeons and operating room personnel? Surg Endosc 27:118–126CrossRefGoogle Scholar
  3. 3.
    Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A tool for training and evaluation of laparoscopic inguinal hernia repair: the global operative assessment of laparoscopic skills-groin hernia (GOALS-GH). Am J Surg 201:54–61CrossRefGoogle Scholar
  4. 4.
    Poudel S, Kurashima Y, Kawarada Y, Watanabe Y, Murakami Y, Matsumura Y, Kato H, Miyazaki K, Shichinohe T, Hirano S (2016) Development and validation of a checklist for assessing recorded performance of laparoscopic inguinal hernia repair. Am J Surg 212:468–474CrossRefGoogle Scholar
  5. 5.
    Palter VN, Grantcharov TP (2012) A prospective study demonstrating the reliability and validity of two procedure-specific evaluation tools to assess operative competence in laparoscopic colorectal surgery. Surg Endosc 26:2489–2503CrossRefGoogle Scholar
  6. 6.
    Zevin B, Bonrath EM, Aggarwal R, Dedy NJ, Ahmed N, Grantcharov TP (2013) Development, feasibility, validity, and reliability of a scale for objective assessment of operative performance in laparoscopic gastric bypass surgery. J Am Coll Surg 216:955–965CrossRefGoogle Scholar
  7. 7.
    Torre LA, Bray F, Siegel RL, Ferlay J, Lortet-Tieulent J, Jemal A (2012) Global cancer statistics. CA Cancer J Clin 65:87–108CrossRefGoogle Scholar
  8. 8.
    Fock KM, Ang TL (2010) Epidemiology of Helicobacter pylori infection and gastric cancer in Asia. J Gastro Hepatol 25:479–486CrossRefGoogle Scholar
  9. 9.
    Japanese Gastric Cancer Association (2017) Japanese gastric cancer treatment guidelines 2014 (ver.). Gastric Cancer 20:1–19CrossRefGoogle Scholar
  10. 10.
    Mori T, Kimura T, Kitajima M (2010) Skill accreditation system for laparoscopic gastroenterologic surgeons in Japan. Minim Invasive Ther Allied Technol 19:18–23CrossRefGoogle Scholar
  11. 11.
    Singh P, Aggarwal R, Zevin B, Grantcharov T, Darzi A (2014) A global Delphi consensus study on defining and measuring quality in surgical training. J Am Coll Surg 219:346–353.e7CrossRefGoogle Scholar
  12. 12.
    Fink A, Kosecoff J, Chassin M, Brook RH (1984) Consensus methods: characteristics and guidelines for use. Am J Public Health 74:979–983CrossRefGoogle Scholar
  13. 13.
    Black N, Murphy M, Lamping D, McKee M, Sanderson C, Askham J, Marteau T (1999) Consensus development methods: a review of best practice in creating clinical guidelines. J Health Serv Res Policy 4:236–248CrossRefGoogle Scholar
  14. 14.
    Stefanidis D, Montero P, Urbach DR, Qureshi A, Perry K, Bachman SL, Madan A, Petersen R, Pryor AD (2014) SAGES research agenda in gastrointestinal and endoscopic surgery: updated results of a Delphi study. Surg Endosc 28:2763–2771CrossRefGoogle Scholar
  15. 15.
    Francis N, Kazaryan AM, Pietrabissa A, Goitein D, Yiannakopoulou E, Agresta F, Khatkov I, Schulze S, Arulampalam T, Tomulescu V, Kim YW, Targarona EM, Zaninotto G (2017) A research agenda for the European Association for Endoscopic Surgeons (EAES). Surg Endosc 31:2042–2049CrossRefGoogle Scholar
  16. 16.
    Pucher PH, Brunt LM, Fanelli RD, Asbun HJ, Aggarwal R (2015) SAGES expert Delphi consensus: critical factors for safe surgical practice in laparoscopic cholecystectomy. Surg Endosc 29:3074–3085CrossRefGoogle Scholar
  17. 17.
    Palter VN, MacRae HM, Grantcharov TP (2011) Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology. Am J Surg 201:251–259CrossRefGoogle Scholar
  18. 18.
    Dijkstra FA, Bosker RJ, Veeger NJ, van Det MJ, Pierie JP (2015) Procedural key steps in laparoscopic colorectal surgery, consensus through Delphi methodology. Surg Endosc 29:2620–2627CrossRefGoogle Scholar
  19. 19.
    Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369:1434–1442CrossRefGoogle Scholar
  20. 20.
    Mackenzie H, Ni M, Miskovic D, Motson RW, Gudgeon M, Khan Z, Longman R, Coleman MG, Hanna GB (2015) Clinical validity of consultant technical skills assessment in the English National Training Programme for laparoscopic colorectal surgery. Br J Surg 102:991–997CrossRefGoogle Scholar
  21. 21.
    Scally CP, Varban OA, Carlin AM, Birkmeyer JD, Dimick JB (2016) Video ratings of surgical skill and late outcomes of bariatric surgery. JAMA Surg 151:e160428CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Yo Kurashima
    • 1
    • 2
    Email author
  • Yusuke Watanabe
    • 1
  • Naoki Hiki
    • 3
  • Saseem Poudel
    • 1
  • Hidehiko Kitagami
    • 4
  • Yuma Ebihara
    • 1
  • Soichi Murakami
    • 1
  • Toshiaki Shichinohe
    • 1
  • Satoshi Hirano
    • 1
  1. 1.Department of Gastroenterological Surgery IIHokkaido University Graduate School of MedicineSapporoJapan
  2. 2.Clinical Simulation CenterHokkaido University Graduate School of MedicineSapporoJapan
  3. 3.Department of Gastroenterological SurgeryThe Cancer Institute Hospital of Japanese Foundation for Cancer ResearchTokyoJapan
  4. 4.Department of Gastroenterological SurgeryKeiyukai Sapporo HospitalSapporoJapan

Personalised recommendations