Skip to main content
SpringerLink
Log in

Validation study of a skill assessment tool for education and outcome prediction of laparoscopic distal gastrectomy

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

The Japanese operative-rating scale for laparoscopic distal gastrectomy (JORS-LDG) was developed through cognitive task analysis together with the Delphi method to measure intraoperative performance during laparoscopic distal gastrectomy. This study aimed to investigate the value of this rating scale as an educational tool and a surgical outcome predictor in laparoscopic distal gastrectomy.

Methods

The surgical performance of laparoscopic distal gastrectomy was assessed by the first assistant, through self-evaluation in the operating room and by video raters blind to the case. We evaluated inter-rater reliability, internal consistency, and correlations between the JORS-LDG scores and the evaluation methods, patient characteristics, and surgical outcomes.

Results

Fifty-four laparoscopic distal gastrectomy procedures performed by 40 surgeons at 16 institutions were evaluated in the operating room and with video recordings using the proposed rating scale. The video inter-rater reliability was > 0.8. Participating surgeons were divided into the low, intermediate, and high groups based on their total scores. The number of laparoscopic surgeries and laparoscopic gastrectomy procedures performed differed significantly among the groups according to laparoscopic distal gastrectomy skill levels. The low, intermediate, and high groups also differed in terms of median operating times (311, 266, and 229 min, respectively, P < 0.001), intraoperative complication rates (27.8, 11.8, and 0%, respectively, P = 0.01), and postoperative complication rates (22.2, 0, and 0%, respectively, P = 0.002).

Conclusions

The JORS-LDG is a reliable and valid measure for laparoscopic distal gastrectomy training and could be useful in predicting surgical outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Martin JA, Regehr G, Reznick R, Macdrae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278

    CAS  PubMed  Google Scholar 

  2. Doyle JD, Webber EM, Sidhu RS (2007) A universal global rating scale for the evaluation of technical skills in the operating room. Am J Surg 193:551–555 (discussion 555)

    Article  PubMed  Google Scholar 

  3. Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ (2012) The Ottawa surgical competency operating room evaluation (O-SCORE): a tool to assess surgical competence. Acad Med 87:1401–1407

    Article  PubMed  Google Scholar 

  4. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113

    Article  PubMed  Google Scholar 

  5. Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC (2011) A tool for training and evaluation of laparoscopic inguinal hernia repair: the global operative assessment of laparoscopic skills-groin hernia (GOALS-GH). Am J Surg 201:54–61

    Article  PubMed  Google Scholar 

  6. Palter VN, MacRae HM, Grantcharov TP (2011) Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology. Am J Surg 201:251–259

    Article  PubMed  Google Scholar 

  7. Kurashima Y, Watanabe Y, Hiki N, Poudel S, Kitagami H, Ebihara Y, Murakami S, Shichinohe T, Hirano S (2019) Development of a novel tool to assess skills in laparoscopic gastrectomy using the Delphi method: the Japanese operative rating scale for laparoscopic distal gastrectomy (JORS-LDG). Surg Endosc 33:3945–3952

    Article  PubMed  Google Scholar 

  8. Poudel S, Kurashima Y, Kawarada Y, Watanabe Y, Murakami Y, Matsumura Y, Kato H, Miyazaki K, Shichinohe T, Hirano S (2016) Development and validation of a checklist for assessing recorded performance of laparoscopic inguinal hernia repair. Am J Surg 212:468–474

    Article  PubMed  Google Scholar 

  9. Fecso AB, Szasz P, Kerezov G, Grantcharov TP (2017) The effect of technical performance on patient outcomes in surgery: a systematic review. Ann Surg 265:492–501

    Article  PubMed  Google Scholar 

  10. Arvidsson D, Berndsen FH, Larsson LG, Leijonmarck CE, Rimbäck G, Rudberg C, Smedberg S, Spangen L, Montgomery A (2005) Randomized clinical trial comparing 5-year recurrence rate after laparoscopic versus Shouldice repair of primary inguinal hernia. Br J Surg 92:1085–1091

    Article  CAS  PubMed  Google Scholar 

  11. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369:1434–1442

    Article  CAS  PubMed  Google Scholar 

  12. Mackenzie H, Ni M, Miskovic D, Motson RW, Gudgeon M, Khan Z, Longman R, Coleman MG, Hanna GB (2015) Clinical validity of consultant technical skills assessment in the english national training programme for laparoscopic colorectal surgery. Br J Surg 102:991–997

    Article  CAS  PubMed  Google Scholar 

  13. Hogg ME, Zenati M, Novak S, Chen Y, Jun Y, Steve J, Kowalsky SJ, Bartlett DL, Zureikat AH, Zeh HJ (2016) Grading of surgeon technical performance predicts postoperative pancreatic fistula for pancreaticoduodenectomy independent of patient-related variables. Ann Surg 264:482–491

    Article  PubMed  Google Scholar 

  14. Fecso AB, Bhatti JA, Stotland PK, Quereshy FA, Grantcharov TP (2019) Technical performance as a predictor of clinical outcomes in laparoscopic gastric cancer surgery. Ann Surg 270:115–120

    Article  PubMed  Google Scholar 

  15. Curtis NJ, Foster JD, Miskovic D, Brown CSB, Hewett PJ, Abbott S, Hanna GB, Stevenson ARL, Francis NK (2020) Association of surgical skill assessment with clinical outcomes in cancer surgery. JAMA Surg 155:590–598

    Article  PubMed  Google Scholar 

  16. Stulberg JJ, Huang R, Kreutzer L, Ban K, Champagne BJ, Steele SR, Johnson JK, Holl JL, Greenberg CC, Bilimoria KY (2020) Association between surgeon technical skills and patient outcomes. JAMA Surg 155:960–968

    Article  PubMed  PubMed Central  Google Scholar 

  17. Varban OA, Thumma JR, Finks JF, Carlin AM, Ghaferi AA, Dimick JB (2021) Evaluating the effect of surgical skill on outcomes for laparoscopic sleeve gastrectomy: a video-based study. Ann Surg 273:766–771

    Article  PubMed  Google Scholar 

  18. Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, Wales PW (2014) Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 67:401–409

    Article  PubMed  Google Scholar 

  19. Walter SD, Eliasziw M, Donner A (1998) Sample size and optimal designs for reliability studies. Stat Med 17:101–110

    Article  CAS  PubMed  Google Scholar 

  20. Mori T, Kimura T, Kitajima M (2010) Skill accreditation system for laparoscopic gastroenterologic surgeons in Japan. Minim Invasive Ther Allied Technol 19:18–23

    Article  PubMed  Google Scholar 

  21. Szasz P, Louridas M, Harris KA, Aggarwal R, Grantcharov TP (2015) Assessing technical competence in surgical trainees: a systematic review. Ann Surg 261:1046–1055

    Article  PubMed  Google Scholar 

  22. Groenier M, Brummer L, Bunting BP, Gallagher AG (2020) Reliability of observational assessment methods for outcome-based assessment of surgical skill: systematic review and meta-analyses. J Surg Educ 77:189–201

    Article  PubMed  Google Scholar 

  23. Shibasaki S, Suda K, Nakauchi M, Nakamura T, Kadoya S, Kikuchi K, Inaba K, Uyama I (2018) Outermost layer-oriented medial approach for infrapyloric nodal dissection in laparoscopic distal gastrectomy. Surg Endosc 32:2137–2148

    Article  PubMed  Google Scholar 

  24. Kitano S (2009) What technique is suitable for laparoscopic suprapancreatic lymph node dissection? Gastric Cancer 12:67–68

    Article  PubMed  Google Scholar 

  25. Fukunaga T, Hiki N, Tokunaga M, Nohara K, Akashi Y, Katayama H, Yoshiba H, Yamada K, Ohyama S, Yamaguchi T (2009) Left-sided approach for suprapancreatic lymph node dissection in laparoscopy-assisted distal gastrectomy without duodenal transection. Gastric Cancer 12:106–112

    Article  PubMed  Google Scholar 

  26. Guerra F, Giuliani G, Iacobone M, Bianchi PP, Coratti A (2017) Pancreas-related complications following gastrectomy: systematic review and meta-analysis of open versus minimally invasive surgery. Surg Endosc 31:4346–4356

    Article  PubMed  Google Scholar 

  27. Washio M, Yamashita K, Niihara M, Hosoda K, Hiki N (2020) Postoperative pancreatic fistula after gastrectomy for gastric cancer. Ann Gastroenterol Surg 4:618–627

    Article  PubMed  PubMed Central  Google Scholar 

  28. Hiki N, Honda M, Etoh T, Yoshida K, Kodera Y, Kakeji Y, Kumamaru H, Miyata H, Yamashita Y, Inomata M, Konno H, Seto Y, Kitano S (2018) Higher incidence of pancreatic fistula in laparoscopic gastrectomy. Real-world evidence from a nationwide prospective cohort study. Gastric Cancer 21:162–170

    Article  PubMed  Google Scholar 

  29. Bonrath EM, Zevin B, Dedy NJ, Grantcharov TP (2013) Error rating tool to identify and analyse technical errors and events in laparoscopic surgery. Br J Surg 100:1080–1088

    Article  CAS  PubMed  Google Scholar 

  30. Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I, Farrell T (2015) Technical skills assessment toolbox: a review using the unitary framework of validity. Ann Surg 261:251–262

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the surgeons who participated in this study. We would have been unable to complete this study without their support.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Gastroenterological Surgery II, Hokkaido University School of Medicine, Kita-15, Nishi-7, Kita-ku, Sapporo, Hokkaido, 060-8638, Japan

    Yo Kurashima, Saseem Poudel, Yuma Ebihara & Satoshi Hirano

  2. Clinical Simulation Center, Hokkaido University School of Medicine, Sapporo, Japan

    Yo Kurashima

  3. Department of Gastroenterological Surgery, Keiyukai Sapporo Hospital, Sapporo, Japan

    Hidehiko Kitagami

  4. Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwa, Japan

    Koichi Teramura

  5. Department of Gastrointestinal Surgery, Kanazawa University Graduate School of Medical Sciences, Kanazawa, Japan

    Noriyuki Inaki

  6. Department of Gastroenterological Surgery, Teine Keijinkai Hospital, Sapporo, Japan

    Fumitaka Nakamura

  7. Department of Gastroenterological Surgery, Aichi Cancer Center Hospital, Nagoya, Japan

    Kazunari Misawa

  8. Department of Surgey1, School of Medicine, University of Occupational and Environmental Health, Kitakyushu, Japan

    Kazunori Shibao

  9. Department of Gastrointestinal Surgery, Japanese Red Cross Fukuoka Hospital, Fukuoka, Japan

    Eishi Nagai

  10. Department of Gastrointestinal and Hepato-Billiary-Pancreatic Surgery, Faculty of Medicine, University of Tsukuba, Tsukuba, Japan

    Yoshimasa Akashi

  11. Department of Gastroenterological Surgery, Nagoya City University Graduate School of Medical Sciences, Nagoya, Japan

    Shuji Takiguchi

  12. Department of Surgery, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan

    Shinichiro Kobayashi

Authors
  1. Yo Kurashima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hidehiko Kitagami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Koichi Teramura
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saseem Poudel
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuma Ebihara
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Noriyuki Inaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Fumitaka Nakamura
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kazunari Misawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kazunori Shibao
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Eishi Nagai
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Yoshimasa Akashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Shuji Takiguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Shinichiro Kobayashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Satoshi Hirano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yo Kurashima.

Ethics declarations

Disclosures

Drs. Yo Kurashima, Hidehiko Kitagami, Koichi Teramura, Saseem Poudel, Yuma Ebihara, Noriyuki Inaki, Fumitaka Nakamura, Kazunari Misawa, Kazunori Shibao, Eishi Nagai, Yoshimasa Akashi, Shuji Takiguchi, Shinichiro Kobayashi, and Satoshi Hirano have no conflicts of interest or financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kurashima, Y., Kitagami, H., Teramura, K. et al. Validation study of a skill assessment tool for education and outcome prediction of laparoscopic distal gastrectomy. Surg Endosc 36, 8807–8816 (2022). https://doi.org/10.1007/s00464-022-09305-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-022-09305-8

Keywords

Search

Navigation