Skip to main content

Advertisement

SpringerLink
Log in

Intelligent Tutoring for Surgical Decision Making: a Planning-Based Approach

  • ARTICLE
  • Published:
International Journal of Artificial Intelligence in Education Aims and scope Submit manuscript

Abstract

Virtual reality simulation has had a significant impact on training of psychomotor surgical skills, yet there is still a lack of work on its use to teach surgical decision making. This is particularly noteworthy given the recognized importance of decision making in achieving positive surgical outcomes. With the objective of filling this gap, we have developed a system for teaching surgical decision making in the field of endodontics, by integrating a virtual reality simulation environment with a conversational intelligent tutor. This work presents SDMentor (Surgical Decision-making Mentor) – the first intelligent tutoring system for teaching surgical decision making. In this paper we focus on presenting the intelligent tutoring component of the training system. The design of the system and the teaching approaches are driven by information gained from an observational study of clinical teaching sessions. The tutoring system represents surgical actions and the procedure using a variant of the planning domain definition language (PDDL). Tutorial interaction places emphasis on teaching the rationale for decisions, as well as aspects of situation awareness. We evaluated the quality of the tutorial content by comparing it with tutorial feedback from ten experienced human tutors. We had three expert dental instructors rate the appropriateness of the tutorial feedback given by the human tutors as well as SDMentor over 20 scenarios. Bayesian analysis showed that tutoring interventions of SDMentor were significantly better than interventions by human tutors. To determine whether the expert scores may have been influenced by raters’ knowledge of whether interventions came from human tutors or SDMentor, we carried out a type of Turing test. Results show that the expert raters were able to correctly guess which interventions came from SDMentor only 15% of the time, compared to a random baseline accuracy of 9%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Andersen, D. K. (2012). How can educators use simulation applications to teach and assess surgical judgment? Academic Medicine, 87(7), 934–941. https://doi.org/10.1097/ACM.0b013e3182583248.

    Article  Google Scholar 

  • Badash, I., Burtt, K., Solorzano, C. A., & Carey, J. N. (2016). Innovations in surgery simulation: A review of past, current and future techniques. Annals of Translational Medicine, 4(23), 453. https://doi.org/10.21037/atm.2016.12.24.

    Article  Google Scholar 

  • Belghith, K., Nkambou, R., Kabanza, F., & Hartman, L. (2012). An intelligent simulator for telerobotics training. IEEE Transactions on Learning Technologies, 5(1), 11–19.

    Article  Google Scholar 

  • Bertoli, P., Cimatti, A., Lago, U. D., & Pistore, M. (2003). Extending PDDL to nondeterminism, limited sensing and iterative conditional plans. ICAPS’03 Workshop on PDDL. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.2.4493&rep=rep1&type=pdf

  • Brachman, R. J., & Levesque, H. J. (2004). Knowledge representation and reasoning. Morgan Kaufmann Publishers Inc.

  • Champagne, B. J. (2013). Effective teaching and feedback strategies in the OR and beyond. Clinics in Colon and Rectal Surgery, 26(4), 244–249. https://doi.org/10.1055/s-0033-1356725.

    Article  Google Scholar 

  • Charles, F., Cavazza, M., Smith, C., Georg, G., & Porteous, J. (2013). Instantiating interactive narratives from patient education documents. Conference on artificial intelligence in medicine in Europe, 273–283. https://doi.org/10.1007/978-3-642-38326-7_39.

  • Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253–278.

    Article  Google Scholar 

  • Cristancho, S. M., Apramian, T., Vanstone, M., Lingard, L., Ott, M., Forbes, T., & Novick, R. (2016). Thinking like an expert: Surgical decision making as a cyclical process of being aware. The American Journal of Surgery, 211(1), 64–69. https://doi.org/10.1016/j.amjsurg.2015.03.010.

    Article  Google Scholar 

  • De Clercq, P. A., Blom, J. A., Korsten, H. H. M. M., & Hasman, A. (2004). Approaches for creating computer-interpretable guidelines that facilitate decision support. Artificial Intelligence in Medicine, 31(1), 1–27. https://doi.org/10.1016/j.artmed.2004.02.003.

    Article  Google Scholar 

  • Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32–64. https://doi.org/10.1518/001872095779049543.

    Article  Google Scholar 

  • Faculty of Dentistry – Thammasat University. (n.d.). Root canal treatment examination assessment form.

  • Flin, R., Yule, S., Paterson-Brown, S., Maran, N., Rowley, D., & Youngson, G. (2007). Teaching surgeons about non-technical skills. The Surgeon, 5(2), 86–89. https://doi.org/10.1016/S1479-666X(07)80059-X.

    Article  Google Scholar 

  • Freedman, R. (1999). Atlas: A plan manager for mixed-initiative, multimodal dialogue. AAAI-99 workshop on mixed-initiative intelligence, 1–8.

  • Gertner, A. S. (1997). Plan recognition and evaluation for on-line critiquing. User Modeling and User-Adapted Interaction, 7(2), 107–140.

    Article  Google Scholar 

  • Gertner, A. S., Conati, C., & Vanlehn, K. (1998). Procedural help in Andes: Generating hints using a Bayesian network student model. AAAI-98, 106–111.

  • Ghallab, M., Howe, A., Knoblock, C., McDermott, D., Ram, A., Veloso, M., Weld, D., Wilkins, D., Barrett, A., Christianson, D., Friedman, M., Kwok, C., Golden, K., Penberthy, S., Smith, D. E., & Sun, Y. (1998). PDDL – the planning domain definition language. Technical Report CVC TR-98-003/DCS TR-1165. https://helios.hud.ac.uk/scommv/IPC-14/repository/mcdermott-et-al-tr-1998.pdf

  • Goldberg, L. J., Ceusters, W., Eisner, J., & Smith, B. (2005). The significance of SNODENT.

  • Graesser, A. C., Wiemer-Hastings, K., Wiemer-Hastings, P., & Kreuz, R. (1999). AutoTutor: A simulation of a human tutor. Cognitive Systems Research, 1(1), 35–51. https://doi.org/10.1016/S1389-0417(99)00005-4.

    Article  Google Scholar 

  • Graesser, A. C., Vanlehn, K., Rosé, C. P., Jordan, P. W., & Harter, D. (2001). Intelligent tutoring systems with conversational dialogue. AI Magazine, 22(4), 39–52. https://doi.org/10.1609/AIMAG.V22I4.1591.

    Article  Google Scholar 

  • Haddawy, P., Doan, A., & Kahn, J. C. E. (1996). Decision-theoretic refinement planning in medical decision making: Management of acute deep venous thrombosis. Medical Decision Making, 16(4), 315–325.

    Article  Google Scholar 

  • Hauge, L. S., Wanzek, J. A., & Godellas, C. (2001). The reliability of an instrument for identifying and quantifying surgeons’ teaching in the operating room. The American Journal of Surgery, 181(4), 333–337. https://doi.org/10.1016/S0002-9610(01)00577-3.

    Article  Google Scholar 

  • Hill, K. A., Dasari, M., Littleton, E. B., & Hamad, G. G. (2017). How can surgeons facilitate resident intraoperative decision-making? American Journal of Surgery, 214(4), 583–588. https://doi.org/10.1016/j.amjsurg.2017.06.006.

    Article  Google Scholar 

  • Hoffman, R. L., Petrosky, J. A., Eskander, M. F., Selby, L. V., & Kulaylat, A. N. (2015). Feedback fundamentals in surgical education: Tips for success. Bulletin of the American College of Surgeons, 100(8), 35–39 https://bulletin.facs.org/2015/08/feedback-fundamentals-in-surgical-education-tips-for-success/.

    Google Scholar 

  • Johnston, M. J., Paige, J. T., Aggarwal, R., Stefanidis, D., Tsuda, S., Khajuria, A., & Arora, S. (2016). An overview of research priorities in surgical simulation: What the literature shows has been achieved during the 21st century and what remains. American Journal of Surgery, 211(1), 214–225. https://doi.org/10.1016/j.amjsurg.2015.06.014.

    Article  Google Scholar 

  • Jordan, P. W., Rose, C., & Kurt. (2001). Tools for authoring tutorial dialogue knowledge. Proceedings of AI in education 2001 conference.

  • Kabanza, F., Belghith, K., Bellefeuille, P., Auder, B., & Hartman, L. (2008). Planning 3D task demonstrations of a teleoperated space robot arm. ICAPS, 164–173.

  • Kazi, H., Haddawy, P., & Suebnukarn, S. (2012). Employing UMLS for generating hints in a tutoring system for medical problem-based learning. Journal of Biomedical Informatics, 45(3), 557–565. https://doi.org/10.1016/j.jbi.2012.02.010.

    Article  Google Scholar 

  • Kovacs, D. L. (2011). Complete BNF description of PDDL3.1. https://helios.hud.ac.uk/scommv/IPC-14/repository/kovacs-pddl-3.1-2011.pdf

  • Kruschke, J. K. (2013). Bayesian estimation supersedes the t-test. Journal of Experimental Psychology: General, 142(2), 573–603. https://doi.org/10.1037/a0029146.

    Article  Google Scholar 

  • Kruschke, J. K. (2014). Null hypothesis significance testing. In Doing Bayesian data analysis, (pp. 297–333). https://doi.org/10.1037//0003-066X.56.1.16.

  • Kruschke, J. K. (2015). Chapter 23: Ordinal predicted variable. In Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd ed., pp. 671–702). Elsevier. https://doi.org/10.1016/B978-0-12-405888-0.00023-4.

  • Kruschke, J. K., & Liddell, T. M. (2018). The Bayesian new statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychonomic Bulletin and Review, 25(1), 178–206. https://doi.org/10.3758/s13423-016-1221-4.

    Article  Google Scholar 

  • Lalys, F., & Jannin, P. (2014). Surgical process modelling: A review. International Journal of Computer Assisted Radiology and Surgery, 9(3), 495–511. https://doi.org/10.1007/s11548-013-0940-5.

    Article  Google Scholar 

  • Lin, D. T., Park, J., Liebert, C. A., & Lau, J. N. (2015). Validity evidence for surgical improvement of clinical knowledge ops: A novel gaming platform to assess surgical decision making. The American Journal of Surgery, 209(1), 79–85. https://doi.org/10.1016/j.amjsurg.2014.08.033.

    Article  Google Scholar 

  • Lingard, L. (2004). Communication failures in the operating room: An observational classification of recurrent types and effects. Quality and Safety in Health Care, 13(5), 330–334. https://doi.org/10.1136/qshc.2003.008425.

    Article  Google Scholar 

  • Luengo, V., Larcher, A., & Tonetti, J. (2011). Design and implementation of a visual and haptic simulator in a platform for a TEL system in percutaneuos orthopedic surgery. In J. D. Westwood, S. W. Westwood, L. Fellander-Tsai, R. S. Haluck, H. M. Hoffman, R. A. Robb, S. Senger, & K. G. Vosburgh (Eds.), Medicine meets virtual reality 18: NextMed (Issue March 2014, pp. 324–328). IOS Press.

  • MacKenzie, C. L., Ibbotson, J. A., Cao, C. G. L., & Lomax, A. J. (2001). Hierarchical decomposition of laparoscopic surgery: A human factors approach to investigating the operating room environment. Minimally Invasive Therapy and Allied Technologies, 10(3), 121–127. https://doi.org/10.1080/136457001753192222.

    Article  Google Scholar 

  • Masson, M. E. J. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43(3), 679–690. https://doi.org/10.3758/s13428-010-0049-5.

    Article  Google Scholar 

  • Miksch, S., Shahar, Y., & Johnson, P. (1997). Asbru: A task-specific, intention-based, and time-oriented language for representing skeletal plans. Proceedings of the 7th workshop on knowledge engineering: Methods & languages (KEML-97), 9–19.

  • Miller, P. L. (1983). Critiquing anesthetic management: The “ATTENDING” computer system. Anesthesiology, 58(4), 362–369.

    Article  Google Scholar 

  • Mitrovic, A. (1998). A knowledge-based teaching system for SQL. Proceedings of ED-MEDIA, 1027–1032.

  • Mitrovic, A. (2005). The effect of explaining on learning: A case study with a data normalization tutor. AIED, 499–506. http://www.cosc.canterbury.ac.nz/tanja.mitrovic/NORMIT-AIED05.pdf

  • Mitrovic, A. (2010). Modeling domains and students with constraint-based modeling. In Advances in intelligent tutoring systems (pp. 63–80). Springer.

  • Neumuth, T. (2012). Surgical process modeling – theory, methods, and applications.

  • Neumuth, T. (2017). Surgical process modeling. Innovative Surgical Sciences, 2(3), 123–137. https://doi.org/10.1515/iss-2017-0005.

    Article  Google Scholar 

  • Nkambou, R. (2010). Modeling the domain: An introduction to the expert module. In R. Nkambou (Ed.), Advances in intelligent tutoring systems (pp. 15–32). Springer.

  • Nkambou, R., Mizoguchi, R., Bourdeau, J., & Mizoguchi, R. (2010). Introduction: What are intelligent tutoring systems, and why this book ? In Advances in intelligent tutoring systems (pp. 1–12). Springer. https://doi.org/10.1007/978-3-642-14363-2.

  • Palter, V. N., & Grantcharov, T. P. (2010). Simulation in surgical education. Canadian Medical Association Journal, 182(11), 1191–1196. https://doi.org/10.1503/cmaj.091743.

    Article  Google Scholar 

  • Pellier, D. (n.d.). PDDL4J. Retrieved November 28, 2016, from https://github.com/pellierd/pddl4j

  • Pellier, D., & Fiorino, H. (2018). PDDL4J: A planning domain description library for java. Journal of Experimental and Theoretical Artificial Intelligence, 30(1), 143–176.

    Article  Google Scholar 

  • Person, N. K., Bautista, L., Kreuz, R. J., & Graesser, A. C. (2000). The dialog advancer network: A conversation manager for AutoTutor. ITS 2000 proceedings of the workshop on modeling human teaching tactics and strategies, 86–92.

  • Puga, J. L., Krzywinski, M., & Altman, N. (2015). Bayesian statistics: Today’s predictions are tomorrow’s priors. Nature Methods, 12(5), 377–379.

    Article  Google Scholar 

  • Pugh, C., Plachta, S., Auyang, E., Pryor, A., & Hungness, E. (2010). Outcome measures for surgical simulators: Is the focus on technical skills the best approach? Surgery, 147(5), 646–654. https://doi.org/10.1016/j.surg.2010.01.011.

    Article  Google Scholar 

  • Rennie, S., Blyth, P., Swan, J., Rudland, J., Hall, K., Baxter, S., Wilkinson, T., Dockerty, J., Rij, A. Van, Loke, S. K., Winikoff, M., Vlugter, P., Cohen, A., & Mcdonald, J. (2009). Developing surgical decision making skills through dynamic branching short cases and reflection. Proceedings Ascilite, 832–836.

  • Reznek, M., Harter, P., & Krummel, T. (2002). Virtual reality and simulation: Training the future emergency physician. Academic Emergency Medicine: Official Journal of the Society for Academic Emergency Medicine, 9(1), 78–87. https://doi.org/10.1197/aemj.9.1.78.

    Article  Google Scholar 

  • Roberts, N. K., Williams, R. G., Kim, M. J., & Dunnington, G. L. (2009). The briefing, intraoperative teaching, debriefing model for teaching in the operating room. Journal of the American College of Surgeons, 208(2), 299–303. https://doi.org/10.1016/j.jamcollsurg.2008.10.024.

    Article  Google Scholar 

  • Sarker, S. K., Rehman, S., Ladwa, M., Chang, A., & Vincent, C. (2009). A decision-making learning and assessment tool in laparoscopic cholecystectomy. Surgical Endoscopy, 23(1), 197–203. https://doi.org/10.1007/s00464-008-9774-6.

    Article  Google Scholar 

  • Servais, E. L., Lamorte, W. W., Agarwal, S., Moschetti, W., Mallipattu, S. K., & Moulton, S. L. (2006). Teaching surgical decision-making: An interactive, web-based approach. The Journal of Surgical Research, 134(1), 102–106. https://doi.org/10.1016/j.jss.2005.11.583.

    Article  Google Scholar 

  • Spencer, F. (1978). Teaching and measuring surgical techniques: The technical evaluation of competence. Bulletin of the American College of Surgeons, 63(3), 9–12.

    Google Scholar 

  • Sugand, K., Mawkin, M., & Gupte, C. (2016). Training effect of using touch surgery™ for intramedullary femoral nailing. Injury, 47(2), 448–452. https://doi.org/10.1016/j.injury.2015.09.036.

    Article  Google Scholar 

  • Susarla, S. C., Adcock, A. B., Eck, R. N. Van, Moreno, K. N., & Graesser, A. (2003). Development and evaluation of a lesson authoring tool for AutoTutor. AIED 2003 supplementary proceedings, 378–387.

  • Thomas, J. M., Member, S., Young, R. M., & Member, S. (2010). Annie: Automated generation of adaptive learner guidance for fun serious games. IEEE Transactions on Learning Technologies, 3(4), 329–343.

    Article  Google Scholar 

  • Torabinejad, M., & Walton, R. E. (2008). Endodontics: Principles and practice (4th ed.). Saunders Elsevier.

  • Toussaint, B., Luengo, V., Jambon, F., & Tonetti, J. (2015). From heterogeneous multisource traces to perceptual-gestural sequences: The PeTra treatment approach. International conference on artificial intelligence in education, 480–491. https://doi.org/10.1007/978-3-319-19773-9.

  • Tran, L. N., Gupta, P., Poniatowski, L. H., Alanee, S., Dall’era, M. A., & Sweet, R. M. (2013). Validation study of a computer-based open surgical trainer: SimPraxis(®) simulation platform. Advances in Medical Education and Practice, 4, 23–30. https://doi.org/10.2147/AMEP.S38422.

    Article  Google Scholar 

  • Tsuda, S., Scott, D., Doyle, J., & Jones, D. B. (2009). Surgical skills training and simulation. Current Problems in Surgery, 46(4), 271–370. https://doi.org/10.1067/j.cpsurg.2008.12.003.

    Article  Google Scholar 

  • Tsui, J., & Edtech, S. (2014). Septris and SICKO: Implementing and using learning analytics and gamification in medical education. Educause, March, 1–7.

  • Vanlehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227–265 http://dl.acm.org/citation.cfm?id=1435351.1435353.

    Google Scholar 

  • Vannaprathip, N., Haddawy, P., Suebnukarn, S., Sangsartra, P., Sasikhant, N., Sangutai, S., Nunnapin, S., Sornram, S., Sasikhant, N., & Sangutai, S. (2016). Desitra: A simulator for teaching situated decision making in dental surgery. The 21st international conference on intelligent user interfaces, 07–10-Mar, 397–401. https://doi.org/10.1145/2856767.2856807.

  • Vickery, A. W., & Lake, F. R. (2005). Teaching on the run tips 10: Giving feedback. Medical Journal of Australia, 183(5), 267–268 vic10464_fm [pii].

    Article  Google Scholar 

  • Wanzel, K. R., Ward, M., & Reznick, R. K. (2002). Teaching the surgical craft: From selection to certification. Current Problems in Surgery, 39(6), 583–659. https://doi.org/10.1067/mog.2002.123481.

  • Williams, J. D. (2007). Applying POMDPs to dialog systems in the troubleshooting domain. Proceedings of the workshop on bridging the gap: Academic and industrial research in dialog technologies, 1–8.

  • Woods, W. A. (1970). Transition network grammars for natural language analysis. Communications of the ACM, 13, 591–606. https://doi.org/10.1145/355598.362773.

    Article  MATH  Google Scholar 

Download references

Acknowledgments

We thank the Faculty of Dentistry, Thammasat University for their support with the observational study and evaluations. We gratefully acknowledge the funding provided by the Hanse-Wissenschaftskolleg Institute for Advanced Study, Delmenhorst, Germany (Haddawy, Suebnukarn), the Santander Bremen International Student Internship Program Scholarship (Vannaprathip), Bremen Spatial Cognition Center Scholarship (Vannaprathip), Mahidol University Faculty of Information and Communication Technology Scholarship (Vannaprathip), the Mahidol University Office of International Relations for the Mahidol Bremen Medical Informatics Research Unit, and the Thailand Research Fund (RDG6050029).

Author information

Authors and Affiliations

  1. Faculty of Information and Communication Technology, Mahidol University, Nakhon Pathom, Thailand

    Narumol Vannaprathip & Peter Haddawy

  2. Bremen Spatial Cognition Center, University of Bremen, Bremen, Germany

    Peter Haddawy & Holger Schultheis

  3. Institute for Artificial Intelligence, University of Bremen, Bremen, Germany

    Holger Schultheis

  4. Faculty of Dentistry, Thammasat University, Pathumthani, Thailand

    Siriwan Suebnukarn

Authors
  1. Narumol Vannaprathip
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter Haddawy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Holger Schultheis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Siriwan Suebnukarn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Haddawy.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

ESM 1

(PDF 479 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vannaprathip, N., Haddawy, P., Schultheis, H. et al. Intelligent Tutoring for Surgical Decision Making: a Planning-Based Approach. Int J Artif Intell Educ 32, 350–381 (2022). https://doi.org/10.1007/s40593-021-00261-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40593-021-00261-3

Keywords

Search

Navigation