Sensor-based machine learning for workflow detection and as key to detect expert level in laparoscopic suturing and knot-tying
- 291 Downloads
The most common way of assessing surgical performance is by expert raters to view a surgical task and rate a trainee’s performance. However, there is huge potential for automated skill assessment and workflow analysis using modern technology. The aim of the present study was to evaluate machine learning (ML) algorithms using the data of a Myo armband as a sensor device for skills level assessment and phase detection in laparoscopic training.
Materials and methods
Participants of three experience levels in laparoscopy performed a suturing and knot tying task on silicon models. Experts rated performance using Objective Structured Assessment of Surgical Skills (OSATS). Participants wore Myo armbands (Thalmic Labs™, Ontario, Canada) to record acceleration, angular velocity, orientation, and Euler orientation. ML algorithms (decision forest, neural networks, boosted decision tree) were compared for skill level assessment and phase detection.
28 participants (8 beginner, 10 intermediate, 10 expert) were included, and 99 knots were available for analysis. A neural network regression model had the lowest mean absolute error in predicting OSATS score (3.7 ± 0.6 points, r2 = 0.03 ± 0.81; OSATS min.-max.: 4–37 points). An ensemble of binary-class neural networks yielded the highest accuracy in predicting skill level (beginners: 82.2% correctly identified, intermediate: 3.0%, experts: 79.5%) whereas standard statistical analysis failed to discriminate between skill levels. Phase detection on raw data showed the best results with a multi-class decision jungle (average 16% correctly identified), but improved to 43% average accuracy with two-class boosted decision trees after Dynamic time warping (DTW) application.
Modern machine learning algorithms aid in interpreting complex surgical motion data, even when standard analysis fails. Dynamic time warping offers the potential to process and compare surgical motion data in order to allow automated surgical workflow detection. However, further research is needed to interpret and standardize available data and improve sensor accuracy.
KeywordsMyo armband Machine learning Neural networks Laparoscopy Surgical education Electromyography Skill assessment Workflow analysis Artificial intelligence Laparoscopic training
Compliance with ethical standards
Felix Nickel reports receiving travel support for conference participation as well as equipment provided for laparoscopic surgery courses by KARL STORZ, Johnson & Johnson, Intuitive, and Medtronic. Karl-Friedrich Kowalewski, Carly R. Garrow, Mona W. Schmidt, Laura Benner and Beat Müller-Stich have no conflicts of interest or financial ties to disclose.
- 10.Ahmadi S-A et al (2006) Recovery of surgical workflow without explicit models. In: International Conference on medical image computing and computer-assisted intervention. Springer, New YorkGoogle Scholar
- 12.Padoy N et al (2007) A boosted segmentation method for surgical workflow analysis. In: International Conference on medical image computing and computer-assisted intervention. Springer, New YorkGoogle Scholar
- 16.Spangenberg N et al (2017) Method for intra-surgical phase detection by using real-time medical device data. In: IEEE 30th International Symposium on computer-based medical systemsGoogle Scholar
- 18.Lin P-J, Chen HY (2018) Design and implement of a rehabilitation system with surface electromyography technology. In: 2018 IEEE International Conference on applied system invention (ICASI). IEEEGoogle Scholar
- 20.Sathiyanarayanan M, Raja S (2016) Myo armband for physiotherapy healtchare: a case study using gesture recognition applicationGoogle Scholar
- 22.Jimenez DA et al (2016) Human-computer interaction for image guided surgery systems using physiological signals: application to deep brain stimulation surgery. In VII Latin American Congress on Biomedical Engineering CLAIB 2016, Bucaramanga, Santander, Colombia, October 26th–28th, 2017. SpringerGoogle Scholar
- 30.Wang Q (2013) Dynamic Time Warping (DTW). MathWorks file exchange. https://www.mathworks.com/matlabcentral/fileexchange/43156-dynamic-time-warping-dtw