Skill-based human–robot cooperation in tele-operated path tracking
This work proposes a shared-control tele-operation framework that adapts its cooperative properties to the estimated skill level of the operator. It is hypothesized that different aspects of an operator’s performance in executing a tele-operated path tracking task can be assessed through conventional machine learning methods using motion-based and task-related features. To identify performance measures that capture motor skills linked to the studied task, an experiment is conducted where users new to tele-operation, practice towards motor skill proficiency in 7 training sessions. A set of classifiers are then learned from the acquired data and selected features, which can generate a skill profile that comprises estimations of user’s various competences. Skill profiles are exploited to modify the behavior of the assistive robotic system accordingly with the objective of enhancing user experience by preventing unnecessary restriction for skilled users. A second experiment is implemented in which novice and expert users execute the path tracking on different pathways while being assisted by the robot according to their estimated skill profiles. Results validate the skill estimation method and hint at feasibility of shared-control customization in tele-operated path tracking.
KeywordsShared-control Active constraints Virtual fixtures Tele-operation Machine learning Surgery
This work has received funding from the European Union’s Horizon 2020 research and innovation program under Grant agreement no. H2020-ICT-26-2016-1-732515 (SMARTsurg project).
- Abbott, J. J., Marayong, P., & Okamura A. M. (2007). Haptic virtual fixtures for robot-assisted manipulation. Robotics Research. https://doi.org/10.1007/978-3-540-48113-3_5
- Ahmidi, N., et al. (2017). A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Transactions on Biomedical Engineering, 9294(c), 1–16.Google Scholar
- Anderson, S. J., Karumanchi, S. B., & Iagnemma, K. (2012). Constraint-based planning and control for safe, semi-autonomous operation of vehicles. In 2012 IEEE intelligent vehicles symposium (pp. 383–88). IEEE.Google Scholar
- Bauzano, E., Estebanez, B., & Muñoz, V. F. (2014). Collaborative human-robot system for HALS suture procedures. IEEE Systems Journal, 10, 1–10.Google Scholar
- Berthet-Rayne, P., Power, M., King, H., & Yang Fellow, G.-Z. (2016). Hubot?: A three state human–robot collaborative framework for bimanual surgical tasks based on learned models. In International conference on robotics and automation (pp. 715–722).Google Scholar
- Bruyninckx, H. (2001). Open robot control software: The OROCOS project. In Proceedings 2001 ICRA. IEEE international conference on robotics and automation (Cat. No. 01CH37164) (pp. 2523–2528). IEEE.Google Scholar
- Enayati, N., Costa, E. C. A., Ferrigno, G., & De Momi, E. (2016a). A dynamic non-energy-storing guidance constraint with motion redirection for robot-assisted surgery. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, Korea: IEEE.Google Scholar
- Enayati, N., De Momi, E., & Ferrigno, G. (2016b). Haptics in robot-assisted surgery: Challenges and benefits. IEEE Reviews in Biomedical Engineering, 9, 49–65.Google Scholar
- Enayati, N., Ferrigno, G., & De Momi, E. (2017). Performance metrics for guidance active constraints in surgical robotics. The International Journal of Medical Robotics and Computer Assisted Surgery. https://doi.org/10.1002/rcs.1873.
- Gao, M., Oberlander, J., Schamm, T., & Zollner, J. M. (2014). Contextual task-aware shared autonomy for assistive mobile robot teleoperation. In 2014 IEEE/RSJ international conference on intelligent robots and systems (pp. 3311–3318). IEEE.Google Scholar
- Liu, Y.-T., Mayer-Kress, G., & Newell, K. M. (2006). Qualitative and quantitative change in the dynamics of motor learning. Journal of Experimental Psychology: Human Perception and Performance, 32(2), 380–393.Google Scholar
- Milliken, L., & Hollinger, G. A. (2016). Modeling user expertise for choosing levels of shared autonomy. In Planning for human–robot interaction shared autonomy and collaborative robotics workshop, robotics: Science and systems conference.Google Scholar
- Nichols, K. A. et al. (2015). Models of human-centered automation in a debridement task. In IEEE international conference on intelligent robots and systems 2015–December (pp. 5784–5789).Google Scholar
- Padoy, N., Hager, G.D. (2011) Human-machine collaborative surgery using learned models. In Proceedings of the IEEE international conference on robotics and automation (pp. 5285–5292).Google Scholar
- Penin, L. F., Matsumoto, K., & Wakabayashi, S. (2000). Force reflection for time-delayed teleoperation of space robots. In Proceedings 2000 ICRA. Millennium conference. IEEE International conference on robotics and automation. Symposia proceedings (Cat. No. 00CH37065) (pp. 3120–3125). IEEE.Google Scholar
- Rafii-Tari, H., et al. (2013). Learning-based modeling of endovascular navigation for collaborative robotic catheterization. In International conference on medical image computing and computer-assisted intervention (pp. 369–377). Berlin, Heidelberg: Springer.Google Scholar
- Reiley, C. E., & Hager, G. D. (2009). Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. In International conference on medical image computing and computer-assisted intervention (pp. 435–442). Berlin, Heidelberg: Springer.Google Scholar
- Schmidt, R. A. (1991). Frequent augmented feedback can degrade learning: Evidence and interpretations. In: Tutorials in motor neuroscience (pp. 59–75). Dordrecht, Netherlands: Springer.Google Scholar
- Takayama, L., Marder-Eppstein, E., Harris, H., & Beer, J. M. (2011). Assisted driving of a mobile remote presence system: System design and controlled user evaluation. In 2011 IEEE international conference on robotics and automation (pp. 1883–1889). IEEE.Google Scholar
- Trejos, A. L., et al. (2013). The application of force sensing to skills assessment in minimally invasive surgery. In 2013 IEEE international conference on robotics and automation (pp. 4370–4375). IEEE.Google Scholar