Abstract
With the application of deep learning, prosthetic rehabilitation can be carried out in a manner that not only emulates human manipulation skills and performance, but can also work more efficiently. In this study, we introduced computer vision capability for a rehabilitation robot using a convolutional neural network (CNN). The human skill of scooping was studied by dividing it into four motion primitives or sub-tasks. For each primitive, optimum human posture was identified in terms of muscular effort. Human motion skills were analyzed in terms of physiological parameters, including wrist pronation-supination angle, elbow flexion angle, shoulder rotation/abduction/flexion angles, and hand accelerations by three dimensional musculoskeletal modeling. This analysis identified how humans execute the same activity for eight different materials. Optimum human motion for each material was mapped to a robotic arm with six degrees-of-freedom (DOFs), which was equipped with a camera. The success ratio while examining the scooping motion over all trials was found to be 85%. Consequently, the activity can be performed efficiently based on human intuition in a dynamic environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Orin, D.E.: Application of robotics to prosthetic control. Ann. Biomed. Eng. 8, 305 (1980). https://doi.org/10.1007/BF02363434
Demircan, E., Wheeler, J., Anderson, F.C., Besier, T., Delp, S.: EMG-informed computed muscle control for dynamic simulations of movement. In: Proceedings of the XXII Congress of the International Society of Biomechanics (ISB), Cape Town, South Africa (2009)
Riener, R., Frey, M., Bernhardt, M., Nef, T., Colombo, G.: Human-centered rehabilitation robotics. In: 9th International Conference on Rehabilitation Robotics 2005. ICORR 2005, pp. 319-322 (2005)
Zhang, X.Y., Wang, K.X.: Robot assisted rehabilitation technology robotics and intelligent devices. Chin. J. Rehabil. 28(4), 246–248 (2013)
Gonzalez, R.V., Buchanan, T.S., Delp, S.L.: How muscle architecture and moment arms affect wrist flexion-extension moments. J. Biomech. 30(7), 705–712 (1997)
Grigore, E.C., Scassellati, B.: Discovering action primitive granularity from human motion for human-robot collaboration. In: Proceedings of Robotics: Science and Systems, July 2017
Lim, B., Ra, S., Park, F.: Movement primitives principal component analysis and the efficient generation of natural motions. In: Proceedings IEEE International Conference on Robotics Automation, pp. 4630-4635 (2005)
Optitrack system. https://optitrack.com/
Khatib, O., Demircan, E., De Sapio, V., Sentis, L., Besier, T., Delp, S.: Robotics-based synthesis of human motion. J. Physiol. Paris 103, 211–219 (2009)
Seth, A., Sherman, M., Reinbolt, J.A., Delp, S.L.: OpenSim: a musculoskeletal modeling and simulation framework for in silico investigations and exchange. PMC, April 2015
Thelen, D.G., Anderson, F.C., Delp, S.L.: Generating dynamic simulations of movement using computed muscle control. J. Biomech. 36(3), 321–328 (2003)
Park, J.S., Park, C., Manocha, D.: Intention-aware motion planning using learning based human motion prediction. In: Proceedings of Robotics: Science and Systems, July 2017
Dantam, N.T., Kingston, Z.K., Chaudhuri, S., Kavraki, L.E.: Incremental task and motion planning: a constraint-based approach. In: Robotics: Science and Systems (2016)
Albrecht, S., Ramırez-Amaro, K., Ruiz-Ugalde, F., Weikersdorfer, D., Leibold, M., Ulbrich, M., Beetz, M.: Imitating human reaching motions using physically inspired optimization principles. In: IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia (2011)
Demircan, E., Besier, T., Menon, S., Khatib, O.: Human motion reconstruction and synthesis of human skills. In: Lenarcic, J., Stanisic, M.M. (Eds.) Advances in Robot Kinematics, pp. 283-292. Springer, Heidelberg (2010)
Holzbaur, K.R., Murray, W.M., Delp, S.L.: A model of the upper extremity for simulating musculoskeletal surgery and analyzing neuromuscular control. Ann. Biomed. Eng. 33(6), 829–840 (2005)
Demircan, E., Sentis, L., De Sapio, V., Khatib, O.: Human motion reconstruction by direct control of marker trajectories. In: Lenarcic, J., Stanisic, M.M. (Eds.) Advances in Robot Kinematics, pp. 263-272. Springer, Heidelberg (2008)
EEGO system. https://www.ant-neuro.com/products/eego_sports
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)
Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, pp. 1717-1724, 23–28 June 2014
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., Kudlur, M., Levenberg, J., Monga, R., Moore, S., Murray, D.G., Steiner, B., Tucker, P., Vasudevan, V., Warden, P., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: a system for large-scale machine learning. Google Brain. In: 12th USENIX Symposium on OSDI, (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556v6 [cs.CV], 10 April 2015
Caffe Model. https://gist.github.com/ksimonyan/211839e770f7b538e2d8#file-readme-md
Imagenet. http://image-net.org/
Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (NIPS) (2012)
K-nearest neighbor. https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Chaudhari, D., Bhagat, K., Demircan, E. (2020). Understanding and Transfer of Human Skills in Robotics Using Deep Learning and Musculoskeletal Modeling. In: Xiao, J., Kröger, T., Khatib, O. (eds) Proceedings of the 2018 International Symposium on Experimental Robotics. ISER 2018. Springer Proceedings in Advanced Robotics, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-030-33950-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-33950-0_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33949-4
Online ISBN: 978-3-030-33950-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)