Abstract
Deep learning with neural networks is dependent on large amounts of annotated training data. For the development of robotic visuomotor skills in complex environments, generating suitable training data is time-consuming and depends on the availability of accurate robot models. Deep reinforcement learning alleviates this challenge by letting robots learn in an unsupervised manner through trial and error at the cost of long training times. In contrast, we present an approach for acquiring visuomotor skills for grasping through fast self-learning: The robot generates suitable training data through interaction with the environment based on initial motor abilities. Supervised end-to-end learning of visuomotor skills is realized with a deep convolutional neural architecture that combines two important subtasks of grasping: object localization and inverse kinematics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
Visit nico.knowledge-technology.info for further information and video material.
References
Cangelosi, A., Schlesinger, M.: Developmental Robotics. From Babies to Robots. MIT Press/Bradford Books, Cambridge (2014)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of Aistats, vol. 9, pp. 249–256 (2010)
Hahnloser, R.H., Sarpeshkar, R., Mahowald, M.A., Douglas, R.J., Seung, H.S.: Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit. Nature 405(6789), 947–951 (2000)
van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. arXiv preprint 2015. arXiv:1509.06461
Kerzel, M., Strahl, E., Magg, S., Navarro-Guerro, N., Heinrich, S., Wermter, S.: NICO - Neuro-inspired companion: a developmental humanoid robot platform for multimodal interaction. In: RO-MAN 2017 (2017, accepted)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Leitner, J., Harding, S., Förster, A., Corke, P.: A Modular software Framework for eyehand coordination in humanoid robots. Front. Robot. AI 3 (2016)
Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies. J. Mach. Learn. Res. 17(39), 1–40 (2016)
Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)
Lungarella, M., Metta, G., Pfeifer, R., Sandini, G.: Developmental robotics: a survey. Connection Sci. 15(4), 151–190 (2003)
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Is object localization for free?-weakly-supervised learning with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 685–694 (2015)
Peng, X.B., Berseth, G., Panne van de, M.: Terrain-adaptive locomotion skills using deep reinforcement learning. ACM Trans. Graph. 35(4) (2016). 81
Pinto, L., Gupta, A.: Supersizing self-supervision: learning to grasp from 50k tries and 700 robot hours. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3406–3413. IEEE Press (2016)
Speck, D., Barros, P., Weber, C., Wermter, S.: Ball localization for robocup soccer using convolutional neural networks. In: RoboCup Symposium, Leipzig, Germany (2016)
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013)
Sutskever, I., Martens, J., Dahl, G.E., Hinton, G.E.: On the importance of initialization and momentum in deep learning. In: Proceedings of The 30th International Conference on Machine Learning, pp. 1139–1147 (2013)
Acknowledgments
This work was partially funded by the German Research Foundation (DFG) in project Crossmodal Learning (TRR-169) and the Hamburg Landesforschungsförderungsprojekt.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Kerzel, M., Wermter, S. (2017). Neural End-to-End Self-learning of Visuomotor Skills by Environment Interaction. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2017. ICANN 2017. Lecture Notes in Computer Science(), vol 10613. Springer, Cham. https://doi.org/10.1007/978-3-319-68600-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-68600-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68599-1
Online ISBN: 978-3-319-68600-4
eBook Packages: Computer ScienceComputer Science (R0)