Abstract
It was confirmed that a real mobile robot with a simple visual sensor could learn appropriate motions to reach a target object by direct-vision-based reinforcement learning (RL). In direct-vision-based RL, raw visual sensory signals are put directly into a layered neural network, and then the neural network is trained using back propagation, with the training signal being generated by reinforcement learning. Because of the time-delay in transmitting the visual sensory signals, the actor outputs are trained by the critic output at two time-steps ahead. It was shown that a robot with a simple monochrome visual sensor can learn to reach a target object from scratch without any advance knowledge of this task by direct-vision-based RL.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson CW (1989) Learning to control an inverted pendulum using neural networks. IEEE Control Syst Mag 9:31–37
Morimoto J, Doya K (2001) Aquisition of stand-up behavior by a real robot using hierarchical reinforcement learning. Robotics Autono Syst 36:37–51
Tesauro GJ (1992) Practical issues in temporal difference learning. Mach Learn 8:257–277
Asada M, Noda S, Tawaratsumida S, et al. (1996) Purposive behavior acquisition for a real robot by vision-based reinforcement learning. Mach Learn 24:279–303
Shibata K, Okabe Y, Ito K (2001) Direct-vision-based reinforcement learning using a layered neural network. For the process from sensors to motors (in Japanese). Transaction of SICE 37(2):168–177
Shibata K, Okabe Y (1997) Reinforcement learning when the visual signals are directly given as inputs. Proceedings of ICNN Houston 3:1716–1720
Shibata K, Sugisaka M, Ito K (2001) Fast and stable learning in direct-vision-based reinforcement learning. Proceedings of AROB 6th'01, Tokyo, pp 200–203
Shibata K, Ito K, Okabe Y (1998) Direct-vision-based reinforcement learning in “Going-to-a-target task” with an obstacle and with a variety of target sizes. Proceedings of NEURAP'98, Marseille, pp 95–102
Barto AG, Sutton RS, Anderson CW (1983) Neuron-like adaptive elements that can solve difficult learning control problems. IEEE Trans SMC 5:835–846
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Iida, M., Sugisaka, M. & Shibata, K. Direct-vision-based reinforcement learning in a real mobile robot. Artif Life Robotics 7, 102–106 (2003). https://doi.org/10.1007/BF02481156
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02481156