Abstract
We present a case study of reinforcement learning on a real robot that learns how to back up a trailer and discuss the lessons learned about the importance of proper experimental procedure and design. We identify areas of particular concern to the experimental robotics community at large. In particular, we address concerns pertinent to robotics simulation research, implementing learning algorithms on real robotic hardware, and the difficulties involved with transferring research between the two.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Asada, M., Noda, S., Tawaratsumida, S., and Hosada, K. 1996. Purposive behavior acquisition for a real robot by vision-based reinforcement learning. Machine Learning, 23(2–3):279–303.
Barto, A.G., Sutton, R.S., and Anderson, C.W. 1983. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems; Man; and Cybernetics, SMC-13:834–846.
Benbrahim, H., Doleac, J., Franklin, J., and Selfridge, O. 1992. Real-time learning: A ball on a beam. In Proceedings of the International Joint Conference on Neural Networks, Vol. 1, pp. 98–103.
Cohen, P.R. 1995. Emperical Methods for Artificial Intelligence, MIT Press: Cambridge, MA.
Colombetti, M., Dorigo, M., and Borghi, G. 1996. Behavior analysis and training: A methodology for behavior engineering. IEEE Transactions on Systems; Man; and Cybernetics, B26(3):365–380.
Hougen, D.F. 1998. Connectionist reinforcement learning for control of robotic systems. Ph.D. thesis, University of Minnesota, Minneapolis, MN.
Hougen, D.F., Fischer, J., Gini, M., and Slagle, J. 1996. Fast connectionist learning for trailer backing using a real robot. In Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1917–1922.
Hougen, D.F., Fischer, J., and Johnam, D. 1994. A neural network pole balancer that learns and operates on a real robot in real time. In Proceedings of the MLC-COLT Workshop on Robot Learning, pp. 73–80.
Hougen, D.F., Gini, M., and Slagle, J. 1997a. Partitioning input space for reinforcement learning for control. In Proceedings of the IEEE International Conference on Neural Networks, pp. 755–760.
Hougen, D.F., Gini, M., and Slagle, J. 1997b. Rapid, unsupervised connectionist learning for backing a robot with two trailers. In Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2950–2955.
Jani, Y. 1993. Application of fuzzy logic-neural network based reinforcement learning to proximity and docking operations: Special approach/docking testcase results. Technical Report NASA-CR-192294, Research Institute for Computing and Information Systems, University of Houston, Clear Lake, TX.
Jervis, T. and Fallside, F. 1992. Pole balancing on a real rig using a reinforcement learning controller, Technical Report CUED/FINFENG/TR 115, Cambridge University Engineering Department, Cambridge, England.
Kohonen, T.K. 1989. Self-Organizing and Associative Memory, 3rd edition, Berlin: Springer-Verlag.
Kong, S.-G. and Kosko, B. 1992. Adaptive fuzzy systems for backing up a truck-and-trailer. IEEE Transactions on Neural Networks, 3(2):211–223.
Koza, J.R. 1992. Agenetic approach to finding a controller to back up a tractor-trailer truck. In Automatic Control Conference, pp. 2307–2311.
Kretchmar, R.M. and Anderson, C.W. 1997. Comparison of CMACs and radial basis functions for local function approximators in reinforcement learning. In Proceedings of the IEEE International Conference on Neural Networks, Houston, TX, pp. 834–837.
Lin, C.-S. and Kim, H. 1991. Use of CMAC neural networks in reinforcement self-learning control. In Proceedings of the 1991 International Conference on Artificial Neural Networks, Espoo, Finland, pp. 1285–1288.
Mahadevan, S. and Connell, J. 1992. Automatic programming of behavior-based robots using reinforcement learning. Artificial Intelligence, 55:311–365.
Martin, F.G. 1995. The mini board technical reference, MIT Media Laboratory, Cambridge, MA.
Martin, F.G. 1998. The handy board technical reference, MIT Media Laboratory, Cambridge, MA.
Matarić, M.J. 1997. Reinforcement learning in the multi-robot domain. Autonomous Robots, 4(1):73–83.
Michie, D. and Chambers, R. 1968. Boxes: An experiment in adaptive control. In E. Dale and D. Michie (Eds.), Machine Intelligence, Oliver and Boyd: Edinburgh.
Nguyen, D. and Widrow, B. 1990. Neural networks for self-learning control systems. IEEE Control Systems Magazine, 10(3):18–23.
Russell, D.W. and Rees, S.J. 1975. System control—A case study of a statistical learning automaton. In R. Trappl and F. de P. Hanika (Eds.), Progress in Cybernetics and Systems Research, John Wiley and Sons: New York, Vol. 2.
Selfridge, O., Sutton, R., and Barto, A. 1985. Training and tracking in robotics. In Proceedings of the International Joint Conference on Artificial Intelligence, Los Angeles, CA, pp. 670–672.
Shelton, R.O. and Peterson, J.K. 1992. Controlling a truck with an adaptive critic CMAC design. Simulation, 58(5):319–326.
Woodcock, N., Hallam, N.J., and Picton, P.D. 1991. Fuzzy BOXES as an alternative to neural networks for difficult control problems. In Proc. of Applications of Artificial Intelligence in Engineering VI, Oxford, UK, pp. 903–919.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hougen, D.F., Rybski, P.E. & Gini, M. Repeatability of Real World Training Experiments: A Case Study. Autonomous Robots 6, 281–292 (1999). https://doi.org/10.1023/A:1008984312527
Issue Date:
DOI: https://doi.org/10.1023/A:1008984312527