Artificial Life and Robotics

, Volume 3, Issue 2, pp 90–96 | Cite as

Learning control of autonomous robots using an instance-based classifier generator in continuous state space

  • M. Svinin
  • K. Kuroyama
  • K. Ueda
  • Y. Nakamura
Original Article


A classifier system for the reinforcement learning control of autonomous mobile robots is proposed. The classifier system contains action selection, rules reproduction, and credit assignment mechanisms. An important feature of the classifier system is that it operates with continuous sensor and action spaces. The system is applied to the control of mobile robots. The local controllers use independent classifiers specified at the wheel-level. The controllers work autonomously, and with respect to each other represent dynamic systems connected through the external environment. The feasibility of the proposed system is tested in an experiment with a Khepera robot. It is shown that some patterns of global behavior can emerge from locally organized classifiers.

Key words

Autonomous robots Reinforcement learning Classifier systems 


  1. 1.
    Connel J, Mahadevan S (eds) (1993) Robot learning. Kluwer, BostonGoogle Scholar
  2. 2.
    Kaelbling L, Littman M, Moore A (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285Google Scholar
  3. 3.
    Asada M, Noda S, Hosoda K (1997) Action-based state space construction for robot learning (in Japanese). J Robotics Soc Jpn 15:76–82Google Scholar
  4. 4.
    Watkins C (1989) Learning from delayed rewards. PhD Thesis, University of CambridgeGoogle Scholar
  5. 5.
    Sutton R (1990) Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In: Proceedings of the 7th International Conference on Machine Learning, pp 216–224Google Scholar
  6. 6.
    Naruse K, Leu M (1997) Autonomous vehicle navigation by layered learning and planning. In: Proceedings of the 29th CIRP International Seminar on Manufacturing Systems, Osaka, pp 87–92Google Scholar
  7. 7.
    Deng K, Moore A (1995) Multiresolution instance-based learning. In: Proceedings of the International Joint Conference on Artificial IntelligenceGoogle Scholar
  8. 8.
    Moore A, Atkeson C (1995) The parti-game algorithm for variable resolution reinforcement learning in multidimensional state space. Mach Learn 21:1–36Google Scholar
  9. 9.
    McCallum R (1996) Hidden state and reinforcement learning with instance-based state identification. IEEE Trans System Man Cybern Part B, 26:464–473CrossRefGoogle Scholar
  10. 10.
    Murao H, Kitamura S (1997) An incremental quantization method of the continuous sensor space for learning agents. Mem Fac Eng Kobe Univ 44:155–164Google Scholar
  11. 11.
    Murao H, Kitamura S (1998) An adaptive state space design for reinforcement learning. In: Sugisaka M (ed) Proceedings of the International Sympasium on Artificial Life and Robotics (AROB 3), vol 1, pp 85–88Google Scholar
  12. 12.
    Goldberg D (1989) Genetic algorithms in search, optimization and machine learning, Addison-Wesley, ReadingMATHGoogle Scholar
  13. 13.
    Holland J, Holyoak K, Nisbett R et al. (1989) Induction. Processes of inference, learning, and discovery. MIT Press, CambridgeGoogle Scholar
  14. 14.
    Holland J (1995) Hidden order. How adaptation builds complexity. Addison-Wesley, New YorkGoogle Scholar
  15. 15.
    Nakamura Y, Ohnishi S, Okhura K et al. (1997) Instance-based reinforcement learning for robot path finding in continuous space. In: Proceedings of the IEEE International Conference on System, Man, and Cybernetics, pp 1228–1234Google Scholar
  16. 16.
    Kuroyama K, Svinin M, Nakamura Y et al. (1998) Learning control of autonomous robots with the use of an instance-based classifier generator in the continuous state space. In: Sugisaka M (ed) Proceedings of the International Sympasium on Artificial Life and Robotics (AROB 3), vol 1, pp 89–92Google Scholar
  17. 17.
    Holland J (1985) Properties of the bucket brigade algorithm. In: Proceedings of the 1st International Conference on Genetic Algorithms and their Applications, pp 1–7Google Scholar

Copyright information

© ISAROB 1999

Authors and Affiliations

  1. 1.Mechanical Engineering DepartmentKobe UniversityKobeJapan
  2. 2.Hitachi Zosen CorporationTechnical Research InstituteOsakaJapan

Personalised recommendations