Advertisement

New Generation Computing

, Volume 19, Issue 3, pp 257–282 | Cite as

Cooperation of categorical and behavioral learning in a practical solution to the abstraction problem

  • Atsushi UenoEmail author
  • Hideaki Takeda
Regular Paper

Abstract

Real robots should be able to adapt autonomously to various environments in order to go on executing their tasks without breaking down. They achieve this by learning how to abstract only useful information from a huge amount of information in the environment while executing their tasks. This paper proposes a new architecture which performs categorical learning and behavioral learning in parallel with task execution. We call the architectureSituation Transition Network System (STNS). In categorical learning, it makes a flexible state representation and modifies it according to the results of behaviors. Behavioral learning is reinforcement learning on the state representation. Simulation results have shown that this architecture is able to learn efficiently and adapt to unexpected changes of the environment autonomously.

Keywords

Abstraction Problem Categorical Learning State Space Segmentation Reinforcement Learning Interleave Planning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1).
    Asada, M., Noda, S. and Hosoda, K., “Action-based Sensor Space Categorization for Robot Learning,” inProc. of the 1996 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 96), pp. 1502–1509, IEEE, 1996.Google Scholar
  2. 2).
    Barto, A. G., Sutton, R. S. and Anderson, C. W., “Neuronlike Adaptive Elements That can Solve Difficult Learning Control Problems,”IEEE Transactions on Systems, Man, and Cybernetics, 13, 5, pp. 834–846, 1983.Google Scholar
  3. 3).
    Cohen, P. R. and Feigenbaum, E. A., eds.,The Handbook of Artificial Intelligence, 3, 14, pp. 323–511, William Kaufmann, 1982.Google Scholar
  4. 4).
    Doya, K., “Temporal Difference Learning in Continuous Time and Space,”Advances in Neural Information Processing System (Touretzky, D. S., Mozer, M. C. and Hasselmo, M. E. eds.),8, MIT Press, 1996.Google Scholar
  5. 5).
    Harnad, S., “Category Induction and Representation,” inCategorical Perception: The Groundwork of Cognition (Harnad, S. ed.),18, Cambridge University Press, 1987.Google Scholar
  6. 6).
    Ishiguro, H., Sato, R. and Ishida, T., “Robot Oriented State Space Construction,” inProc. of the 1996 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 96), pp. 1496–1501, IEEE, 1996.Google Scholar
  7. 7).
    Kröse, B. J. A. and Eecen, M., “A Self-organizing Representation of Sensor Space for Mobile Robot Navigation,” inProc. of the 1994 IEEE/RSJ/GI International Conference on Intelligent Robots and Systems (IROS 94), pp. 9–14, IEEE, 1994.Google Scholar
  8. 8).
    Kröse, B. J. A. and Van Dam, J. W. M., “Adaptive State Space Quantisation for Reinforcement Learning of Collision-free Navigation,” inProc. of the 1992 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 92), pp. 1327–1332, IEEE, 1992.Google Scholar
  9. 9).
    Lin, L., “Self-improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching,”Machine Learning, 8, 3/4, pp. 293–321, 1992.CrossRefGoogle Scholar
  10. 10).
    Martín, P. and Millán, J. Del R., “Reinforcement Learning of Sensor-based Reaching Strategies for a Two-link Manipulator,” inProc. of the 1996 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 96), pp. 1345–1352, IEEE, 1996.Google Scholar
  11. 11).
    Mohri, T., “Nearest Neighbor Rule and Memory-based Reasoning,”Journal of Japanese Society for Artificial Intelligence, 12, 2, pp. 188–195, 1997. (in Japanese)Google Scholar
  12. 12).
    Moore, A. W., “Variable Resolution Dynamic Programming: Efficiently Learning Action Maps in Multivariate Real-valued State-spaces,” inProc. of the Eighth International Workshop on Machine Learning (ML91), pp. 333–337, 1991.Google Scholar
  13. 13).
    Moore, A. W. and Atkeson, C. G., “The Parti-game Algorithm for Variable Resolution Reinforcement Learning in Multidimensional State-spaces,”Machine Learning, 21, 3, pp. 199–233, 1995.Google Scholar
  14. 14).
    Nakamura, T., Takamura, S. and Asada, M., “Behavior-based Map Representation for a Sonar-based Mobile Robot by Statistical Methods,” inProc. of the 1996 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 96), pp. 276–283, IEEE, 1996.Google Scholar
  15. 15).
    Russell, S. and Norvig, P.,Artificial Intelligence: A Modern Approach, Prentice Hall, 1995.Google Scholar
  16. 16).
    Simons, J., Van Brussel, H., De Schutter, J. and Verhaert, J., “A Self-learning Automaton with Variable Resolution for High Precision Assembly by Industrial Robots,”IEEE Transactions on Automatic Control, 27, 5, pp. 1190–1113, 1982.CrossRefGoogle Scholar
  17. 17).
    Sutton, R. S., “Learning to Predict by the Method of Tempoal Differences,”Machine Learning, 3, 1, pp. 9–44, 1988.Google Scholar
  18. 18).
    Sutton, R. S., “Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding,” inProc. of Neural Information Processing Systems (NIPS 95), 1995.Google Scholar
  19. 19).
    Takahashi, Y., Asada, M. and Hosoda, K., “Reasonable Performance in less Learning Time by Real Robot Based on Incremental State Space Segmentation,” inProc. of the 1996 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 96), pp. 1518–1524, IEEE, 1996.Google Scholar
  20. 20).
    Tesauro, G., “Practical Issues in Temporal Difference Learning,”Machine Learning, 8, 3/4, pp. 257–277, 1992.zbMATHCrossRefGoogle Scholar
  21. 21).
    Unemi, T., “Instance-based Reinforcement Learning Method,”Journal of Japanese Society for Artificial Intelligence, 7, 4, pp. 697–707, 1992. (in Japanese)Google Scholar
  22. 22).
    Yairi, T., Nakasuka, S. and Hori, K., “Autonomous State Abstraction from Heterogeneous and Redundant Sensor Information,”Journal of Japanese Society for Artificial Intelligence, 14, 4, pp. 667–678, 1999. (in Japanese)Google Scholar
  23. 23).
    Yamada, S., “Reactive Planning with Uncertainty of a Plan,” inProc. of The Third Annual Conference on AI, Simulation and Planning in High Autonomy Systems, pp. 201–208, 1992.Google Scholar

Copyright information

© Ohmsha, Ltd. and Springer 2001

Authors and Affiliations

  1. 1.Graduate School of Information ScienceNara Institute of Science and TechnologyIkoma, NaraJapan

Personalised recommendations