Skip to main content
SpringerLink
Log in

Online exploratory behavior acquisition model based on reinforcement learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Discernment behavior is an exploratory behavior that supports object feature extraction and is a fundamental tool used by robots to orient themselves, operate objects, and establish knowledge. The main contribution of this paper is to propose an active perception model and analyzes the acquired motion patterns. In this study, we propose an active perception model in which a robot autonomously learns discernment behavior by interacting with multiple objects in its environment. During such interactions, the robot receives reinforcement signals according to the cluster distance of the observed data. In other words, we use a reinforcement learning approach to reward the successful recognition of objects. We apply our proposed model to a mobile robot simulation to observe its effectiveness. Results show that our proposed model effectively established intelligent strategies based on the relationship between object features and the robot’s configuration. In addition, we perform our experiments using real mobile robots and observe the suitability of the observed learned behaviors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Anderson CW (1989) Learning to Control an Inverted Pendulum Using Neural Networks. IEEE Control Syst Mag 9:31–37

    Article  Google Scholar 

  2. İlkay A, Dag̃ N, Kalkan S, Şahin E (2010) Affordances and emergence of concepts. In: Proceedings of the 10th International Conference on Epigenetic Robotics, 11–18

  3. Gibson E (1988) Exploratory behavior in the development of perceiving, acting, and the acquiring of knowledge. Annu Rev Psychol 39(1):1–42

    Article  Google Scholar 

  4. Gibson JJ (1962) Observations on active touch. Psychol Rev 69(6):477–491

    Article  MathSciNet  Google Scholar 

  5. Griffith S, Sinapov J, Sukhoy V, Stoytchev A (2010) How to separate containers from non-containers? a behavior-grounded approach to acoustic object categorization. In: Proceedings of 2010 IEEE International Conference on Robotics and Automation (ICRA2010), 1852–1859

  6. Held R, Hein A (1963) Movement-produced stimulation in the development of visually guided behavior. J Comp Physiol Psychol 56(5):872–876

    Article  Google Scholar 

  7. Shibata K, Nishinoz T, Okabe Y (2001) Actor-q based active perception learning system. In: Proceedings of International Conference on Robotics and Automation 2001 (ICRA2001), 1000–1005

  8. Maris M, Boeckhorst R (1996) Exploiting physical constraints: heap formation through behavioral error in a group of robots. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS96), 1655–1660

  9. Morimoto G, Ikegami T (2004) Evolution of plastic sensory-motor coupling and dynamic categorization. In: Pollack J, et al. (eds) Artificial Life IX. The MIT press, Massachusetts, pp 188–193

    Google Scholar 

  10. Nishide S, Ogata T, Tani J, Komatani K, Okuno G H (2008) Predicting object dynamics from visual images through active sensing experiences. Adv Robot 22(5):527–546

    Google Scholar 

  11. Nolfi S, Marocco D (2002) Active perception: A sensorimotor account of object categorization. In: Proceedings of the 7th international conference on simulation of adaptive behavior on From animals to animats (SAB2002), 266–271

  12. Pfeifer R, Scheier C (1999) Understanding Intelligence. The MIT Press, Cambridge

    Google Scholar 

  13. Sinapov J, Sukhoy V, Sahai R, Stoytchev A (2009) Vibrotactile Recognition and Categorization of Surfaces by a Humanoid Robot. IEEE Trans Robot 27(3):488–497

    Article  Google Scholar 

  14. Sinapov J, Wiemer M, Stoytchev A (2009) Interactive learning of the acoustic properties of household objects. In: Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA2009), 2518–2524

  15. Stoytchev A (2005) Behavior-grounded representation of tool affordances. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation (ICRA2005), 3060–3065

  16. Sutton RS, Barto AG (1998) Reinforcement Learning: An Introduction. The MIT Press, Massachusetts

    Google Scholar 

  17. Takamuku S, Hosoda K, Asada M (2008) Object category acquisition by dynamic touch. Adv Robot 22(10):1143–1154

    Article  Google Scholar 

  18. Takamuku S, Takahashi Y, Asada M (2006) Lexicon acquisition based on object-oriented behavior learning. Adv Robot 20(10):1127–1145

    Article  Google Scholar 

  19. Tsukada M, Madokoro H, Sato K (2010) Unsupervised and adaptive category classification for a vision-based mobile robot. In: Proceedings of the 2010 International Joint Conference on Neural Networks (IJCNN2010), 1–6

  20. Turvey MT (1996) Dynamic touch. Am Psychol 51(11):1134–1152

    Article  Google Scholar 

Download references

Acknowledgments

This research was partially supported by the Ministry of Education, Science, Sports and Culture, Grant-in-Aid for Young Scientists (B), 24700196.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering and Intelligent Systems, Faculty of Engineering, Tohoku Gakuin University, 1-13-1 Chuo, Tagajo-shi, Miyagi, 985-8537, Japan

    Manabu Gouko

  2. Department of Mechanical Engineering, Graduate School of Engineering, Shizuoka University, 3-5-1 Johoku, Naka-ku, Hamamatsu-shi, Shizuoka, 432-8561, Japan

    Yuichi Kobayashi

  3. Department of Electrical Engineering and Computer Science, Faculty of Engineering, Iwate University, 4-3-5, Ueda, Morioka-shi, Iwate, 020-8551, Japan

    Chyon Hae Kim

Authors
  1. Manabu Gouko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuichi Kobayashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chyon Hae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manabu Gouko.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gouko, M., Kobayashi, Y. & Kim, C.H. Online exploratory behavior acquisition model based on reinforcement learning. Appl Intell 42, 75–86 (2015). https://doi.org/10.1007/s10489-014-0567-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-014-0567-4

Keywords

Search

Navigation