Skip to main content

Advertisement

SpringerLink
Log in

Infusing common-sensical prior knowledge into topological representations of learning robots

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

Good initializations are always beneficial for efficiently training neural networks. In this study, we propose a method for initializing a layered neural network with common-sensical human knowledge in an intuitive manner. To make this idea possible, a hierarchical neural network containing a two-dimensional topological map is adopted as an adaptive controller for physical robots. The low dimensionality of the map allows humans to hand-design the topological structure to infuse human common-sensical knowledge. The prior knowledge initialization subsequently allows the robot to execute efficient reinforcement learning in a real environment. The primary novelty of this study is the idea for transferring human common-sensical knowledge that may be subjective and not necessarily logically or mathematically expressible. While transfer learning between two different neural networks is common, transfer learning from human to neural networks is not sufficiently studied, and hence, this study offers a new idea for collaborative learning between human and neural networks. The idea is tested in the context of the real-time reinforcement learning of real robots.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Adadi A, Berrada M (2018) Peeking inside the black-box: a survey on explainable artificial intelligence (xai). IEEE Access 6:52138–52160. https://doi.org/10.1109/ACCESS.2018.2870052

    Article  Google Scholar 

  2. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848

  3. Hartono P (2020) Mixing autoencoder with classifier: conceptual data visualization. IEEE Access 8:105,301-105,310. https://doi.org/10.1109/ACCESS.2020.2999155

    Article  Google Scholar 

  4. Hartono P, Hollensen P, Trappenberg T (2015) Learning-regulated context relevant topographical map. IEEE Trans Neural Netw Learn Syst 26(10):2323–2335. https://doi.org/10.1109/TNNLS.2014.2379275

    Article  MathSciNet  Google Scholar 

  5. Hartono P, Kakita S (2009) Fast reinforcement learning for simple physical robots. Memetic Comput 1(4):305–313. https://doi.org/10.1007/s12293-009-0015-x

    Article  Google Scholar 

  6. Hwangbo J et al (2019) Learning agile and dynamic motor skills for legged robots. Sci Robot 4(26):eaau5872. https://doi.org/10.1126/scirobotics.aau5872

    Article  Google Scholar 

  7. Koh PW, Liang P (2017) Understanding black-box predictions via influence functions. In: Precup D, Teh YW (eds) Proceedings of the 34th international conference on machine learning. Proceedings of machine learning research, vol 70. PMLR, pp 1885–1894. https://proceedings.mlr.press/v70/koh17a.html

  8. Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43:59–69. https://doi.org/10.1007/BF00337288

    Article  MathSciNet  MATH  Google Scholar 

  9. LeCun Y et al (2015) Deep learning. Nature 521:436–444. https://doi.org/10.1038/nature14539

    Article  MathSciNet  Google Scholar 

  10. Mikolov T, Sutskever I, Chen K, Corrado G, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th international conference on neural information processing systems, vol 2. NIPS’13, pp 3111-3119

  11. Mnih V et al (2015) Human-level control through deep reinforcement learning. Nature 518:529–533. https://doi.org/10.1038/nature14236

    Article  Google Scholar 

  12. Peng XB, Andrychowicz M, Zaremba W, Abbeel P (2018) Sim-to-real transfer of robotic control with dynamics randomization. In: 2018 IEEE international conference on robotics and automation (ICRA), pp 3803–3810. https://doi.org/10.1109/ICRA.2018.8460528

  13. Peng XB et al (2020) Learning agile robotic locomotion skills by imitating animals. https://arxiv.org/abs/2004.00784

  14. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE Computer Society, pp 779–788. https://doi.org/10.1109/CVPR.2016.91

  15. Rumelhart D, Hinton G, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533–536. https://doi.org/10.1038/323533a0

    Article  MATH  Google Scholar 

  16. Sabol P et al (2020) Explainable classifier for improving the accountability in decision-making for colorectal cancer diagnosis from histopathological images. J Biomed Inf 109:103–523. https://doi.org/10.1016/j.jbi.2020.103523

    Article  Google Scholar 

  17. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: 2017 IEEE international conference on computer vision (ICCV), pp 618–626. https://doi.org/10.1109/ICCV.2017.74

  18. Silver D et al (2017) Mastering the game of go without human knowledge. Nature 550:354–358. https://doi.org/10.1038/nature24270

    Article  Google Scholar 

  19. Silver D et al (2018) A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science 362(6419):1140–1144. https://doi.org/10.1126/science.aar6404

    Article  MathSciNet  MATH  Google Scholar 

  20. Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. In: Proceedings of the 34th international conference on machine learningm, vol 70. ICML’17, pp 3319-3328

  21. Tan C et al (2018) A survey on deep transfer learning. In: Kůrková V, Manolopoulos Y, Hammer B, Iliadis L, Maglogiannis I (eds) Artificial neural networks and machine learning–ICANN 2018. Springer International Publishing, New York, pp 270–279

    Chapter  Google Scholar 

  22. Tan J, Zhang T, Coumans E, Iscen A, Bai Y, Hafner D, Bohez S, Vanhoucke V (2018) Sim-to-real: Learning agile locomotion for quadruped robots. In: Proceedings of robotics: science and systems. https://doi.org/10.15607/RSS.2018.XIV.010.https://arxiv.org/pdf/1804.10332.pdf

  23. Weiss K et al (2016) A survey of transfer learning. J Big Data 3(9). https://doi.org/10.1186/s40537-016-0043-6

  24. Xie Z, Clary P, Dao J, Morais P, Hurst J, van de Panne M (2020) Learning locomotion skills for Cassie: iterative design and sim-to-real. In: Kaelbling LP, Kragic D, Sugiura K (eds) Proceedings of the conference on robot learning. Proceedings of machine learning research, vol 100, pp 317–329

  25. Yu W, Kumar VC, Turk G, Liu CK (2019) Sim-to-real transfer for biped locomotion .https://doi.org/10.48550/ARXIV.1903.01390.https://arxiv.org/abs/1903.01390

  26. Zeng A et al. (2018) Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 4238–4245 .https://doi.org/10.1109/IROS.2018.8593986

Download references

Author information

Authors and Affiliations

  1. Graduate School of Engineering, Chukyo University, Nagoya, Japan

    Kana Ogawa

  2. School of Engineering, Chukyo University, Nagoya, Japan

    Pitoyo Hartono

Authors
  1. Kana Ogawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pitoyo Hartono
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pitoyo Hartono.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ogawa, K., Hartono, P. Infusing common-sensical prior knowledge into topological representations of learning robots. Artif Life Robotics 27, 576–585 (2022). https://doi.org/10.1007/s10015-022-00776-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-022-00776-5

Keywords

Search

Navigation