Skip to main content

Scaffolding Haptic Attention with Controller Gating

  • 2329 Accesses

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 11727)

Abstract

A powerful concept that emerged within the field of educational psychology is scaffolding. Characterizing favourable expert-learner interaction, it can be defined as a temporal support that provides a novice an adaptable guidance to either learn tasks that would usually be beyond own capabilities or to speed up and refine the learning of manageable problems. In this work we apply the above-mentioned concept to implement a novel multi-strategy haptic exploration controller that is able to perform object identification using a robot.

In our previous work we have proposed a reinforcement learner that acquires haptic exploration capabilities for a goal-directed task by optimizing motor control in a strongly restricted attentional framework, called the haptic attention model (HAM). The resulting policy however was not characterized by a smooth energy-efficient exploration suitable for execution on a robot. In this work, we scaffold the designed learning architecture by imposing the so-called controller gating that is trained to switch between orientation and position control. Integrated in the same reinforcement learning setting as the HAM, controller gating guides and monitors the data acquisition. Inspired by the human expert scaffolding, it analyzes the HAM internal data representation, modulates the HAM weight update process, and forces data acquisition that achieves efficient and successful completion of the goal. Our computational scaffold adapts to the learner model, while it masters the skill. The evaluation demonstrated that it is more likely for the trained model to change either location or orientation than simultaneously change both, which significantly improves the smoothness and the energy-efficiency of the resulting exploration.

Keywords

  • Haptic exploration
  • Reinforcement learning
  • Scaffolding
  • Gating
  • Haptic glances
  • Exploratory procedures
  • Robotics

A. Moringen and S. Fleer—Contributed equally to this work.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-30487-4_51
  • Chapter length: 16 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   99.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-30487-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   129.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

Notes

  1. 1.

    A visualization of a participant performing haptic search [13] includes both jumps and a focused local shape explorations can be found in the supplementary material (Haptic_Search_MHSB.mp4).

  2. 2.

    http://gazebosim.org.

  3. 3.

    https://www.ros.org.

  4. 4.

    An example of several haptic glances can be found in the supplementary material (KUKA.mp4).

  5. 5.

    Myrmex-Gazebo plugin library: https://github.com/ubi-agni/gazebo_tactile_plugins.

References

  1. Ba, J., Mnih, V., Kavukcuoglu, K.: Multiple object recognition with visual attention. CoRR abs/1412.7755 (2014). http://arxiv.org/abs/1412.7755

  2. Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, ACM, pp. 41–48 (2009)

    Google Scholar 

  3. Chu, V., et al.: Using robotic exploratory procedures to learn the meaning of haptic adjectives. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), IEEE, pp. 3048–3055 (2013)

    Google Scholar 

  4. Fleer, S., Moringen, A., Klatzky, R.L., Ritter, H.J.: Learning efficient haptic shape exploration with a rigid tactile sensor array. CoRR abs/1902.07501 (2019). http://arxiv.org/abs/1902.07501

  5. Hammond, J.: Scaffolding: teaching and learning in language and literacy education. ERIC (2001)

    Google Scholar 

  6. Hayhoe, M., Ballard, D.: Eye movements in natural behavior. Trends Cogn. Sci. 9(4), 188–194 (2005)

    CrossRef  Google Scholar 

  7. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    CrossRef  Google Scholar 

  8. Klatzky, R.L., Lederman, S.J., Reed, C.L.: There’s more to touch than meets the eye: the salience of object attributes for haptics with and without vision. J. Exp. Psychol. 116, 356 (1987)

    CrossRef  Google Scholar 

  9. Klatzky, R.L., Lederman, S.J.: Identifying objects from a haptic glance. Percept. Psychophysics 57(8), 1111–1123 (1995). https://doi.org/10.3758/BF03208368

    CrossRef  Google Scholar 

  10. Lee, M.A., et al.: Making sense of vision and touch: self-supervised learning of multimodal representations for contact-rich tasks. arxiv (2019)

    Google Scholar 

  11. Mathe, S., Sminchisescu, C.: Action from still image dataset and inverse optimal control to learn task specific visual scanpaths. In: Advances in Neural Information Processing Systems, pp. 1923–1931 (2013)

    Google Scholar 

  12. Mnih, V., Heess, N., Graves, A., Kavukcuoglu, K.: Recurrent models of visual attention. CoRR abs/1406.6247 (2014). http://arxiv.org/abs/1406.6247

  13. Moringen, A., Aswolinskij, W., Buescher, G., Walck, G., Haschke, R., Ritter, H.: Modeling target-distractor discrimination for haptic search in a 3D environment. In: BioRob (2018)

    Google Scholar 

  14. Nesterov, Y.: A method for solving the convex programming problem with convergence rate \(O(1/k^2)\). In: Dokl. Akad. Nauk SSSR, pp. 543–547 (1983)

    Google Scholar 

  15. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Applied Optimization. Springer, Boston (2004). https://doi.org/10.1007/978-1-4419-8853-9

    CrossRef  MATH  Google Scholar 

  16. Noë, A.: Action in Perception. Representation and Mind. MIT Press, Cambridge (2004)

    Google Scholar 

  17. van de Pol, J., Volman, M., Beishuizen, J.: Scaffolding in teacher-student interaction: a decade of research. Educ. Psychol. Rev. 22(3), 271–296 (2010)

    CrossRef  Google Scholar 

  18. Pratama, M., Lu, J., Lughofer, E., Zhang, G., Anavatti, S.: Scaffolding type-2 classifier for incremental learning under concept drifts. Neurocomputing 191, 304–329 (2016). https://doi.org/10.1016/j.neucom.2016.01.049

    CrossRef  Google Scholar 

  19. Rouhafzay, G., Cretu, A.: Object recognition from haptic glance at visually salient locations. IEEE Trans. Instrum. Measur. 1–11 (2019). https://doi.org/10.1109/TIM.2019.2905906

  20. Schurmann, C., Koiva, R., Haschke, R., Ritter, H.: A modular high-speed tactile sensor for human manipulation research. In: 2011 IEEE World Haptics Conference (WHC 2011), IEEE, pp. 339–344 (2011)

    Google Scholar 

  21. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge (2018)

    MATH  Google Scholar 

  22. Thomaz, A.L.: Socially guided machine learning. Ph.D. thesis, Computer Science Department Faculty Publication Series (2006)

    Google Scholar 

  23. Thomaz, A.L., Breazeal, C.: Reinforcement learning with human teachers: evidence of feedback and guidance with implications for learning performance. In: Association for the Advancement of Artificial Intelligence - 2006, Boston, MA, pp. 1000–1005 (2006)

    Google Scholar 

  24. Tian, S., et al.: Manipulation by feel: touch-based control with deep predictive models. arxiv (2019)

    Google Scholar 

  25. Williams, R.J.: Toward a theory of reinforcement-learning connectionist systems. Technical Report NU-CCS-88-3, Northeastern University (1988)

    Google Scholar 

  26. Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)

    MATH  Google Scholar 

  27. Zydney, J.M.: Scaffolding. In: Seel, N.M. (ed.) Encyclopedia of the Sciences of Learning, pp. 2913–2916. Springer, Boston (2012). https://doi.org/10.1007/978-1-4419-1428-6

    CrossRef  Google Scholar 

Download references

Acknowledgments

This research/work was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG). We would also like to express our great appreciation to Guillaume Walck.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sascha Fleer .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Moringen, A., Fleer, S., Ritter, H. (2019). Scaffolding Haptic Attention with Controller Gating. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation. ICANN 2019. Lecture Notes in Computer Science(), vol 11727. Springer, Cham. https://doi.org/10.1007/978-3-030-30487-4_51

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30487-4_51

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30486-7

  • Online ISBN: 978-3-030-30487-4

  • eBook Packages: Computer ScienceComputer Science (R0)