Abstract
A powerful concept that emerged within the field of educational psychology is scaffolding. Characterizing favourable expert-learner interaction, it can be defined as a temporal support that provides a novice an adaptable guidance to either learn tasks that would usually be beyond own capabilities or to speed up and refine the learning of manageable problems. In this work we apply the above-mentioned concept to implement a novel multi-strategy haptic exploration controller that is able to perform object identification using a robot.
In our previous work we have proposed a reinforcement learner that acquires haptic exploration capabilities for a goal-directed task by optimizing motor control in a strongly restricted attentional framework, called the haptic attention model (HAM). The resulting policy however was not characterized by a smooth energy-efficient exploration suitable for execution on a robot. In this work, we scaffold the designed learning architecture by imposing the so-called controller gating that is trained to switch between orientation and position control. Integrated in the same reinforcement learning setting as the HAM, controller gating guides and monitors the data acquisition. Inspired by the human expert scaffolding, it analyzes the HAM internal data representation, modulates the HAM weight update process, and forces data acquisition that achieves efficient and successful completion of the goal. Our computational scaffold adapts to the learner model, while it masters the skill. The evaluation demonstrated that it is more likely for the trained model to change either location or orientation than simultaneously change both, which significantly improves the smoothness and the energy-efficiency of the resulting exploration.
Keywords
A. Moringen and S. Fleer—Contributed equally to this work.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
A visualization of a participant performing haptic search [13] includes both jumps and a focused local shape explorations can be found in the supplementary material (Haptic_Search_MHSB.mp4).
- 2.
- 3.
- 4.
An example of several haptic glances can be found in the supplementary material (KUKA.mp4).
- 5.
Myrmex-Gazebo plugin library: https://github.com/ubi-agni/gazebo_tactile_plugins.
References
Ba, J., Mnih, V., Kavukcuoglu, K.: Multiple object recognition with visual attention. CoRR abs/1412.7755 (2014). http://arxiv.org/abs/1412.7755
Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, ACM, pp. 41–48 (2009)
Chu, V., et al.: Using robotic exploratory procedures to learn the meaning of haptic adjectives. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), IEEE, pp. 3048–3055 (2013)
Fleer, S., Moringen, A., Klatzky, R.L., Ritter, H.J.: Learning efficient haptic shape exploration with a rigid tactile sensor array. CoRR abs/1902.07501 (2019). http://arxiv.org/abs/1902.07501
Hammond, J.: Scaffolding: teaching and learning in language and literacy education. ERIC (2001)
Hayhoe, M., Ballard, D.: Eye movements in natural behavior. Trends Cogn. Sci. 9(4), 188–194 (2005)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Klatzky, R.L., Lederman, S.J., Reed, C.L.: There’s more to touch than meets the eye: the salience of object attributes for haptics with and without vision. J. Exp. Psychol. 116, 356 (1987)
Klatzky, R.L., Lederman, S.J.: Identifying objects from a haptic glance. Percept. Psychophysics 57(8), 1111–1123 (1995). https://doi.org/10.3758/BF03208368
Lee, M.A., et al.: Making sense of vision and touch: self-supervised learning of multimodal representations for contact-rich tasks. arxiv (2019)
Mathe, S., Sminchisescu, C.: Action from still image dataset and inverse optimal control to learn task specific visual scanpaths. In: Advances in Neural Information Processing Systems, pp. 1923–1931 (2013)
Mnih, V., Heess, N., Graves, A., Kavukcuoglu, K.: Recurrent models of visual attention. CoRR abs/1406.6247 (2014). http://arxiv.org/abs/1406.6247
Moringen, A., Aswolinskij, W., Buescher, G., Walck, G., Haschke, R., Ritter, H.: Modeling target-distractor discrimination for haptic search in a 3D environment. In: BioRob (2018)
Nesterov, Y.: A method for solving the convex programming problem with convergence rate \(O(1/k^2)\). In: Dokl. Akad. Nauk SSSR, pp. 543–547 (1983)
Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Applied Optimization. Springer, Boston (2004). https://doi.org/10.1007/978-1-4419-8853-9
Noë, A.: Action in Perception. Representation and Mind. MIT Press, Cambridge (2004)
van de Pol, J., Volman, M., Beishuizen, J.: Scaffolding in teacher-student interaction: a decade of research. Educ. Psychol. Rev. 22(3), 271–296 (2010)
Pratama, M., Lu, J., Lughofer, E., Zhang, G., Anavatti, S.: Scaffolding type-2 classifier for incremental learning under concept drifts. Neurocomputing 191, 304–329 (2016). https://doi.org/10.1016/j.neucom.2016.01.049
Rouhafzay, G., Cretu, A.: Object recognition from haptic glance at visually salient locations. IEEE Trans. Instrum. Measur. 1–11 (2019). https://doi.org/10.1109/TIM.2019.2905906
Schurmann, C., Koiva, R., Haschke, R., Ritter, H.: A modular high-speed tactile sensor for human manipulation research. In: 2011 IEEE World Haptics Conference (WHC 2011), IEEE, pp. 339–344 (2011)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge (2018)
Thomaz, A.L.: Socially guided machine learning. Ph.D. thesis, Computer Science Department Faculty Publication Series (2006)
Thomaz, A.L., Breazeal, C.: Reinforcement learning with human teachers: evidence of feedback and guidance with implications for learning performance. In: Association for the Advancement of Artificial Intelligence - 2006, Boston, MA, pp. 1000–1005 (2006)
Tian, S., et al.: Manipulation by feel: touch-based control with deep predictive models. arxiv (2019)
Williams, R.J.: Toward a theory of reinforcement-learning connectionist systems. Technical Report NU-CCS-88-3, Northeastern University (1988)
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)
Zydney, J.M.: Scaffolding. In: Seel, N.M. (ed.) Encyclopedia of the Sciences of Learning, pp. 2913–2916. Springer, Boston (2012). https://doi.org/10.1007/978-1-4419-1428-6
Acknowledgments
This research/work was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG). We would also like to express our great appreciation to Guillaume Walck.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Moringen, A., Fleer, S., Ritter, H. (2019). Scaffolding Haptic Attention with Controller Gating. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation. ICANN 2019. Lecture Notes in Computer Science(), vol 11727. Springer, Cham. https://doi.org/10.1007/978-3-030-30487-4_51
Download citation
DOI: https://doi.org/10.1007/978-3-030-30487-4_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30486-7
Online ISBN: 978-3-030-30487-4
eBook Packages: Computer ScienceComputer Science (R0)