Scaffolding Haptic Attention with Controller Gating

Abstract

A powerful concept that emerged within the field of educational psychology is scaffolding. Characterizing favourable expert-learner interaction, it can be defined as a temporal support that provides a novice an adaptable guidance to either learn tasks that would usually be beyond own capabilities or to speed up and refine the learning of manageable problems. In this work we apply the above-mentioned concept to implement a novel multi-strategy haptic exploration controller that is able to perform object identification using a robot.

In our previous work we have proposed a reinforcement learner that acquires haptic exploration capabilities for a goal-directed task by optimizing motor control in a strongly restricted attentional framework, called the haptic attention model (HAM). The resulting policy however was not characterized by a smooth energy-efficient exploration suitable for execution on a robot. In this work, we scaffold the designed learning architecture by imposing the so-called controller gating that is trained to switch between orientation and position control. Integrated in the same reinforcement learning setting as the HAM, controller gating guides and monitors the data acquisition. Inspired by the human expert scaffolding, it analyzes the HAM internal data representation, modulates the HAM weight update process, and forces data acquisition that achieves efficient and successful completion of the goal. Our computational scaffold adapts to the learner model, while it masters the skill. The evaluation demonstrated that it is more likely for the trained model to change either location or orientation than simultaneously change both, which significantly improves the smoothness and the energy-efficiency of the resulting exploration.

A. Moringen and S. Fleer—Contributed equally to this work.