Robot Adaptivity

Overview

These lecture notes review neural network techniques for achieving adaptivity in autonomous agents. They are structured around three main topics:

• Neural learning algorithms. A distinction is made between the short- term and long-term levels of processing in neural networks. Long-term processing is what is referred to as “learning”, although it should be more properly called “adaptation”, and it amounts to modifying the connection weights. A classification of the learning tasks, learning rules and learning models that have appeared in the literature is presented. In particular, learning rules are grouped into three classes: correlational rules (Hebbian), error-minimization rules (perception, LMS, back-propagation) and reinforcement rules (associative search, associative reward-penalty)

• Applications to robot control. Within the field of control, neural networks have been applied to system identification and to the design of controllers. The two main approaches that have arisen are direct-inverse modelling and forward modelling. In the more specific field of robot control, efforts have been oriented to the learning of inverse models (inverse robot kinematics and dynamics) and of goal-oriented sensorimotor mappings (path finding, hand-eye coordination). Two systems are next described in detail: topology-conserving maps for learning visuomotor coordination of a robot arm [32], where a correlational rule is combined with an error-minimization rule; and the reinforcement-based path finder for mobile robots described in [26]

• Limitations of neural control: a need for planning? This block is intended to foster a discussion of what the advantages and limitations of subsymbolic and symbolic approaches are. Neural controllers obviate the programming phase by exploiting learning, but their generality and opacity make it impossible to take advantage of problem-specific information. This results in very long learning times. Motion planners, on the other hand, rely on geometric reasoning and heuristic search, thus allowing the use of domain-specific knowledge to gain efficiency. However, they are hard to program and computationally expensive when high precision is required. A case is made out for the combination of a one-shot symbolic acquisition of knowledge (initial setting of the system) and a subsymbolic adaptation of skills through repetitive trial and error (subsequent tuning).

The author acknowledges support from the comisión Interministerial de Ciencia y Tecnología (CICYT) under the project SUBSIM (TAP93-0451) and from the ESPRIT III Program of the European Community under contract No. 7274 (project “B-LEARN II: Behavioural Learning: Combining Sensing and Action”).