Towards Sensomotory Coordination of Vision and Action in Humanoid Agents

Abstract

The classical approach corncerning the coordination of sensory and motor systems is characterized by purely reactive behaviour: all sensory input (tactile, accoustical, visual) is unidirectionally transformed into output signals that are sent to the motor system. Both involved cortical systems (parietal cortex and premotor cortex) act in a strictly unitary way. Recent progress in anatomical and neurophysiological research in primate brains has shown that both cortical systems are far from acting unitarily. They rather consist of several distinct coupled areas exhibiting specific functional properties called basic behaviours. These functional properties arise from the reciprocal coupling of one parietal area and its corresponding premotor area. This coupling is based on the bidirectional interaction of multimodal neurons located in both areas. Similar functional properties are arranged in streams: AIP-F5 enables grasping behaviour, LIP-F4 enables reaching behaviour, and LIP-FEF controls eye movement. These pragmatic streams together with semantic streams like PIP-IT may be regarded as the distinct building blocks of the associative cortex which is traditionally viewed as a monolithic computational unit. Basic behaviours act as active action detectors that are selected by overall controller systems like the ratiomorph apparatus.