Cognitive Architecture for a Companion Robot: Speech Comprehension and Real-World Awareness

Abstract

Companion robots should support natural communication with humans referencing the events in the environment and should respond with compound communicative reactions, reacting to incoming utterances, actions, gazes and other surrounding events. We represent a project of F-2 companion robot, where we implement unified representation types for (a) incoming speech semantics and (b) real events in the environment, registered by computer vision system. We rely of a classic linguistic notion of “case-frame” and represent each event as a predicate and a number of valencies: agent, patient, instrument, etc. Predicate and each valency refer to a referent (“id” of event or object) and is also represented by a list of semantic markers. These unified representations allow the robot to react to speech semantics and real events in a unified way. To simulate communicative reactions, we use a number of scripts (productions) which are activated by incoming case-frames, generate behavioral cues and get disactivated, once the cue is executed on the robot. The balance of activations allows the robot to select the most significant reactions or to simulate emotional behavior, expressively reacting to some minor stimulus. This architecture allows to eliminate the bottleneck on the stage of robot reactions, effectively process diverse incoming stimuli and simulate rich and compound communicative behavior on the robot.