Asynchronous Classification of Error-Related Potentials in Human-Robot Interaction

Abstract

The use of implicit evaluations of humans such as electroencephalogram (EEG)-based human feedback is relevant for robot applications, e.g., robot learning or corrections of robot’s actions. In the presented study, we implemented a scenario, in which a simulated robot communicates with its human partner through speech and gestures. The robot announces its intention verbally and selects the appropriate action using pointing gestures. The human partner in turn implicitly evaluates whether the robot’s verbal announcement matches the robot’s action choice. Error-related potentials (ErrPs) are expressions of this implicit evaluation, which are triggered in case of discrepancies between the robot’s verbal announcement and the corresponding actions (pointing gestures) chosen by the robot. In our scenario, the task takes a long time. Therefore, asynchronous EEG classifications that continuously segment EEGs are advantageous or even necessary. However, asynchronous EEG classifications are challenging due to the large number of false positives during the long task time of the robot. In this work, we propose an approach to improve asynchronous classification performance by selecting and extracting features that are only relevant for EEG classifications in the long time series that the robot needs to perform tasks. We achieved a high classification performance, i.e., a mean balanced accuracy of $91\%$ across all subjects. However, we also found some differences between subjects in classification performance. In future work, it is useful to extend the proposed approach of forward and backward sliding windows and their combinations with individual feature selection adaptation to avoid the variability of classification performance between subjects.

Keywords

Supported by the Federal Ministry for Economic Affairs and Climate Action (BMWK) FKZ: FKZ 50RA1701 and Federal Ministry for Education and Research (BMBF) FKZ: 01IW21002.