Biological and Behavioral Information-Based Method of Predicting Listener Emotions Toward Speaker Utterances During Group Discussion

Abstract

There are many types of learning environments presented in higher education venues, requiring the development of a diverse repertoire of learning abilities. Group discussion (GD) is one such learning environment, and the students who participate require multiple communication skills. Technically, it is desirable for a student participating in a GD to understand other participants’ emotional reactions toward their utterances to improve their locution, content, etc. The purpose of this research is to predict listeners’ emotions in response to speakers’ utterances using multimodal sensors. In experiments, GDs were conducted with 20 students. Six basic emotions were recorded as responses to speakers’ utterances during the GDs using an emotional annotation tool. This study predicted the occurrence of the emotions by using an accelerometer, an electrocardiogram (ECG), and an electromyography (EMG). From sensor data, 56 features in the time and frequency domains were calculated, and Kruskal–Wallis tests and multiple comparison tests were performed to investigate whether there were significant differences among the features collected. As a result, there were significant differences among the groups of six basic emotions (${p}<0.01$). As an application, it has been shown that negative and positive emotions could be distinguished by support vector machine (SVM) with 76% F1.