Two-Level Fusion to Improve Emotion Classification in Spoken Dialogue Systems

Abstract

This paper proposes a technique to enhance emotion classification in spoken dialogue systems by means of two fusion modules. The first combines emotion predictions generated by a set of classifiers that deal with different kinds of information about each sentence uttered by the user. To do this, the module employs several fusion methods that produce other predictions about the emotional state of the user. The predictions are the input to the second fusion module, where they are combined to deduce the user’s emotional state. Experiments have been carried out considering two emotion categories (‘Non-negative’ and ‘Negative’) and classifiers that deal with prosodic, acoustic, lexical and dialogue acts information. The results show that the first fusion module significantly increases the classification rates of a baseline and the classifiers working separately, as has been observed previously in the literature. The novelty of the technique is the inclusion of the second fusion module, which enhances classification rate by 2.25% absolute.

This work has been funded by the Spanish project HADA TIN2007-64718, and the grant no. 1QS108040569 of the Agency of Sciences of the Czech Republic.