Fusion of Multichannel Biosignals Towards Automatic Emotion Recognition
Endowing the computer with the ability to recognize human emotional states is the most important prerequisites for realizing an affect-sensitive human-computer interaction. In this paper, we deal with all the essential stages of an automatic emotion recognition system using multichannel physiological measures, from data collection to the classification process. Particularly we develop two different classification methods, feature-level fusion and emotion-specific classification scheme. Four-channel biosensors were used to measure electromyogram, electrocardiogram, skin conductivity, and respiration changes while subjects were listening to music. A wide range of physiological features from various analysis domains is proposed to correlate them with emotional states. Classification of four musical emotions is performed by using feature-level fusion combined with an extended linear discriminant analysis (pLDA). Furthermore, by exploiting a dichotomic property of the 2D emotion model, we developed a novel scheme of emotion-specific multilevel dichotomous classification (EMDC) and compare its performance with direct multiclass classification using the pLDA feature-level fusion. Improved recognition accuracy of 95% and 70% for subject-dependent and subject-independent classification, respectively, is achieved by using the EMDC scheme.
KeywordsBiosignal Multisensor data fusion ECG EMG SC RSP Emotion recognition Multichannel biosignals Pattern recognition Human-computer interaction Music and emotion
Unable to display preview. Download preview PDF.
- 2.J. Healey and R. W. Picard, Digital processing of affective signals, in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Proc., Seattle, WA, 1998, pp. 3749–3752.Google Scholar
- 4.F. Nasoz, K. Alvarez, C. Lisetti, and N. Finkelstein, Emotion recognition from physiological signals for presence technologies, Int. J. Cogn., Technol., and Work, 6(1), 4–14, 2003.Google Scholar
- 8.P. W. Kamen, H. Krum, and A. M. Tonkin, Poincare plot of heart rate variability allows quantitative display of parasympathetic nervous activity, Clin. Sci., 91, 201–208, 1996.Google Scholar
- 9.J. Richmann and J. Moorman, Physiological time series analysis using approximate entropy and sample entropy, Am. J. Physiol. Heart Circ. Physiol. 278, H2039, 2000.Google Scholar
- 11.R. Thuraisingham and G. Gottwald, On multiscale entropy analysis for physiological data, Physica A, 336, 323–333, 2006.Google Scholar
- 12.J. Ye and Q. Li, A two-stage linear discriminant analysis via Qr-decomposition, IEEE Trans. on PAMI, 27(6), 929–941, 2005.Google Scholar
- 13.J. Kittler, Feature Selection and Extraction. San Diego, CA:Academic Press, Inc., 59–83, 1986.Google Scholar