Advertisement

Adaptive On-Line Neural Network Retraining for Real Life Multimodal Emotion Recognition

  • Spiros Ioannou
  • Loic Kessous
  • George Caridakis
  • Kostas Karpouzis
  • Vered Aharonson
  • Stefanos Kollias
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4131)

Abstract

Emotions play a major role in human-to-human communication enabling people to express themselves beyond the verbal domain. In recent years, important advances have been made in unimodal speech and video emotion analysis where facial expression information and prosodic audio features are treated independently. The need however to combine the two modalities in a naturalistic context, where adaptation to specific human characteristics and expressivity is required, and where single modalities alone cannot provide satisfactory evidence, is clear. Appropriate neural network classifiers are proposed for multimodal emotion analysis in this paper, in an adaptive framework, which is able to activate retraining of each modality, whenever deterioration of the respective performance is detected. Results are presented based on the IST HUMAINE NoE naturalistic database; both facial expression information and prosodic audio features are extracted from the same data and feature-based emotion analysis is performed through the proposed adaptive neural network methodology.

Keywords

Emotion Recognition Network Weight Gradient Projection Method Neural Network Classifier Weight Increment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Young, J.W.: Head and face anthropometry of adult U.S. civilians, FAA Civil Aeromedical Institute (1993)Google Scholar
  2. 2.
    Doulamis, A., Doulamis, N., Kollias, S.: On-line Retrainable Neural Networks: Improving the Performance of Neural Networks in Image Analysis Problems. IEEE Transactions on Neural Networks 11(1), 137–157 (2000)CrossRefGoogle Scholar
  3. 3.
    Krog, A., Vedelsby, J.: Neural network ensembles, cross validation and active learning. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in neural information processing systems 7, pp. 231–238. MIT Press, Cambridge (1995)Google Scholar
  4. 4.
    Ioannou, S., Raouzaiou, A., Tzouvaras, V., Mailis, T., Karpouzis, K., Kollias, S.: Emotion recognition through facial expression analysis based on a neurofuzzy network. Special Issue on Emotion: Understanding & Recognition, Neural Networks 18(4), 423–435 (2005)Google Scholar
  5. 5.
    HUMAINE, Human-Machine Interaction Network on Emotion IST-2002-2.3.1.6, http://emotion-research.net/
  6. 6.
    Fransens, R., De Prins, J.: SVM-based Nonparametric Discriminant Analysis. In: An Application to Face Detection, October 13 - 16, 2003. Ninth IEEE International Conference on Computer Vision, vol. 2 (2003)Google Scholar
  7. 7.
    Kollias, S., Anastassiou, D.: An adaptive least squares algorithm for the efficient training of artificial neural networks. IEEE Transactions on Circuits and Systems 36(8), 1092–1101 (1989)CrossRefGoogle Scholar
  8. 8.
    Hagan, M.T., Menhaj, M.: Training feedforward networks with the Marquardt algorithm. IEEE Transactions on Neural Networks 5(6), 989–993 (1994)CrossRefGoogle Scholar
  9. 9.
    Raouzaiou, A., Tsapatsoulis, N., Karpouzis, K., Kollias, S.: Parameterized facial expression synthesis based on MPEG-4. EURASIP Journal on Applied Signal Processing 2002(10) (October 2002)Google Scholar
  10. 10.
    Baldonado, M., Chang, C.-C.K., Gravano, L., Paepcke, A.: The Stanford Digital Library Metadata Architecture. Int. J. Digit. Libr. 1, 108–121 (1997)CrossRefGoogle Scholar
  11. 11.
    Mertens, P.: The Prosogram: Semi-Automatic Transcription of Prosody based on a Tonal Perception Model. In: Bel, B., Marlien, I. (eds.) Proceedings of Speech Prosody, Nara (Japan), March 23-26 (2004), ISBN 2-9518233-1-2Google Scholar
  12. 12.
    Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.: Emotion Recognition in Human-Computer Interaction. IEEE Signal Processing Magazine (2001)Google Scholar
  13. 13.
    Picard, R.W.: Affective Computing. MIT Press, Cambridge (2000)Google Scholar
  14. 14.
    Cowie, R., Douglas-Cowie, E.: Automatic statistical analysis of the signal and prosodic signs of emotion in speech. In: Proceedings of the 4th International Conference of Spoken Language Processing, Philadelphia, USA, pp. 1989–1992 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Spiros Ioannou
    • 1
  • Loic Kessous
    • 2
  • George Caridakis
    • 1
  • Kostas Karpouzis
    • 1
  • Vered Aharonson
    • 2
  • Stefanos Kollias
    • 1
  1. 1.School of Electrical and Computer EngineeringNational Technical University of AthensPolitechnioupoli, ZographouGreece
  2. 2.Tel Aviv Academic College of EngineeringTel AvivIsrael

Personalised recommendations