Advertisement

Analysis and Assessment of AvID: Multi-Modal Emotional Database

  • Rok Gajšek
  • Vitomir Štruc
  • Boštjan Vesnicer
  • Anja Podlesek
  • Luka Komidar
  • France Mihelič
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5729)

Abstract

The paper deals with the recording and the evaluation of a multi modal (audio/video) database of spontaneous emotions. Firstly, motivation for this work is given and different recording strategies used are described. Special attention is given to the process of evaluating the emotional database. Different kappa statistics normally used in measuring the agreement between annotators are discussed. Following the problems of standard kappa coefficients, when used in emotional database assessment, a new time-weighted free-marginal kappa is presented. It differs from the other kappa statistics in that it weights each utterance’s particular score of agreement based on the duration of the utterance. The new method is evaluated and the superiority over the standard kappa, when dealing with a database of spontaneous emotions, is demonstrated.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    LDC: SUSAS (Speech Under Simulated and Actual Stress). Language Data Consortium (1999), http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC99S78
  2. 2.
    LDC: Emotional Prosody Speech and Transcripts. Language Data Consortium (2002), http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2002S28
  3. 3.
    Martin, O., Kotsia, I., Macq, B., Pitas, I.: The eNTERFACE 2005 Audio-Visual Emotion Database. In: Proc. of the 22nd International Conference on Data Engineering Workshops (ICDEW 2006), p. 8 (2006)Google Scholar
  4. 4.
    Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., Weiss, B.: A Database of German Emotional Speech. In: Interspeech 2005, pp. 1517–1520 (2005)Google Scholar
  5. 5.
    Battocchi, A., Pianesi, F.: DAFEX: Un Database Di Espressioni Facciali Dinamiche. In: Proceedings of the SLI-GSCP Workshop ”Comunicazione Parlata e Manifestazione delle Emozioni” (2004)Google Scholar
  6. 6.
    Randolph, J.J.: Free-marginal multirater kappa: An alternative to Fleiss’ fixed-marginal multirater kappa. In: Joensuu University Learning and Instruction Symposium 2005, Joensuu, Finland (2005)Google Scholar
  7. 7.
    Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychological Bulletin 76(5), 378–382 (1971)CrossRefGoogle Scholar
  8. 8.
    Cohen, J.: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1), 37–46 (1960)CrossRefGoogle Scholar
  9. 9.
    Randolph, J.J.: Free-marginal multirater kappa: An alternative to Fleiss’ fixed-marginal multirater kappa. In: Joensuu University Learning and Instruction Symposium 2005, Joensuu, Finland (2005)Google Scholar
  10. 10.
    Callejas, Z., López-Cózar, R.: On the Use of Kappa Coefficients to Measure the Reliability of the Annotation of Non-acted Emotions. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Pieraccini, R., Weber, M. (eds.) PIT 2008. LNCS (LNAI), vol. 5078, pp. 221–232. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Gajšek, R., Štruc, V., Mihelič, F., Podlesek, A., Komidar, L., Sočan, G., Bajec, B.: Multi-Modal Emotional Database: AvID. Informatica (Ljubljana) 33(1), 101–106 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Rok Gajšek
    • 1
  • Vitomir Štruc
    • 1
  • Boštjan Vesnicer
    • 1
  • Anja Podlesek
    • 2
  • Luka Komidar
    • 2
  • France Mihelič
    • 1
  1. 1.Faculty of Electrical EngineeringUniversity of LjubljanaLjubljanaSlovenia
  2. 2.Faculty of ArtsUniversity of LjubljanaLjubljanaSlovenia

Personalised recommendations