Cross-Corpus Experiments on Laughter and Emotion Detection in HRI with Elderly People

  • Marie TahonEmail author
  • Mohamed A. Sehili
  • Laurence Devillers
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9388)


Social Signal Processing such as laughter or emotion detection is a very important issue, particularly in the field of human-robot interaction (HRI). At the moment, very few studies exist on elderly-people’s voices and social markers in real-life HRI situations. This paper presents a cross-corpus study with two realistic corpora featuring elderly people (ROMEO2 and ARMEN) and two corpora collected in laboratory conditions with young adults (JEMO and OFFICE). The goal of this experiment is to assess how good data from one given corpus can be used as a training set for another corpus, with a specific focus on elderly people voices. First, clear differences between elderly people real-life data and young adults laboratory data are shown on acoustic feature distributions (such as \(F_0\) standard deviation or local jitter). Second, cross-corpus emotion recognition experiments show that elderly people real-life corpora are much more complex than laboratory corpora. Surprisingly, modeling emotions with an elderly people corpus do not generalize to another elderly people corpus collected in the same acoustic conditions but with different speakers. Our last result is that laboratory laughter is quite homogeneous across corpora but this is not the case for elderly people real-life laughter.


Laughter recognition Emotion recognition Human-Robot Interaction Elderly people Cross-corpus protocol 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vinciarelli, A., Pantic, M., Bourlard, H., Pentland, A.: Social signals, their function, and automatic analysis: a survey. In: Conference on Multimodal Interfaces (ACM), Chania, Greece, pp. 61–68 (2008)Google Scholar
  2. 2.
    Delaborde, A., Devillers, L.: Use of nonverbal speech cues in social interaction between human and robot: emotional and interactional markers. In: International Workshop on Affective Interaction in Natural Environements (AFFINE), Firenze, Italy (2010)Google Scholar
  3. 3.
    Breazeal, C.: Emotion and sociable humanoid robots. Human Computer Studies 59, 119–155 (2003)CrossRefGoogle Scholar
  4. 4.
    Scherer, S., Glodek, M., Schwenker, F., Campbell, N., Palm, G.: Spotting laughter in natural multiparty conversations: A comparison of automatic online and offline approaches using audiovisual data. ACM Transactions on Interactive Intelligent Systems (TiiS) 2(1), Article No. 4 (2012). Special Issue on Affective Interaction in Natural EnvironmentsGoogle Scholar
  5. 5.
    Schuller, B., Batliner, A., Steidl, S., Seppi, D.: Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Communication 53(9), 1062–1087 (2011). Special Issue on Sensing Emotion and Affect - Facing Realism in Speech ProcessingCrossRefGoogle Scholar
  6. 6.
    Batliner, A., Steidl, S., Nöth, E.: Laryngealizations and emotions: how many babushkas? In: Proc. Internat. Workshop on Paralinguistic Speech - Between Models and Data (ParaLing’ 07), Saarbrucken, Germany, pp. 17–22 (2007)Google Scholar
  7. 7.
    Batliner, A., Hacker, C., Steidl, S., Nöth, E., D’Arcy, S., Russell, M., Wong, M.: You stupid tin box - children interacting with the aibo robot: a cross-linguistic emotional speech corpus. In: LREC, Lisbon, Portugal, pp. 171–174 (2004)Google Scholar
  8. 8.
    Delaborde, A., Tahon, M., Barras, C., Devillers, L.: Affective links in a child-robot interaction. In: LREC, Valletta, Malta (2010)Google Scholar
  9. 9.
    McKeown, G., Valstar, M., Cowie, R., Pantic, M., Schröder, M.: The semaine database: annotated multimodal records of emotionally coloured conversations between a person and a limited agent. IEEE Transactions on Affective Computing 3(1), 5–17 (2012)CrossRefGoogle Scholar
  10. 10.
    Tahon, M., Delaborde, A., Devillers, L.: Real-life emotion detection from speech in human-robot interaction: experiments across diverse corpora with child and adult voices. In: Interspeech, Firenze, Italia (2011)Google Scholar
  11. 11.
    Chastagnol, C., Clavel, C., Courgeon, M., Devillers, L.: Designing an emotion detection system for a socially-intelligent human-robot interaction. In: Towards a Natural Interaction with Robots, Knowbots and Smartphones: Putting Spoken Dialog Systems into Practice. Springer (2013)Google Scholar
  12. 12.
    Sehili, M.A., Yang, F., Leynaert, V., Devillers, L.: A corpus of social interaction between nao and elderly people. In: International Workshop on Emotion, Social Signals, Sentiment & Linked Open Data, Satellite of LREC (2014)Google Scholar
  13. 13.
    Tahon, M., Delaborde, A., Barras, C., Devillers, L.: A corpus for identification of speakers and their emotions. In: LREC, Valletta, Malta (2010)Google Scholar
  14. 14.
    Schuller, B., Zhang, Z., Weninger, F., Rigoll, G.: Selecting training data for cross-corpus speech emotion recognition: prototypicality vs. generalization. In: AVIOS Speech Processing, Tel-Aviv, Israel (2011)Google Scholar
  15. 15.
    Tahon, M., Devillers, L.: Laughter detection for on-line human-robot interaction. In: Interdisciplinary Workshop on Laughter and Non-verbal Vocalisations in Speech, Enschede, Netherlands (2015)Google Scholar
  16. 16.
    Brendel, M., Zaccarelli, R., Devillers, L.: Building a system for emotions detection from speech to control an affective avatar. In: LREC, Valletta, Malta (2010)Google Scholar
  17. 17.
    Ververidis, D., Kotropoulos, C.: Emotional speech recognition: Ressources, features and methods. Speech Communication 48(9), 1162–1181 (2006)CrossRefGoogle Scholar
  18. 18.
    Schuller, B., Batliner, A.: Computational Paralinguistics: Emotion, Affect and Personality in Speech and Language Processing. John Wiley & Sons (2013)Google Scholar
  19. 19.
    Bachorowski, J.-A., Smoski, M.J., Owren, M.J.: The acoustic features of human laughter. Journal of the Acoustical Society of America 110(3), 1581–1597 (2001)CrossRefGoogle Scholar
  20. 20.
    Campbell, N.: Perception of affect in speech - towards an automatic processing of paralinguistic information in spoken conversation. In: International Conference on Spoken Language Processing, Jeju Island, Korea (2004)Google Scholar
  21. 21.
    Szameitat, D.P., Darwin, C.J., Szameitat, A.J., Wildgruber, D., Alter, K.: Formant characteristics of human laughter. Journal of Voice 25(1), 32–38 (2011)CrossRefGoogle Scholar
  22. 22.
    Devillers, L., Tahon, M., Sehili, M., Delaborde, A.: Inference of human beings’ emotional states from speech in human-robot interactions. International Journal of Social Robotics, Special Issue on Developmental Social Robotics (in press, 2015)Google Scholar
  23. 23.
    Schröder, M.: Experimental study of affect bursts. Speech Communication 40(1–2), 99–116 (2003). Special Session on Speech and EmotionCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is distributed under the terms of the Creative Commons Attribution Noncommercial License, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Authors and Affiliations

  • Marie Tahon
    • 1
    Email author
  • Mohamed A. Sehili
    • 1
  • Laurence Devillers
    • 1
    • 2
  1. 1.Human-Machine Communication DepartmentLIMSI-CNRSOrsayFrance
  2. 2.University Paris-Sorbonne IVParisFrance

Personalised recommendations