Multimodal Analysis of Laughter for an Interactive System

  • Jérôme Urbain
  • Radoslaw Niewiadomski
  • Maurizio Mancini
  • Harry Griffin
  • Hüseyin Çakmak
  • Laurent Ach
  • Gualtiero Volpe
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 124)

Abstract

In this paper, we focus on the development of new methods to detect and analyze laughter, in order to enhance human-computer interactions. First, the general architecture of such a laughter-enabled application is presented. Then, we propose the use of two new modalities, namely body movements and respiration, to enrich the audiovisual laughter detection and classification phase. These additional signals are acquired using easily constructed affordable sensors. Features to characterize laughter from body movements are proposed, as well as a method to detect laughter from a measure of thoracic circumference.

Keywords

laughter multimodal analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Vettin, J., Todt, D.: Laughter in conversation: Features of occurrence and acoustic structure. Journal of Nonverbal Behavior 28(2), 93–115 (2004)CrossRefGoogle Scholar
  2. 2.
    Niewiadomski, R., Hofmann, J., Urbain, J., Platt, T., Wagner, J., Piot, B., Cakmak, H., Pammi, S., Baur, T., Dupont, S., Geist, M., Lingenfelser, F., McKeown, G., Pietquin, O., Ruch, W.: Laugh-aware virtual agent and its impact on user amusement, Saint Paul, Minnesota, USA (May 2013)Google Scholar
  3. 3.
    Niewiadomski, R., Bevacqua, E., Le, Q.A., Obaid, M., Looser, J., Pelachaud, C.: Cross-media agent platform. In: Web3D ACM Conference, Paris, France, pp. 11–19 (2011)Google Scholar
  4. 4.
    Urbain, J., Niewiadomski, R., Bevacqua, E., Dutoit, T., Moinet, A., Pelachaud, C., Picart, B., Tilmanne, J., Wagner, J.: AVLaughterCycle: Enabling a virtual agent to join in laughing with a conversational partner using a similarity-driven audiovisual laughter animation. JMUI 4(1), 47–58 (2010)Google Scholar
  5. 5.
    Urbain, J., Dutoit, T.: A phonetic analysis of natural laughter, for use in automatic laughter processing systems. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011, Part I. LNCS, vol. 6974, pp. 397–406. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Niewiadomski, R., Pammi, S., Sharma, A., Hofmann, J., Platt, T., Cruz, R., Qu, B.: Visual laughter synthesis: Initial approaches. In: Interdisciplinary Workshop on Laughter and other Non-Verbal Vocalisations in Speech, Dublin, Ireland (2012)Google Scholar
  7. 7.
    Ekman, P., Friesen, W., Hager, J.: Facial action coding system: A technique for the measurement of facial movement (2002)Google Scholar
  8. 8.
    Kennedy, L., Ellis, D.: Laughter detection in meetings. In: NIST ICASSP 2004 Meeting Recognition Workshop, Montreal, pp. 118–121 (May 2004)Google Scholar
  9. 9.
    Truong, K.P., van Leeuwen, D.A.: Automatic discrimination between laughter and speech. Speech Communication 49, 144–158 (2007)CrossRefGoogle Scholar
  10. 10.
    Knox, M.T., Mirghafori, N.: Automatic laughter detection using neural networks. In: Proceedings of Interspeech 2007, Antwerp, Belgium, pp. 2973–2976 (August 2007)Google Scholar
  11. 11.
    Petridis, S., Pantic, M.: Fusion of audio and visual cues for laughter detection. In: Proceedings of the 2008 International Conference on Content-Based Image and Video Retrieval, pp. 329–338. ACM (2008)Google Scholar
  12. 12.
    Petridis, S., Pantic, M.: Audiovisual discrimination between speech and laughter: Why and when visual information might help. IEEE Transactions on Multimedia 13(2), 216–234 (2011)CrossRefGoogle Scholar
  13. 13.
    Ruch, W., Ekman, P.: The expressive pattern of laughter. In: Kaszniak, A. (ed.) Emotion, Qualia and Consciousness, pp. 426–443. World Scientific Publishers, Tokyo (2001)CrossRefGoogle Scholar
  14. 14.
    Filippelli, M., Pellegrino, R., Iandelli, I., Misuri, G., Rodarte, J., Duranti, R., Brusasco, V., Scano, G.: Respiratory dynamics during laughter. Journal of Applied Physiology 90(4), 1441 (2001)Google Scholar
  15. 15.
    Feleky, A.: The influence of the emotions on respiration. Journal of Experimental Psychology 1(3), 218–241 (1916)CrossRefGoogle Scholar
  16. 16.
    Fukushima, S., Hashimoto, Y., Nozawa, T., Kajimoto, H.: Laugh enhancer using laugh track synchronized with the user’s laugh motion. In: Proceedings of CHI 2010, pp. 3613–3618 (2010)Google Scholar
  17. 17.
    Ruch, W.: Exhilaration and humor. Handbook of Emotions 1, 605–616 (1993)Google Scholar
  18. 18.
    Camurri, A., Coletta, P., Varni, G., Ghisio, S.: Developing multimodal interactive systems with eyesweb xmi. In: NIME 2007, pp. 302–305 (2007)Google Scholar
  19. 19.
    Lukas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th IJCAI (1981)Google Scholar
  20. 20.
    Winter, D.: Biomechanics and motor control of human movement (1990)Google Scholar
  21. 21.
    Sethares, W., Staley, T.: Periodicity transforms. IEEE Transactions on Signal Processing 47(11), 2953–2964 (1999)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Mancini, M., Varni, G., Glowinski, D., Volpe, G.: Computing and evaluating the body laughter index. In: Salah, A.A., Ruiz-del-Solar, J., Meriçli, Ç., Oudeyer, P.-Y. (eds.) HBU 2012. LNCS, vol. 7559, pp. 90–98. Springer, Heidelberg (2012)CrossRefGoogle Scholar

Copyright information

© ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering 2013

Authors and Affiliations

  • Jérôme Urbain
    • 1
  • Radoslaw Niewiadomski
    • 2
  • Maurizio Mancini
    • 3
  • Harry Griffin
    • 4
  • Hüseyin Çakmak
    • 1
  • Laurent Ach
    • 5
  • Gualtiero Volpe
    • 3
  1. 1.Université de MonsMonsBelgium
  2. 2.LTCI UMR 5141 - Telecom ParisTechParisFrance
  3. 3.Università degli Studi di GenovaGenovaItaly
  4. 4.UCL Interaction CentreUniversity College LondonLondonUK
  5. 5.LA CANTOCHE PRODUCTIONParisFrance

Personalised recommendations