Meeting Modelling in the Context of Multimodal Research

  • Dennis Reidsma
  • Rutger Rienks
  • Natas̃a Jovanović
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3361)


This paper presents a framework for corpus based multimodal research. Part of this framework is applied in the context of meeting modelling. A generic model for different aspects of meetings is discussed. This model leads to a layered description of meetings where each layer adds a level of interpretation for distinct aspects based on information provided by lower layers. This model should provide a starting point for selecting annotation schemes for layers of the meeting and for defining a hierarchy between individual layers.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Armstrong, S., Clark, A., et al.: Natural language queries on natural language data: a database of meeting dialogues. In: Proc. of NLP and Information Systems-NLDB 2003, pp. 14–27 (2003)Google Scholar
  2. 2.
    Vertegaal, R.: Who is looking at whom. PhD thesis, University of Twente (1998)Google Scholar
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
    Hersey, P., Blanchard, K.: Management of Organizational Behavior: Utilizing Human Resources. Prentice Hall, Englewood Cliffs (1988)Google Scholar
  8. 8.
    Kunz, W., Rittel, H.W.J.: Issues as elements of information systems. Working Paper WP-131, Univ. Stuttgart, Inst. Fuer Grundlagen der Planung (1970)Google Scholar
  9. 9.
    Pelachaud, C., Bilvi, M.: Computational model of believable conversational agents. Communication in MAS: background, current trends and future (2003)Google Scholar
  10. 10.
    Padilha, E., Carletta, J.: Nonverbal behaviours improving a simulation of small group discussion. In: Proc. 1st Nordic Symp. on Multimodal Comm., pp. 93–105 (2003)Google Scholar
  11. 11.
    Nijholt, A.: Meetings, gatherings and events in smart environments. In: Proc. ACM SIGGRAPH VRCAI 2004 (2004) (to appear)Google Scholar
  12. 12.
    Nijholt, A.: Where computers disappear, virtual humans appear. Computers and Graphics 28 (2004) (to appear)Google Scholar
  13. 13.
    Vilhjalmsson, H., Cassell, J.: Bodychat: Autonomous communicative behaviors in avatars. In: Proc. of the 2nd Annual ACM Int. Conf. on Autonomous Agents (1998)Google Scholar
  14. 14.
    Carletta, J., Anderson, A., Garrod, S.: Seeing eye to eye: an account of grounding and understanding in work groups. Cognitive Studies: Bulletin of the Japanese Cognitive Science Society 9(1), 1–20 (2002)Google Scholar
  15. 15.
    McGrath, J.: Groups: Interaction and Performance. Prentice Hall, Englewood Cliffs (1984)Google Scholar
  16. 16.
    Shi, Y., Rose, R.T., Quek, F., McNeill, D.: Gaze, attention, and meeting dynamics. In: ICASSP2004 Meeting Recognition Workshop (2004) (to appear)Google Scholar
  17. 17.
    Marchand-Maillet, S.: Meeting record modelling for enhanced browsing. Technical Report 03.01, Computer Vision and Multimedia Laboratory, University of Geneva (2003)Google Scholar
  18. 18.
    Clark, H.: Using language. Cambridge University Press, Cambridge (1996)CrossRefGoogle Scholar
  19. 19.
    McCowan, I., Bengio, S., Gatica-Perez, D., Lathoud, G.: Automatic analysis of multimodal group actions in meetings. Technical Report IDIAP-RR 03-27, IDIAP (2003)Google Scholar
  20. 20.
    Jovanovic, N.: Recognition of meeting actions using information obtained from different modalities. Report TR-CTIT-03-48, CTIT (2003)Google Scholar
  21. 21.
    Dhillon, R., Bhagat, S., Carvey, H., Shriberg, E.: Meeting recorder project: Dialogue act labeling guide. Technical report, ICSI Speech Group, Berkeley, USA (2003)Google Scholar
  22. 22.
    Jurafsky, D., Shriberg, E., Biaska, D.: Switchboard SWBD-DAMSL shallow-discourse-function annotation (coders manual, draft 13). Technical report, Univ. of Colorado, Inst. of Cognitive Science (1997)Google Scholar
  23. 23.
    Bales, R.F.: Interaction Process Analysis. Addison-Wesley, Reading (1951)Google Scholar
  24. 24.
    Pallotta, V., Hatem, G.: Argumentative segmentation and annotation guidelines. Technical report (2003)Google Scholar
  25. 25.
    Gilbert, M.A.: Goals in argumentation. In: Proc. of the conf. on Formal and Applied Practical Reasoning (1996)Google Scholar
  26. 26.
    Carofiglio, V., de Rosis, F.: Combining logical with emotional reasoning in natural argumentation. In: 3rd Workshop on Affective and Attitude User Modeling (2003)Google Scholar
  27. 27.
    Zhang, D., Daniel Gatica-Perez, S.B.I.M., Lathoud, G.: Multimodal Group Action Clustering in Meetings. In: ACM 2nd International Workshop on Video Surveillance & Sensor Networks in conjunction with 12th ACM International Conference on Multimedia (2004), IDIAP-RR 04-24Google Scholar
  28. 28.
    Shriberg, E., Dhillon, R., Bhagat, S., et al.: The icsi meeting recorder dialog act (mrda) corpus. In: Proc. HLT-NAACL SIGDIAL Workshop (2004) (to appear)Google Scholar
  29. 29.
    Syrdal, A.K., Hirschberg, J., McGory, J., Beckman, M.: Automatic tobi prediction and alignment to speed manual labelling of prosody. Speech communication 33, 135–151 (2001)MATHCrossRefGoogle Scholar
  30. 30.
    Brants, T., Skut, W., Uszkoreit, H.: Syntactic annotation of a German newspaper corpus. Kluwer, Dordrecht (2003)Google Scholar
  31. 31.
    Lin, C.Y., Tseng, B.L., Smith, J.R.: Video collaborative annotation forum: Establishing ground-truth labels on large multimedia datasets. In: Proc. of the TRECVID 2003 (2003)Google Scholar
  32. 32.
    McNeill, D.: Hand and Mind: What gestures reveal about thought. University of Chicago Press (1995)Google Scholar
  33. 33.
    Brethes, L., Menezes, P., Lerasle, F., Hayet, J.: Face tracking and hand gesture recognition for human-robot interaction. In: Proc. of the International Conference on Robotics and Automation (2004)Google Scholar
  34. 34.
  35. 35.
    Carletta, J., Evert, S., Heid, U., Kilgour, J., Robertson, J., Voormann, H.: The NITE XML toolkit: flexible annotation for multi-modal language data. Behavior Research Methods, Instruments, and Computers 35(3), 353–363 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Dennis Reidsma
    • 1
  • Rutger Rienks
    • 1
  • Natas̃a Jovanović
    • 1
  1. 1.Dept. of Computer Science, HMI GroupUniversity of TwenteEnschedethe Netherlands

Personalised recommendations