A Multimodal Analysis of Floor Control in Meetings

  • Lei Chen
  • Mary Harper
  • Amy Franklin
  • Travis R. Rose
  • Irene Kimbara
  • Zhongqiang Huang
  • Francis Quek
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4299)


The participant in a human-to-human communication who controls the floor bears the burden of moving the communication process along. Change in control of the floor can happen through a number of mechanisms, including interruptions, delegation of the floor, and so on. This paper investigates floor control in multiparty meetings that are both audio and video taped; hence, we are able to analyze patterns not only of speech (e.g., discourse markers) but also of visual cues (e.g, eye gaze exchanges) that are commonly involved in floor control changes. Identifying who has control of the floor provides an important focus for information retrieval and summarization of meetings. Additionally, without understanding who has control of the floor, it is impossible to identify important events such as challenges for the floor. In this paper, we analyze multimodal cues related to floor control in two different meetings involving five participants each.


Control Event Conversational Agent Meeting Participant Discourse Marker Floor Control 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Argyle, M., Cook, M.: Gaze and Mutual Gaze. Cambridge Univ. Press, Cambridge (1976)Google Scholar
  2. 2.
    Bosch, L., Oostdijk, N., Ruiter, J.P.: Durational aspects of turn-taking in spontaneous face-to-face and telephone dialogues. In: Sojka, P., Kopeček, I., Pala, K. (eds.) TSD 2004. LNCS, vol. 3206, pp. 563–570. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Burger, S., MacLaren, V., Yu, H.: The ISL meeting corpus: The impact of meeting type on speech type. In: Proc. of Int. Conf. on Spoken Language Processing (ICSLP) (2002)Google Scholar
  4. 4.
    Cassell, J., Sullivan, J., Prevost, S., Churchill, E.: Embodied Conversational Agents. MIT Press, CambridgeGoogle Scholar
  5. 5.
    Kimbara, I., Welji, H., Harper, M., Quek, F., McNeill, D., Duncan, S., Tuttle, R., Huang, T.: VACE multimodal meeting corpus. In: Renals, S., Bengio, S. (eds.) MLMI 2005. LNCS, vol. 3869, pp. 40–51. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Duncan, S.: Some signals and rules for taking speaking turns in conversations. Journal of Personality and Social Psychology 23, 283–292 (1972)CrossRefGoogle Scholar
  7. 7.
    Ford, C.E., Thompson, S.A.: Interactional units in conversation: syntactic, intonational, and pragmatic resources for the management of turns. In: Ochs, T., Schegloff (eds.) Interaction and Grammar. Cambridge Univ. Press, Cambridge (1996)Google Scholar
  8. 8.
    Garofolo, J., Laprum, C., Michel, M., Stanford, V., Tabassi, E.: The NIST Meeting Room Pilot Corpus. In: Proc. of Language Resource and Evaluation Conference (2004)Google Scholar
  9. 9.
    Jovanovic, N., Akker, R.: Towards automatic addressee identification in multi-party dialogues. In: Proceedings of SIGDial (2004)Google Scholar
  10. 10.
    Jovanovic, N., Akker, R., Nijholt, A.: A corpus for studying addressing behavior in multi-party dialogues. In: Proc. of SIGdial Workshop on Discourse and Dialogue (2005)Google Scholar
  11. 11.
    Jurafsky, D., Rebecca, B., et al.: Automatic detection of discourse structure for speech recognition and understanding. In: Proc. of IEEE Workshop on Speech Recognition and Understanding (1997)Google Scholar
  12. 12.
    Kipp, M.: Anvil: A generic annotation tool for multimodal dialogue. In: Proc. of European Conf. on Speech Processing (EuroSpeech) (2001)Google Scholar
  13. 13.
    Liu, Y.: Structural Event Detection for Rich Transcription of Speech. Ph.D thesis, Purdue University (2004)Google Scholar
  14. 14.
    Local, J., Kelly, J.: Projection and ’silences’: Notes on phonetic and conversational structure. Human Studies 9, 185–204 (1986)CrossRefGoogle Scholar
  15. 15.
    McCowan, I., Gatica-Perez, D., Bengio, S., Lathoud, G., Barnard, M., Zhang, D.: Automatic analysis of multimodal group actions in meetings. IEEE Trans. on Pattern Analysis and Machine Intelligence 27(3), 305–317 (2005)CrossRefGoogle Scholar
  16. 16.
    Morgan, N., et al.: Meetings about meetings: Research at ICSI on speech in multiparty conversations. In: Proc. of ICASSP, Hong Kong, vol. 4, pp. 740–743 (2003)Google Scholar
  17. 17.
    Novick, D.G., Hansen, B., Ward, K.: Coordinating turn-taking with gaze. In: Proc. of Int. Conf. on Spoken Language Processing (ICSLP) (1996)Google Scholar
  18. 18.
    Padilha, E., Carletta, J.: A simulation of small group discussion. In: Proc. of the sixth Workshop on the Semantics and Pragmatics of Dialogue (EDILOG 2002), Edinburgh, UK, pp. 117–124 (2002)Google Scholar
  19. 19.
    Padilha, E., Carletta, J.: Nonverbal behaviours improving a simulation of small group discussion. In: Proceedings of the First International Nordic Symposium of Multi-modal Communication (2003)Google Scholar
  20. 20.
    Rose, T., Quek, F., Shi, Y.: MacVisSTA: A system for multimodal analysis. In: Proc. of Int. Conf. on Multimodal Interface (ICMI) (2004)Google Scholar
  21. 21.
    Sacks, H., Schegloff, E., Jefferson, G.: A simplest systematics for the organisation of turn taking for conversation. Language 50, 696–735 (1974)CrossRefGoogle Scholar
  22. 22.
    Shriberg, E., Dhillon, R., Bhagat, S., Ang, J., Carvey, H.: The ICSI meeting recorder dialog act (MRDA) corpus. In: Proc. of SIGdial Workshop on Discourse and Dialogue (2004)Google Scholar
  23. 23.
    Torres, O., Cassell, J., Prevost, S.: Modeling gaze behavior as a function of discourse structure. In: Proc. of the First International Workshop on Human-Computer Conversations, Bellagio, Italy (1997)Google Scholar
  24. 24.
    Weilhammer, K., Rabold, S.: Durational aspects in turn taking. In: International Congresses of Phonetic Sciences (2003)Google Scholar
  25. 25.
    Wichmann, A., Caspers, J.: Melodic cues to turn-taking in English: Evidence from perception. In: Proc. of SIGdial Workshop on Discourse and Dialogue (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Lei Chen
    • 1
  • Mary Harper
    • 1
  • Amy Franklin
    • 2
  • Travis R. Rose
    • 3
  • Irene Kimbara
    • 2
  • Zhongqiang Huang
    • 1
  • Francis Quek
    • 3
  1. 1.School of Electrical EngineeringPurdue UniversityWest Lafayette
  2. 2.Department of PsychologyUniversity of ChicagoChicago
  3. 3.CHCI, Department of Computer ScienceVirginia TechBlacksburg

Personalised recommendations