Language Resources and Evaluation

, Volume 41, Issue 3–4, pp 409–429

A multimodal annotated corpus of consensus decision making meetings

  • Fabio Pianesi
  • Massimo Zancanaro
  • Bruno Lepri
  • Alessandro Cappelletti


In this paper we present an annotated audio–video corpus of multi-party meetings. The multimodal corpus provides for each subject involved in the experimental sessions six annotation dimensions referring to group dynamics; speech activity and body activity. The corpus is based on 11 audio and video recorded sessions which took place in a lab setting appropriately equipped with cameras and microphones. Our main concern in collecting this multimodal corpus was to explore the possibility of providing feedback services to facilitate group processes and to enhance self awareness among small groups engaged in meetings. We therefore introduce a coding scheme for annotating relevant functional roles that appear in a small group interaction. We also discuss the reliability of the coding scheme and we present the first results for automatic classification.


Behaviour analysis Small groups Meetings Multimodality 


  1. Bales, R. F. (1970). Personality and interpersonal behavior. New York: Holt, Rinehart and Winston.Google Scholar
  2. Benne, K. D., & Sheats, P. (1948). Functional roles of group members. Journal of Social Issues, 4, 41–49.CrossRefGoogle Scholar
  3. Brdiczka, O., Maisonnasse, J., & Reignier, P. (2005). Automatic detection of interaction groups. In Proc. of the 7th International Conference on Multimodal Interface, Trento, Italy.Google Scholar
  4. Carli, G., & Gretter, G. (1992). A start-end point detection algorithm for a real-time acoustic front-end based on DSP32C VME Board. In Proc. ICSPAT.Google Scholar
  5. Chen, L., Rose, R. T., Parrill, F., Han, X., Tu, J., Huang, Z., Harper, M., Quek, F., McNeill, D., Tuttle, R., & Huang, T. (2005). VACE multimodal meeting corpus. In Proc. of Multimodal Interaction and Related Machine Learning Algorithms.Google Scholar
  6. Chippendale, P. (2006). Towards automatic body language annotation. In Talk delivered at the International Conference on Automatic Face and Gesture Recognition – FG2006 (IEEE), Southampton, UK.Google Scholar
  7. Cristianini, N., & Shawe-Taylor, J. (2000). Support vector machines and other kernel-based learning methods. Cambridge: Cambridge University Press.Google Scholar
  8. Dietterich, T. G. (2002). Machine learning for sequential data: A review. In T. Caelli (Ed.), Lectures Notes in Computer Science. Springer-Verlag.Google Scholar
  9. Falcon, V., Leonardi, C., Pianesi, F., & Zancanaro, M. (2005). Annotation of group behaviour: A proposal for a coding scheme. In Proc. of Workshop on Multimodal Multiparty Multimodal Processing at ICMI 2005, pp. 39–46.Google Scholar
  10. Hall, J. W., & Watson, W. H. (1970). The effects of a normative intervention on group decision-making performance. Human Relations, 23(4), 299–317.CrossRefGoogle Scholar
  11. Hare, P. (2003). Roles, relationships, and groups in organizations: Some conclusions and recommendations. Small Group Research, 34(2), 123–154.CrossRefGoogle Scholar
  12. Hsu, C.-W., & Lin, C.-J. (2002). A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks, 13, 415–425.CrossRefGoogle Scholar
  13. Kaiser, E., Demirdjian, D., Gruenstein, A., Li, X., Niekrasz, J., Wesson, M., & Kumar, S. (2004). A multimodal learning interface for sketch, speak and point creation of a schedule chart. In Proceedings of ICMI 2004, pp. 329–330.Google Scholar
  14. Katz, D., & Kahn, R. L. (1978). The social psychology of organizations (2nd ed.). New York: Wiley.Google Scholar
  15. Kressel, U. (1999). Pairwise classification and support vector machines. In B. Scholkopf, C. J. C. Burges, & A. J. Smola (Eds.), Advances in Kernel methods – Support vector learning. Cambridge, MA: MIT Press.Google Scholar
  16. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.CrossRefGoogle Scholar
  17. McCowan, I., Gatica-Perez, D., Bengio, S., Moore, D., & Bourlard, H. (2004). Towards computer understanding of human interactions. In E. Aarts, R. Collier, E. van Loenen, & B. de Ruyter (Eds.), Ambient intelligence. Lecture Notes in Computer Science (pp. 235–251). Heidelberg: Springer-Verlag.Google Scholar
  18. Otzuka, K., Yamato, J., Takemae, Y., & Murase, H. (2005). A probabilistic inference of multiparty-conversation structure based on Markov-switching models of gaze patterns, head directions, and utterances. In Proceedings of IMCI’05, Trento, Italy.Google Scholar
  19. Pianesi, F., Zancanaro, M., Falcon, V., & Not, E. (2006). Toward supporting group dynamics. In Proceedings of AIAI’06, Athens.Google Scholar
  20. Rienks, R., Zhang, D., Gatica-Perez, D., & Post, W. (2006). Detection and application of influence rankings in small group meetings. In Proceedings of ICMI’06, Banff, CA.Google Scholar
  21. Salazar, A. (1996). An analysis of the development and evolution of roles in the small group. Small Group Research, 27, 475–503.CrossRefGoogle Scholar
  22. Stiefelhagen, R., Zhan, J., & Waibel, A. (2002). Modeling focus of attention for meeting indexing. In CHI ‘02 extended abstracts on Human factors in computing systems.Google Scholar
  23. Waibel, A., Steusloff, H., & Stiefelhagen, R. (2004). CHIL: Computer in the human interaction loop. In NIST ICASSP Meeting Recognition Workshop, Montreal, Canada.Google Scholar
  24. Zancanaro, M., Lepri, B., & Pianesi, F. (2006). Automatic detection of group functional roles in face to face interactions. In Proceedings of International Conference on Multimodal Interaction, Banff.Google Scholar
  25. Zhang, D., Gatica-Perez, D., Bengio, S., & Roy, D. (2006). The team-player influence model. Pattern Analysis and Machine Intelligence.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  • Fabio Pianesi
    • 1
  • Massimo Zancanaro
    • 1
  • Bruno Lepri
    • 1
  • Alessandro Cappelletti
    • 1
  1. 1.Fondazione Bruno Kessler FBK-irstTrentoItaly

Personalised recommendations