Abstract
This chapter presents computational methods for the analysis of social interaction. We focus on nonverbal behavior of social interaction, in particular social verticality, such as dominance, leadership, and roles. We describe processing, feature extraction, and inference methods that are widely used in the computational social interaction analysis literature. In the last section of the chapter, we present four case studies on dominance estimation, identifying emergent leadership, role recognition, and analysis of leadership styles.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ambady, N., Rosenthal, R.: Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. Psychol. Bull. 111, 256–274 (1992)
Aran, O., Gatica-Perez, D.: Fusing audio-visual nonverbal cues to detect dominant people in small group conversations. In: 20th International Conference on Pattern Recognition (ICPR), Istanbul, Turkey (2010)
Aran, O., Hung, H., Gatica-Perez, D.: A multimodal corpus for studying dominance in small group conversations. In: LREC Workshop on Multimodal Corpora, Malta (LREC MMC’10) (2010)
Asavathiratham, C., Roy, S., Lesieutre, B., Verghese, G.: The influence model. IEEE Control Syst. Mag. 21, 52–64 (2001)
Ba, S.O., Odobez, J.-M.: Multiperson visual focus of attention from head pose and meeting contextual cues. IEEE Trans. Pattern Anal. Mach. Intell. 33, 101–116 (2011)
Barzilay, R., Collins, M., Hirschberg, J., Whittaker, S.: The rules behind roles: Identifying speaker role in radio broadcasts. In: 17th National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, pp. 679–684. AAAI, Washington (2000)
Basu, S., Choudhury, T., Clarkson, B., Pentland, A.: Learning human interactions with the influence model. Technical report, MIT Media Lab, Cambridge, MA (2001)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Bobick, A.F., Davis, J.W.: The recognition of human movement using temporal templates. IEEE Trans. Pattern Anal. Mach. Intell. 23, 257–267 (2001)
Campbell, N., Douxchamps, D.: Processing image and audio information for recognising discourse participation status through features of face and voice. In: INTERSPEECH 2007, pp. 730–733 (2007)
Carletta, J., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Karaiskos, V., Kraaij, W., Kronenthal, M., Lathoud, G., Lincoln, M., Lisowska, A., McCowan, I., Post, W., Reidsma, D., Wellner, P.: The AMI meeting corpus: A pre-announcement. In: Workshop Mach. Learn. for Multimodal Interaction (MLMI’05), Edinburgh, UK, pp. 28–39 (2005)
Charfuelan, M., Schröder, M., Steiner, I.: Prosody and voice quality of vocal social signals: the case of dominance in scenario meetings. In: Interspeech 2010, Makuhari, Japan, Sept. (2010)
Chippendale, P.: Towards automatic body language annotation. In: 7th International Conference on Automatic Face and Gesture Recognition (FG ’06), Washington, DC (2006)
Chittaranjan, G., Aran, O., Gatica-Perez, D.: Exploiting observers’ judgments for multimodal nonverbal group interaction analysis. In: 9th IEEE Conference on Automatic Face and Gesture Recognition, Santa Barbara, CA (2011)
Dong, W., Lepri, B., Cappelletti, A., Pentland, A.S., Pianesi, F., Zancanaro, M.: Using the influence model to recognize functional roles in meetings. In: 9th International Conference on Multimodal Interfaces (ICMI’07), pp. 271–278 (2007)
Dovidio, J.F., Ellyson, S.L.: Decoding visual dominance: Attributions of power based on relative percentages of looking while speaking and looking while listening. Soc. Psychol. Q. 45(2), 106–113 (1982)
Dunbar, N.E., Burgoon, J.K.: Perceptions of power and interactional dominance in interpersonal relationships. J. Soc. Pers. Relatsh. 22(2), 207–233 (2005)
Funder, D.C.: Personality. Annu. Rev. Psychol. 52, 197–221 (2001)
Garg, N.P., Favre, S., Salamin, H., Hakkani Tür, D., Vinciarelli, A.: Role recognition for meeting participants: an approach based on lexical information and social network analysis. In: 16th ACM International Conference on Multimedia (MM’08), pp. 693–696 (2008)
Gatica-Perez, D.: Automatic nonverbal analysis of social interaction in small groups: a review. Image Vis. Comput. 27(12), 1775–1787 (2009)
Gorga, S., Otsuka, K.: Conversation scene analysis based on dynamic bayesian network and image-based gaze detection. In: ICMI-MLMI 2010 (2010)
Hall, J.A., Coats, E.J., Smith LeBeau, L.: Nonverbal behavior and the vertical dimension of social relations: A meta-analysis. Psychol. Bull. 1313(6), 898–924 (2005)
Hung, H., Jayagopi, D.B., Ba, S., Gatica-Perez, D., Odobez, J.-M.: Investigating automatic dominance estimation in groups from visual attention and speaking activity. In: Int. Conf. on Multimodal Interfaces (ICMI), Chania, Greece (2008)
Hung, H., Huang, Y., Friedland, G., Gatica-Perez, D.: Estimating dominance in multi-party meetings using speaker diarization. IEEE Trans. Audio Speech Lang. Process. 19(4), 847–860 (2011)
Jayagopi, D.B., Gatica-Perez, D.: Mining group nonverbal conversational patterns using probabilistic topic models. IEEE Trans. Multimed. (2010)
Jayagopi, D.B., Ba, S., Odobez, J.-M., Gatica-Perez, D.: Predicting two facets of social verticality in meetings from five-minute time slices and nonverbal cues. In: Int. Conf. on Multimodal Interfaces (ICMI), Special Session on Social Signal Processing, Chania, Greece (2008)
Jayagopi, D.B., Hung, H., Yeo, C., Gatica-Perez, D.: Modeling dominance in group conversations from nonverbal activity cues. IEEE Trans. Audio Speech Lang. Process. 17(3), 501–513 (2009). Special Issue on Multimodal Processing for Speech-based Interactions
Kalma, A.K., Visser, L., Peeters, A.: Sociable and aggressive dominance: Personality differences in leadership style? Leadersh. Q. 4(1), 45–64 (1993)
Kickul, J., Neuman, G.: Emergent leadership behaviours: The function of personality and cognitive ability in determining teamwork performance and KSAs. J. Bus. Psychol. 15(1) (2000)
Kim, T., Chang, A., Pentland, A.: Meeting mediator: Enhancing group collaboration with sociometric feedback. In: ACM Conference on Computer Supported Collaborative Work, San Diego, CA, pp. 457–466 (2008)
Knapp, M.L., Hall, J.A.: Nonverbal Communication in Human Interaction, 7th edn. Wadsworth, Belmont (2009)
Kumano, S., Otsuka, K., Mikami, D., Yamato, J.: Recognizing communicative facial expressions for discovering interpersonal emotions in group meetings. In: 11th International Conference on Multimodal interfaces (ICMI’09), ICMI-MLMI ’09, pp. 99–106 (2009)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, New York (2004)
Lane, N.D., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., Campbell, A.T.: A survey of mobile phone sensing. IEEE Commun. Mag. 48, 140–150 (2010)
Lewin, K., Lippit, R., White, R.K.: Patterns of aggressive behavior in experimentally created social climates. J. Soc. Psychol. 10, 271–301 (1939)
Otsuka, K., Yamato, J., Takemae, Y., Murase, H.: Conversation scene analysis with dynamic Bayesian network based on visual head tracking. In: ICME 2006 (2006)
Otsuka, K., Sawada, H., Yamato, J.: Automatic inference of cross-modal nonverbal interactions in multiparty conversations. In: ACM 9th Int. Conf. Multimodal Interfaces (ICMI2007), pp. 255–262 (2007)
Pentland, A.: Socially aware computation and communication. Computer 38(3), 33–40 (2005)
Pentland, A.: Honest Signals: How They Shape Our World. MIT Press, Cambridge (2008)
Pianesi, F., Zancanaro, M., Lepri, B., Cappelletti, A.: A multimodal annotated corpus of consensus decision making meetings. Lang. Resour. Eval. 41, 409–429 (2007)
Raducanu, B., Gatica-Perez, D.: Inferring competitive role patterns in reality TV show through nonverbal analysis. Multimed. Tools Appl. 1–20 (2010)
Rienks, R.J., Heylen, D.: Automatic dominance detection in meetings using easily detectable features. In: Workshop Mach. Learn. for Multimodal Interaction (MLMI’05), Edinburgh, UK (2005)
Salamin, H., Favre, S., Vinciarelli, A.: Automatic role recognition in multiparty recordings: Using social affiliation networks for feature extraction. IEEE Trans. Multimed. 11(7), 1373–1380 (2009)
Sanchez-Cortes, D., Aran, O., Schmid-Mast, M., Gatica-Perez, D.: Identifying emergent leadership in small groups using nonverbal communicative cues. In: 12th International Conference on Multimodal Interfaces (ICMI’10), Beijing, China (2010)
Schmid-Mast, M.: Dominance as expressed and inferred through speaking time: A meta-analysis. Hum. Commun. Res. 28(3), 420–450 (2002)
Stein, R.T.: Identifying emergent leaders from verbal and nonverbal communications. J. Pers. Soc. Psychol. 32(1), 125–135 (1975)
Steyvers, M., Griffiths, T.: Probabilistic Topic Models. Erlbaum, Hillsdale (2007)
Varni, G., Volpe, G., Camurri, A.: A system for real-time multimodal analysis of nonverbal affective social interaction in user-centric media. IEEE Trans. Multimed. 12(6), 576–590 (2010)
Yeo, C., Ahammad, P., Ramchandran, K., Sastry, S.S.: High-speed action recognition and localization in compressed domain videos. IEEE Trans. Circuits Syst. Video Technol. 18(8), 1006–1015 (2008)
Zancanaro, M., Lepri, B., Pianesi, F.: Automatic detection of group functional roles in face to face interactions. In: 8th International Conference on Multimodal Interfaces (ICMI’06), pp. 28–34 (2006)
Acknowledgements
This work is supported by the EU FP7 Marie Curie Intra-European Fellowship project “Automatic Analysis of Group Conversations via Visual Cues in nonverbal Communication” (NOVICOM), and by the Swiss National Science Foundation under the National Centre of Competence in Research (NCCR) on “Interactive Multimodal Information Management” (IM2) and by the Sinergia project on “Sensing and Analysing Organizational Nonverbal Behaviour” (SONVB). The authors would like to thank Dinesh Jayagopi, Dairazalia Sanchez-Cortes, and Gokul Chittaranjan for their contributions to several studies presented in this chapter.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag London Limited
About this chapter
Cite this chapter
Aran, O., Gatica-Perez, D. (2011). Analysis of Group Conversations: Modeling Social Verticality. In: Salah, A., Gevers, T. (eds) Computer Analysis of Human Behavior. Springer, London. https://doi.org/10.1007/978-0-85729-994-9_11
Download citation
DOI: https://doi.org/10.1007/978-0-85729-994-9_11
Publisher Name: Springer, London
Print ISBN: 978-0-85729-993-2
Online ISBN: 978-0-85729-994-9
eBook Packages: Computer ScienceComputer Science (R0)