Journal of Nonverbal Behavior

, Volume 38, Issue 3, pp 389–408 | Cite as

Automatically Detected Nonverbal Behavior Predicts Creativity in Collaborating Dyads

  • Andrea Stevenson WonEmail author
  • Jeremy N. Bailenson
  • Suzanne C. Stathatos
  • Wenqing Dai
Original Paper


In the current study we administered a creative task in which two people collaboratively generated novel strategies to conserve resources. During this task, the nonverbal behavior of 104 participants in 52 pairs was tracked and recorded using the Kinect computer vision algorithm. We created a measure of synchrony by correlating movements between the two dyad members, and showed that synchrony occurred—that is, correlations decreased when we increased delay between the recorded movements of pair members. We also demonstrated a link between nonverbal synchrony and creativity, as operationalized by the number of new, valid ideas produced. Linear correlations demonstrated a significant relationship between synchrony and creativity. Finally, models using synchrony scores as input predicted whether dyads were high or low in creativity with a success rate as high as 86.7 % in the more exclusive subsets. We discuss implications for methodological approaches to measuring nonverbal behavior and synchrony, and suggest practical applications which can leverage the current findings.


Nonverbal behavior Synchrony Gesture Collaboration Creativity Kinect Interpersonal communication Contingency 



The work presented herein was funded in part by Konica Minolta as part of a Stanford Media-X grant, and we thank them for the valuable insights provided by their visiting researchers, in particular Dr. Haisong Gu. In addition, it was funded in part by grant 108084-5031715-4 from the National Science Foundation. The authors also thank lab manager Cody Karutz for coordinating the administrative aspects of running this study, and Jimmy Lee, Pamela Martinez, Evan Shieh, Alex Zamoshchin, Angel Olvera, Christine Tataru, Mark Diaz and Mark Peng for their help in coding and data analysis, and Dr. Laura Aymerich-Franch, Ketaki Shriram, Michelle Friend, Brian Perone, and Jakki Bailey for helpful comments on an earlier draft of this paper.


  1. Baas, M., De Dreu, C. K., & Nijstad, B. A. (2008). A meta-analysis of 25 years of mood-creativity research: Hedonic tone, activation, or regulatory focus? Psychological Bulletin, 134(6), 779.PubMedCrossRefGoogle Scholar
  2. Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359.CrossRefGoogle Scholar
  3. Bernieri, F. J. (1988). Coordinated movement and rapport in teacher student interactions. Journal of Nonverbal Behavior, 12, 120–138.CrossRefGoogle Scholar
  4. Bernieri, F. J., Davis, J. M., Rosenthal, R., & Knee, C. R. (1994). Interactional synchrony and rapport: Measuring synchrony in displays devoid of sound and facial affect. Personality and Social Psychology Bulletin, 20(3), 303–311.CrossRefGoogle Scholar
  5. Bernieri, F. J., Reznick, J. S., & Rosenthal, R. (1988). Synchrony, pseudosynchrony, and dissynchrony: Measuring the entrainment process in mother-infant interactions. Journal of Personality and Social Psychology, 54(2), 243.CrossRefGoogle Scholar
  6. Castellano, G., Villalba, S. D., & Camurri, A. (2007). Recognising human emotions from body movement and gesture dynamics. Affective Computing and Intelligent Interaction, 4738, 71–82.CrossRefGoogle Scholar
  7. Condon, W. S., & Ogston, W. D. (1966). Sound film analysis of normal and pathological behavior patterns. Journal of Nervous Mental Disorders, 143, 338–347.CrossRefGoogle Scholar
  8. Delaherche, E., Chetouani, M., Mahdhaoui, A., Saint-Georges, C., Viaux, S., & Cohen, D. (2012). Interpersonal synchrony: A survey of evaluation methods across disciplines. Affective Computing, IEEE Transactions on, 3(3), 349–365.Google Scholar
  9. Drolet, A. L., & Morris, M. W. (2000). Rapport in conflict resolution: Accounting for how face-to-face contact fosters mutual cooperation in mixed-motive conflicts. Journal of Experimental Social Psychology, 36(1), 26–50.CrossRefGoogle Scholar
  10. Goncalo, J. A., & Staw, B. M. (2006). Individualism–collectivism and group creativity. Organizational Behavior and Human Decision Processes, 100(1), 96–109.CrossRefGoogle Scholar
  11. Grahe, J. E., & Bernieri, F. J. (1999). The importance of nonverbal cues in judging rapport. Journal of Nonverbal Behavior, 23(4), 253–269.CrossRefGoogle Scholar
  12. Guilford, J. P. (1957). Creative abilities in the arts. Psychological Review, 64(2), 110–118.PubMedCrossRefGoogle Scholar
  13. Harrigan, J. A., Oxman, T. E., & Rosenthal, R. (1985). Rapport expressed through nonverbal behavior. Journal of Nonverbal Behavior, 9(2), 95–110.CrossRefGoogle Scholar
  14. Hoque, M. E., McDuff, D. J., & Picard, R. W. (2012). Exploring temporal patterns in classifying frustrated and delighted smiles. Journal of IEEE Transactions on Affective Computing, 99, 1–13.Google Scholar
  15. Huang, L., Morency, L. P., & Gratch, J. (2011). Virtual rapport 2.0. In Intelligent virtual agents (pp. 68–79). Berlin: Springer.Google Scholar
  16. Jabon, M. E., Ahn, S. J., & Bailenson, J. N. (2011a). Automatically analyzing facial-feature movements to identify human errors. IEEE Journal of Intelligent Systems, 26(2), 54–63.Google Scholar
  17. Jabon, M. E., Bailenson, J. N., Pontikakis, E. D., Takayama, L., & Nass, C. (2011b). Facial expression analysis for predicting unsafe driving behavior. IEEE Pervasive Computing, 10(4), 84–95.CrossRefGoogle Scholar
  18. Kapur, A., Kapur, A., Virji-Babul, N., Tzanetakis, G., & Driessen, P. F. (2005). Gesture-based affective computing on motion capture data. In Affective Computing and Intelligent Interaction (pp. 1–7). Berlin, Heidelberg: Springer.Google Scholar
  19. Kendon, A. (1970). Movement coordination in social interaction: Some examples described. Acta Psychologica, 32, 100–125.PubMedCrossRefGoogle Scholar
  20. Kleinsmith, A., & Bianchi-Berthouze, N. (2007). Recognizing affective dimensions from body posture. In Affective computing and intelligent interaction (pp. 48–58). Berlin: Springer.Google Scholar
  21. Kurtzberg, T. R., & Amabile, T. M. (2001). From Guilford to creative synergy: Opening the black box of team-level creativity. Creativity Research Journal, 13(3–4), 285–294.CrossRefGoogle Scholar
  22. La France, M., & Broadbent, M. (1976). Group rapport: Posture sharing as a nonverbal indicator. Group and Organization Studies, 1, 328–333.CrossRefGoogle Scholar
  23. Lumsden, J., Miles, L. K., & Macrae, C. N. (2012). Perceptions of synchrony: Different strokes for different folks? Perception, 41(12), 1529.PubMedCrossRefGoogle Scholar
  24. Martin, C. C., Burkert, D. C., Choi, K. R., Wieczorek, N. B., McGregor, P. M., Herrmann, R. A., et al. (2012, April). A real-time ergonomic monitoring system using the Microsoft Kinect. In IEEE Systems and Information Design Symposium (SIEDS) (pp. 50–55).Google Scholar
  25. McLeod, P. L., Lobel, S. A., & Cox, T. H. (1996). Ethnic diversity and creativity in small groups. Small Group Research, 27(2), 248–264.CrossRefGoogle Scholar
  26. Meservy, T. O., Jensen, M. L., Kruse, J., Burgoon, J. K., & Jay, F. (2005). Detecting deception through automatic, unobtrusive analysis of nonverbal behavior. IEEE Intelligent Systems, 20(5), 36–43.Google Scholar
  27. Oppezzo, M., & Schwartz, D. L. (2014). Give your ideas some legs: The positive effect of walking on creative thinking. Journal of Experimental Psychology: Learning, Memory, and Cognition. doi: 10.1037/a0036577.
  28. Paxton, A., & Dale, R. (2013). Frame-differencing methods for measuring bodily synchrony in conversation. Behavior Research Methods, 45(2), 329–343.PubMedCrossRefGoogle Scholar
  29. Pentland, A. S. (2010). Honest signals. Cambridge: MIT press.Google Scholar
  30. Ramseyer, F., & Tschacher, W. (2011). Nonverbal synchrony in psychotherapy: Coordinated body-movement reflects relationship quality and outcome. Journal of Consulting and Clinical Psychology, 79(3), 284–295.PubMedCrossRefGoogle Scholar
  31. Schmidt, R. C., Morr, S., Fitzpatrick, P., & Richardson, M. J. (2012). Measuring the dynamics of interactional synchrony. Journal of Nonverbal Behavior, 36(4), 263–279.Google Scholar
  32. Sung, J., Ponce, C., Selman, B., & Saxena, A. (2011). Human activity detection from RGBD images. AAAI 2011 workshop: Plan, activity, and intent recognition.Google Scholar
  33. Tickle-Degnen, L., & Rosenthal, R. (1990). The nature of rapport and its nonverbal correlates. Psychological Inquiry, 1(4), 285–293.CrossRefGoogle Scholar
  34. Torrance, E. P. (1970). Influence of dyadic interaction on creative functioning. Psychological Reports, 26(2), 391–394.PubMedCrossRefGoogle Scholar
  35. Vinciarelli, A., Pantic, M., Bourlard, H., & Pentland, A. (2008). Social signal processing: State-of-the-art and future perspectives of an emerging domain. In Proceedings of the 16th ACM international conference on multimedia (pp. 1061–1070). New York: ACM.Google Scholar
  36. Witten, I., Eibe, F., & Hall, M. A. (2011). Data mining: Practical machine learning tools and techniques. Burlington MA: Morgan Kaufman.Google Scholar
  37. Won, A. S., Bailenson, J. N., & Janssen, J. H. (2014). Automatic detection of nonverbal behavior predicts learning in dyadic interactions. Manuscript submitted for publication.Google Scholar
  38. Microsoft Corp. Redmond WA. Kinect for Xbox 360.Google Scholar
  39. Zebrowitz, L. A., & Montepare, J. M. (2006). The ecological approach to person perception: Evolutionary roots and contemporary offshoots. In M. Schaller, J. A. Simpson, & D. T. Kenrick (Eds.) Evolution and social psychology. First Edition (pp. 81–113), New York: Psychology Press.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Andrea Stevenson Won
    • 1
    Email author
  • Jeremy N. Bailenson
    • 1
  • Suzanne C. Stathatos
    • 1
  • Wenqing Dai
    • 1
  1. 1.Department of CommunicationStanford UniversityStanfordUSA

Personalised recommendations