Automatically Detected Nonverbal Behavior Predicts Creativity in Collaborating Dyads

Abstract

In the current study we administered a creative task in which two people collaboratively generated novel strategies to conserve resources. During this task, the nonverbal behavior of 104 participants in 52 pairs was tracked and recorded using the Kinect computer vision algorithm. We created a measure of synchrony by correlating movements between the two dyad members, and showed that synchrony occurred—that is, correlations decreased when we increased delay between the recorded movements of pair members. We also demonstrated a link between nonverbal synchrony and creativity, as operationalized by the number of new, valid ideas produced. Linear correlations demonstrated a significant relationship between synchrony and creativity. Finally, models using synchrony scores as input predicted whether dyads were high or low in creativity with a success rate as high as 86.7 % in the more exclusive subsets. We discuss implications for methodological approaches to measuring nonverbal behavior and synchrony, and suggest practical applications which can leverage the current findings.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Baas, M., De Dreu, C. K., & Nijstad, B. A. (2008). A meta-analysis of 25 years of mood-creativity research: Hedonic tone, activation, or regulatory focus? Psychological Bulletin, 134(6), 779.

    PubMed  Article  Google Scholar 

  2. Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359.

    Article  Google Scholar 

  3. Bernieri, F. J. (1988). Coordinated movement and rapport in teacher student interactions. Journal of Nonverbal Behavior, 12, 120–138.

    Article  Google Scholar 

  4. Bernieri, F. J., Davis, J. M., Rosenthal, R., & Knee, C. R. (1994). Interactional synchrony and rapport: Measuring synchrony in displays devoid of sound and facial affect. Personality and Social Psychology Bulletin, 20(3), 303–311.

    Article  Google Scholar 

  5. Bernieri, F. J., Reznick, J. S., & Rosenthal, R. (1988). Synchrony, pseudosynchrony, and dissynchrony: Measuring the entrainment process in mother-infant interactions. Journal of Personality and Social Psychology, 54(2), 243.

    Article  Google Scholar 

  6. Castellano, G., Villalba, S. D., & Camurri, A. (2007). Recognising human emotions from body movement and gesture dynamics. Affective Computing and Intelligent Interaction, 4738, 71–82.

    Article  Google Scholar 

  7. Condon, W. S., & Ogston, W. D. (1966). Sound film analysis of normal and pathological behavior patterns. Journal of Nervous Mental Disorders, 143, 338–347.

    Article  Google Scholar 

  8. Delaherche, E., Chetouani, M., Mahdhaoui, A., Saint-Georges, C., Viaux, S., & Cohen, D. (2012). Interpersonal synchrony: A survey of evaluation methods across disciplines. Affective Computing, IEEE Transactions on, 3(3), 349–365.

  9. Drolet, A. L., & Morris, M. W. (2000). Rapport in conflict resolution: Accounting for how face-to-face contact fosters mutual cooperation in mixed-motive conflicts. Journal of Experimental Social Psychology, 36(1), 26–50.

    Article  Google Scholar 

  10. Goncalo, J. A., & Staw, B. M. (2006). Individualism–collectivism and group creativity. Organizational Behavior and Human Decision Processes, 100(1), 96–109.

    Article  Google Scholar 

  11. Grahe, J. E., & Bernieri, F. J. (1999). The importance of nonverbal cues in judging rapport. Journal of Nonverbal Behavior, 23(4), 253–269.

    Article  Google Scholar 

  12. Guilford, J. P. (1957). Creative abilities in the arts. Psychological Review, 64(2), 110–118.

    PubMed  Article  Google Scholar 

  13. Harrigan, J. A., Oxman, T. E., & Rosenthal, R. (1985). Rapport expressed through nonverbal behavior. Journal of Nonverbal Behavior, 9(2), 95–110.

    Article  Google Scholar 

  14. Hoque, M. E., McDuff, D. J., & Picard, R. W. (2012). Exploring temporal patterns in classifying frustrated and delighted smiles. Journal of IEEE Transactions on Affective Computing, 99, 1–13.

    Google Scholar 

  15. Huang, L., Morency, L. P., & Gratch, J. (2011). Virtual rapport 2.0. In Intelligent virtual agents (pp. 68–79). Berlin: Springer.

  16. Jabon, M. E., Ahn, S. J., & Bailenson, J. N. (2011a). Automatically analyzing facial-feature movements to identify human errors. IEEE Journal of Intelligent Systems, 26(2), 54–63.

    Google Scholar 

  17. Jabon, M. E., Bailenson, J. N., Pontikakis, E. D., Takayama, L., & Nass, C. (2011b). Facial expression analysis for predicting unsafe driving behavior. IEEE Pervasive Computing, 10(4), 84–95.

    Article  Google Scholar 

  18. Kapur, A., Kapur, A., Virji-Babul, N., Tzanetakis, G., & Driessen, P. F. (2005). Gesture-based affective computing on motion capture data. In Affective Computing and Intelligent Interaction (pp. 1–7). Berlin, Heidelberg: Springer.

  19. Kendon, A. (1970). Movement coordination in social interaction: Some examples described. Acta Psychologica, 32, 100–125.

    PubMed  Article  Google Scholar 

  20. Kleinsmith, A., & Bianchi-Berthouze, N. (2007). Recognizing affective dimensions from body posture. In Affective computing and intelligent interaction (pp. 48–58). Berlin: Springer.

  21. Kurtzberg, T. R., & Amabile, T. M. (2001). From Guilford to creative synergy: Opening the black box of team-level creativity. Creativity Research Journal, 13(3–4), 285–294.

    Article  Google Scholar 

  22. La France, M., & Broadbent, M. (1976). Group rapport: Posture sharing as a nonverbal indicator. Group and Organization Studies, 1, 328–333.

    Article  Google Scholar 

  23. Lumsden, J., Miles, L. K., & Macrae, C. N. (2012). Perceptions of synchrony: Different strokes for different folks? Perception, 41(12), 1529.

    PubMed  Article  Google Scholar 

  24. Martin, C. C., Burkert, D. C., Choi, K. R., Wieczorek, N. B., McGregor, P. M., Herrmann, R. A., et al. (2012, April). A real-time ergonomic monitoring system using the Microsoft Kinect. In IEEE Systems and Information Design Symposium (SIEDS) (pp. 50–55).

  25. McLeod, P. L., Lobel, S. A., & Cox, T. H. (1996). Ethnic diversity and creativity in small groups. Small Group Research, 27(2), 248–264.

    Article  Google Scholar 

  26. Meservy, T. O., Jensen, M. L., Kruse, J., Burgoon, J. K., & Jay, F. (2005). Detecting deception through automatic, unobtrusive analysis of nonverbal behavior. IEEE Intelligent Systems, 20(5), 36–43.

    Google Scholar 

  27. Oppezzo, M., & Schwartz, D. L. (2014). Give your ideas some legs: The positive effect of walking on creative thinking. Journal of Experimental Psychology: Learning, Memory, and Cognition. doi:10.1037/a0036577.

  28. Paxton, A., & Dale, R. (2013). Frame-differencing methods for measuring bodily synchrony in conversation. Behavior Research Methods, 45(2), 329–343.

    PubMed  Article  Google Scholar 

  29. Pentland, A. S. (2010). Honest signals. Cambridge: MIT press.

  30. Ramseyer, F., & Tschacher, W. (2011). Nonverbal synchrony in psychotherapy: Coordinated body-movement reflects relationship quality and outcome. Journal of Consulting and Clinical Psychology, 79(3), 284–295.

    PubMed  Article  Google Scholar 

  31. Schmidt, R. C., Morr, S., Fitzpatrick, P., & Richardson, M. J. (2012). Measuring the dynamics of interactional synchrony. Journal of Nonverbal Behavior, 36(4), 263–279.

  32. Sung, J., Ponce, C., Selman, B., & Saxena, A. (2011). Human activity detection from RGBD images. AAAI 2011 workshop: Plan, activity, and intent recognition.

  33. Tickle-Degnen, L., & Rosenthal, R. (1990). The nature of rapport and its nonverbal correlates. Psychological Inquiry, 1(4), 285–293.

    Article  Google Scholar 

  34. Torrance, E. P. (1970). Influence of dyadic interaction on creative functioning. Psychological Reports, 26(2), 391–394.

    PubMed  Article  Google Scholar 

  35. Vinciarelli, A., Pantic, M., Bourlard, H., & Pentland, A. (2008). Social signal processing: State-of-the-art and future perspectives of an emerging domain. In Proceedings of the 16th ACM international conference on multimedia (pp. 1061–1070). New York: ACM.

  36. Witten, I., Eibe, F., & Hall, M. A. (2011). Data mining: Practical machine learning tools and techniques. Burlington MA: Morgan Kaufman.

    Google Scholar 

  37. Won, A. S., Bailenson, J. N., & Janssen, J. H. (2014). Automatic detection of nonverbal behavior predicts learning in dyadic interactions. Manuscript submitted for publication.

  38. Microsoft Corp. Redmond WA. Kinect for Xbox 360.

  39. Zebrowitz, L. A., & Montepare, J. M. (2006). The ecological approach to person perception: Evolutionary roots and contemporary offshoots. In M. Schaller, J. A. Simpson, & D. T. Kenrick (Eds.) Evolution and social psychology. First Edition (pp. 81–113), New York: Psychology Press.

Download references

Acknowledgments

The work presented herein was funded in part by Konica Minolta as part of a Stanford Media-X grant, and we thank them for the valuable insights provided by their visiting researchers, in particular Dr. Haisong Gu. In addition, it was funded in part by grant 108084-5031715-4 from the National Science Foundation. The authors also thank lab manager Cody Karutz for coordinating the administrative aspects of running this study, and Jimmy Lee, Pamela Martinez, Evan Shieh, Alex Zamoshchin, Angel Olvera, Christine Tataru, Mark Diaz and Mark Peng for their help in coding and data analysis, and Dr. Laura Aymerich-Franch, Ketaki Shriram, Michelle Friend, Brian Perone, and Jakki Bailey for helpful comments on an earlier draft of this paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Andrea Stevenson Won.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Won, A.S., Bailenson, J.N., Stathatos, S.C. et al. Automatically Detected Nonverbal Behavior Predicts Creativity in Collaborating Dyads. J Nonverbal Behav 38, 389–408 (2014). https://doi.org/10.1007/s10919-014-0186-0

Download citation

Keywords

  • Nonverbal behavior
  • Synchrony
  • Gesture
  • Collaboration
  • Creativity
  • Kinect
  • Interpersonal communication
  • Contingency