Human Nonverbal Behaviour Understanding in the Wild for New Media Art
Over the course of the London 2012 Olympics a large public installation took place in Central London. Its premise was to enable members of the public to express themselves by controlling the lights around the rim of the London Eye. The installation’s design and development was undertaken as a collaborative project between an interactive arts studio and researchers in the field of affective and behavioural computing. Over 800 people participated, taking control of the lights using their heart rates and hand gestures. This paper approaches nonverbal and affective behaviour understanding for new media art as a case study, and reports the design of this installation and the subsequent analysis of over one million frames of physiological and motion capture data. In doing so it sheds light on how the intersection of affective and behavioural computing and new media art could be beneficial to both researchers and artists.
KeywordsNew media art affective behaviour understanding in the wild gestural interaction mood nonverbal behaviour
Unable to display preview. Download preview PDF.
- 1.Tribe, M., Jana, R.: New Media Art. Taschen (2006)Google Scholar
- 8.Knapp, R.B., Jaimovich, J., Coghlan, N.: Measurement of motion and emotion during musical performance. In: 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, pp. 1–5. IEEE (September 2009)Google Scholar
- 9.Anttonen, J., Surakka, V.: Emotions and heart rate while sitting on a chair. In: Proc. CHI, p. 491 (2005)Google Scholar
- 10.Billinghurst, M.: Gesture based interaction. In: Haptic Input, pp. 1–35 (2011)Google Scholar
- 12.Castro, B.P., Velho, L., Kosminsky, D.: Integrarte: digital art using body interaction. In: Proc. of the Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging, pp. 11–15 (2012)Google Scholar
- 13.Visnjic, F., Smith, G.J., Scholz, A.: Creative applications (2013)Google Scholar
- 14.Scherer, K.R.: Emotion as a Multicomponent Process: A model and some cross-cultural data. Review of Personality and Social Psychology 5, 37–63 (1984)Google Scholar
- 15.Fox, E.: Emotion Science Cognitive and Neuroscientific Approaches to Understanding Human Emotions. Palgrave Macmillan (September 2008)Google Scholar
- 16.Pollick, F.E., Paterson, H.M., Bruderlin, A., Sanford, A.J.: Perceiving affect from arm movement. Cognition 82(2), B51–B61 (2001)Google Scholar
- 19.Saerbeck, M., Bartneck, C.: Perception of affect elicited by robot motion. In: Proc. of ACM/IEEE Int’l Conf. on Human-Robot Interaction, pp. 53–60 (2010)Google Scholar
- 22.Alaoui, S.F., Caramiaux, B., Serrano, M., Bevilacqua, F.: Movement qualities as interaction modality. In: Proceedings of the Designing Interactive Systems Conference on DIS 2012, pp. 761–769. ACM Press, New York (2012)Google Scholar