Abstract
In a world full of sensors that gather personal data and digital solutions that use these data to provide feedback and personalized experiences, biofeedback is increasingly involved in the definition of new paradigms for tailoring interactions. Companies are collecting and using personal data to propose personalized services. Content providers are pushing users to produce data in order to create personalized storytelling experiences. In this context, the tech market is offering new low-cost solutions able to gather biodata. The paper reports the results of evidence-based explorations aimed at formalizing knowledge regarding the use of passive and unconscious interaction to control the fruition of storytelling artifacts. We investigate a new interaction paradigm that promise to seamlessly enable unconscious and enactive interactions for movie experiences. We propose the use of emotion recognition and eye-tracking as exploratory technologies that promise to be a potential contribution to richer access to the spectators’ emotional involvement. We reflect on disruptive power of non-invasive technologies, given by the possibility to be used for home-cinema experiences. Investigating on emotional states of users in their decision we leverage on the emotive-cognitive data as a matter of creation and enabling of tailored movie experiences. Our research intends to explore the possibility of extracting knowledge from recognition of facial expressions that will contribute to foster its use in real-time passive interaction using emotion recognition as a trigger of enactivity that is not limited to interactive storytelling but opens new scenarios in the design of proactive systems for screens, spaces and environments. Furthermore, we provide suggestions as guidelines for the design of enactive experiences that leverage on emotion recognition and eye-tracking.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Calvo, A., et al.: Eye tracking impact on quality-of-life of ALS patients. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds.) ICCHP 2008. LNCS, vol. 5105, pp. 70–77. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-70540-6_9
Tikka, P., Korkeakoulu, T.: Enactive cinema: simulatorium Eisensteinense. University of Art and Design Helsinki, Helsinki (2008)
Ekman, P., et al.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford University Press, Oxford (1997)
Mitchell, W.J.: Me++: The Cyborg Self and the Networked City. MIT, Cambridge (2010)
Young, N.: The Virtual Self: How Our Digital Lives are Altering the World Around Us. McClelland & Stewart, Plattsburgh (2013)
Neff, G., Nafus, D.: Self-Tracking. The MIT Press, Cambridge (2016)
Lupton, D.: The digitally engaged patient: self-monitoring and self-care in the digital health era. Soc. Theory Health 11(3), 256–270 (2013). https://doi.org/10.1057/sth.2013.10
Quesenbery, W., Brooks, K.: Storytelling for User Experience: Crafting Stories for Better Design, p. 569 (2010)
Netflix. https://www.netflix.com/
Slade, D.: Black Mirror: Bandersnatch. Netflix (2018)
McManus, S.: Minecraft: Story Mode. Telltale Games, Mojang (2015)
Buchta, R., Grylls, B.: You vs. Wild. Netflix (2019)
Burdine, R., Castucciano, J.: Puss in Book: Trapped in an Epic Tale. Netflix (2017)
Damasio, A.R.: Looking for Spinoza: Joy, Sorrow, and the Feeling Brain, First Harvest edn. Harcourt, Orlando, Toronto, London (2003)
Ekman, P.: Facial expressions of emotion: new findings, new questions. Psychol. Sci. 3(1), 34–38 (1992). https://doi.org/10.1111/j.1467-9280.1992.tb00253.x
Tikka, P., Vuori, R., Kaipainen, M.: Narrative logic of enactive cinema: Obsession. Digit. Creat. 17(4), 205–212 (2006). https://doi.org/10.1080/14626260601074078
Maison, D., Pawłowska, B.: Using the FaceReader method to detect emotional reaction to controversial advertising referring to sexuality and homosexuality. In: Nermend, K., Łatuszyńska, M. (eds.) Neuroeconomic and Behavioral Aspects of Decision Making. SPBE, pp. 309–327. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-62938-4_20
Kahneman, D.: Thinking, Fast and Slow, 1st pbk. edn. Farrar, Straus and Giroux, New York (2013)
Unity. Unity Technologies. https://unity.com/
FaceReader. Noldus. https://doi.org/www.noldus.com/facereader
Lewinski, P., den Uyl, T.M., Butler, C.: Automated facial coding: validation of basic emotions and FACS AUs in FaceReader. J. Neurosci. Psychol. Econ. 7(4), 227–236 (2014). https://doi.org/10.1037/npe0000028
Jansen, J.: Automated Identification and Measurement of Suppressed Emotions using Emotion Recognition Software, p. 16 (2015)
Mandolfo, M.: You trust me, and I feel it. Influence of foreign live biofeedback on interpersonal trust-related behaviour. Politecnico di Milano (2017)
Sumi, K., Ueda, T.: Micro-expression recognition for detecting human emotional changes. In: Kurosu, M. (ed.) HCI 2016. LNCS, vol. 9733, pp. 60–70. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39513-5_6
Schneier, B.: Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World, Norton Paperback. W.W. Norton & Company, New York, London (2016)
Polaine, A., Lovlie, L., Reason, B.: Service Design: From Insight to Implementation. Rosenfeld Media, Brooklyn (2013)
Pillan, M., Varisco, L., Bertolo, M.: Facing digital dystopias: a discussion about responsibility in the design of smart products. In: Alonso, M.B., Ozcan, E. (eds.) Proceedings of the Conference on Design and Semantics of Form and Movement - Sense and Sensitivity, DeSForM 2017, pp. 121–131. InTech (2017)
Varisco, L.: Personal Interaction Design: introducing the discussion on the consequences of the use of personal information in the design process. Ph.D. dissertation, Politecnico di Milano, Milan, Italy (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Varisco, L., Interlandi, G. (2020). Designing Unconscious and Enactive Interaction for Interactive Movie Experience. In: Streitz, N., Konomi, S. (eds) Distributed, Ambient and Pervasive Interactions. HCII 2020. Lecture Notes in Computer Science(), vol 12203. Springer, Cham. https://doi.org/10.1007/978-3-030-50344-4_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-50344-4_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50343-7
Online ISBN: 978-3-030-50344-4
eBook Packages: Computer ScienceComputer Science (R0)