Abstract
We seek to investigate possibilities of extending the emotional and cognitive experience of using products or services through the cross-modality of vision with other senses (synesthesia). Through multi-sensory, interactive environments, consumers become more engaged in the use of a product or service and may, in fact, participate as co-creators of their own experiences. To achieve the highest level of spontaneity and provoke human activation to enable us to study this, we suggest an experimental context based on interactive technologies, aesthetics and design. For this purpose, we use an artistic environment in the form of an interactive installation. Two examples of such experimental interactive art installations, Art Machine: Mindcatcher and Re-Digital, are described in this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
van Leeuwen, T.M., Singer, W., Nikolić, D.: The merit of synesthesia for consciousness research. Front. Psychol. 6, 1850 (2015)
Simner, J.: Defining synaesthesia. Br. J. Psychol. 103(1), 1–15 (2012)
Afra, P., Anderson, J., Funke, M., et al.: Neurophysiological investigation of idiopathic acquired auditory–visual synesthesia. Neurocase 18(4), 323–329 (2012)
Ginsberg, L.: A case of synaesthesia. Am. J. Psychol. 34, 582–589 (1923)
Cytowic, R.E.: The Man Who Tasted Shapes: A Bizarre Medical Mystery Offers Revolutionary Insights Into Reasoning, Emotion, and Consciousness. Putman, New York (1993)
Beeli, G., Esslen, M., Jäncke, L.: Synaesthesia: when coloured sounds taste sweet. Nature 434(7029), 38 (2005)
O’Callaghan, C.: Synesthesia vs. crossmodal illusions (2017)
Jewanski, J., Simner, J., Day, S.A., Ward, J.: The development of a scientific understanding of synesthesia from early case studies (1849–1873). J. History Neurosci. 20(4), 284–305 (2011)
Ramachandran, V.S., Hubbard, E.M.: The phenomenology of synaesthesia. J. Conscious. Stud. 10(8), 49–57 (2003)
Wu, Y.L., Chen, P.C.: The Synesthesia effects of Online Advertising Stimulus Design on Word-of-Mouth and Purchase Intention: From the Perspective of Consumer Olfactory and Gustatory (2016)
Rich, A.N., Bradshaw, J.L., Mattingley, J.B.: A systematic, large-scale study of synaesthesia: implications for the role of early experience in lexical-colour associations. Cognition 98(1), 53–84 (2005)
Stenslie, S.: Virtual touch: a study of the use and experience of touch in artistic, multimodal and computer-based environments (2010)
Burnham, J.: The aesthetics of intelligent systems. https://monoskop.org/images/0/02/BurnhamJack1970TheAestheticsofIntelligentSystems.pdf. Accessed 02 Oct 2019
Ascott, R.: Telematic Embrace: Visionary Theories of Art, Technology, and Consciousness. University of California Press, Berkeley (2007)
Willats, S.: Art and Social Function, Batsford (2002)
Walker, J.A., Phillpot, C.: Glossary of Art, Architecture and Design Since 1945. Library Association Publishing, Chicago (1992)
Giaccardi E.: Interactive strategies of network art. relationships and agency. In: CADE 1999: Third Conference on Computers in Art & Design Education. Citeseer (1999)
Gonzales, A., Finley, T., Duncan, S.: Interactive art: effects on user identity user satisfaction. In: CHI 2009 Proceedings of the 27th International Conference on Human Factors in Computing Systems (2008)
Spence, C.: Audiovisual multisensory integration. Acoust. Sci. Technol. 28(2), 61–70 (2007)
Tao, C.C., Bucy, E.P.: Conceptualizing media stimuli in experimental research: psychological versus attribute-based definitions. Hum. Commun. Res. 33(4), 397–426 (2007)
Vorderer, P., Knobloch, S., Schramm, H.: Does entertainment suffer from interactivity? The impact of watching an interactive tv movie on viewers’ experience of entertainment. Med. Psychol. 3(4), 343–363 (2001)
Bahrick, L.E., Lickliter, R.: Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. Dev. Psychol. 36(2), 190 (2000)
Spence, C.: The report on the secret of the senses. The Communication Group, London (2002)
Stein, B.E., Meredith, M.A.: The Merging of the Senses. The MIT Press, Cambridge (1993)
Washburn, D.A., Jones, L.M., Satya, R.V., Bowers, C.A., Cortes, A.: Olfactory use in virtual environment training. Model. Simul. 2(3), 19–25 (2003)
Ferreira, A.A., Brito, G., Silva, L.N.D., Mouzinho, J.V., Morais, R., Pereira, J.R.: Synesthesia vision integration with Recife’s public transport. In: Proceedings of the 16th Web For All 2019 Personalization-Personalizing the Web, p. 15. ACM, May 2019
Fleischmann, M., Strauss, W., Bohn, C.A.: Liquid views: rigid waves. In: ACM SIGGRAPH 98 Electronic Art and Animation Catalog. ACM, p. 21 (1998)
Hayes, S.C., Strosahl, K.D., Wilson, K.G.: Review of acceptance and commitment therapy: an experiential approach to behavior change (2002)
Schifferstein, H.N., Spence, C.: Multisensory product experience. In: Product Experience, pp. 133–161. Elsevier (2008)
Sense, L.M.B.: Build Powerful Brands Through Touch, Taste, Smell. Sight, and Sound New York. Free Press (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Nikolić, P.K., Russo, N.L. (2020). Crossing Sensory Boundaries with Creative Productions. In: Santos, H., Pereira, G., Budde, M., Lopes, S., Nikolic, P. (eds) Science and Technologies for Smart Cities. SmartCity 360 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 323. Springer, Cham. https://doi.org/10.1007/978-3-030-51005-3_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-51005-3_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-51004-6
Online ISBN: 978-3-030-51005-3
eBook Packages: Computer ScienceComputer Science (R0)