A Design Practice on Communicating Emotions Through Visual, Tactile and Auditory Simulations

  • Secil Ugur YavuzEmail author
  • Monica Bordegoni
  • Marina Carulli
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 34)


Emotional expression is an important human behaviour, which enriches communication. Sensory organs play crucial role in emotional perception. Today communication is mostly done via digital mediators, which dominantly address to vision excluding the other senses; therefore, communication becomes less affective. Wearable technology can appeal to sensory organs from very close distance due to its intimate interaction with human body. Hence, this technology can be used in order to make distant communication more affective by enabling multi-sensory interaction. This paper represents a user-centred design practice on wearable products that simulate sensorial feedbacks (tactile, visual and auditory) to express basic emotions. Three prototypes that transmit emotional messages were designed, built and tested to observe user behaviour. This paper discusses how user experience obtained through the user test can be taken further to design new communication products.


Emotion Sensory perception Wearable technology Design practice 


  1. 1.
    Konijn, E.A., Utz, S., Tanis, M., Barnes, S.B.: Mediated Interpersonal Communication. Abingdon. Routledge, Oxon (2008)Google Scholar
  2. 2.
    Picard, R.W.: Affective Computing. The MIT Press, Cambridge (1997)Google Scholar
  3. 3.
    Labs, P.S.F.K.: The future of wearable tech. Des. Stud. 19, 103–117 (1998)CrossRefGoogle Scholar
  4. 4.
    Darwin, C.: The Expression of the Emotions in Man and Animals, 3rd edn. Oxford University Press, New York (1998)Google Scholar
  5. 5.
    Russell, J.A.: Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychol. Bull. 115, 102–141 (1994)CrossRefGoogle Scholar
  6. 6.
    Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System. A Human Face, Salt Lake City, UT (2002)Google Scholar
  7. 7.
    Glowinsky, D., Camurri, A., Volpe, G., Chiarra, N., Cowie, R., McMahon, E., Jaimovich, J., Knapp, R.B.: Using induction and multimodal assessment to understand the role of emotion in musical performance. In: Proceedings of WS Emotion in HCI—Designing for People, pp. 8–12. Fraunhof, Stuttgart (2008)Google Scholar
  8. 8.
    Shan, S.G.C., McOwan, P.W.: Beyond facial expressions: learning human emotion from body gestures. In: British Machine Vision Conference (2007)Google Scholar
  9. 9.
    Wallbott, H.G.: Bodily expression of emotion. Eur. J. Soc. Psychol. 28(6), 879–896 (1998)CrossRefGoogle Scholar
  10. 10.
    Andersson, G., Höök, K., Mourao, D., Paiva, A., Costa, M.: Using a wizard of Oz study to inform the design of SenToy. In: Proceedings of Designing Interactive Systems DIS’02, ACM, London (2002)Google Scholar
  11. 11.
    Crilly, N., Moultrie, J., Clarkson, P.J.: Seeing things: consumer response to the visual domain in product design. Des. Stud. 25(6), 547–577 (2004)CrossRefGoogle Scholar
  12. 12.
    Rafaeli, A., Vilnai-Yavetz, I.: Emotion as a connection of physical artifacts and organizations. Organ. Sci. 15(6), 671–686 (2004)CrossRefGoogle Scholar
  13. 13.
    Meyer, L.B.: Emotion and Meaning in Music. University of Chicago Press, Chicago (1961)CrossRefGoogle Scholar
  14. 14.
    Zwicker, E.F.: Psyhoacoustics Facts and Models. Springer, Berlin (1990)Google Scholar
  15. 15.
    Khulage, A.A., Pathak, B.V.: Analysis of Speech Under Stress Using Linear Techniques and Non-Linear Techniques for Emotion Recognition System. CoRR (2012)Google Scholar
  16. 16.
    Scherer, K.: Vocal communication of emotion: a review of research paradigms. Speech Commun. 40, 227–256 (2003)CrossRefzbMATHGoogle Scholar
  17. 17.
    Mohammadi, G., Vinciarelli, A.: Towards a technology of nonverbal communication: vocal behavior in social and affective phenomena. In: Gokcay, D., Yildirim, G. (eds.) Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives, pp. 133–156. IGI, Hershey (2010)Google Scholar
  18. 18.
    Hertenstein, M.J., Keltner, D., App, B., Bulleit, B., Jaskolka, A.: Touch communicates distinct emotions. Emotion 6, 528–533 (2006)CrossRefGoogle Scholar
  19. 19.
    Jones, S.E., Yarbrough, A.E.: A naturalistic study of the meanings of touch. Commun. Monogr. 52, 19–56 (1985)CrossRefGoogle Scholar
  20. 20.
    Grandin, T.: Calming effects of deep touch pressure in patients with autistic disorder, college students, and animals. J. Child Adoloscent psychopharmacol. 2, 63–70 (1992)CrossRefGoogle Scholar
  21. 21.
    Vetere, F., Gibbs, M.R., Kjeldskov, J., Howard, S., Mueller, F., Pedell, S., Mecoles, K., Bunyan, M.: Mediating intimacy: designing technologies to support strong-tie relationships. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 471–480 (2005)Google Scholar
  22. 22.
    Gaver, B.: Provocative Awareness. Comput. Support. Coop. Work 11, 475–493 (2002)CrossRefGoogle Scholar
  23. 23.
    Brown, L.M., Williamson, J.: Shake2Talk: Multimodal Messaging for Interpersonal Communication HAID 2007. LNCS, pp. 44–55 (2007)Google Scholar
  24. 24.
    Chang, A., O’Modhrain, S., Jacob, R., Gunther, E., Ishii, H.: ComTouch: design of a vibrotactile communication device. In: Proceedings of DIS 2002, pp. 312–320. ACM (2002)Google Scholar
  25. 25.
    Motamedi, N.: The aesthetics of touch in interaction design. In: Proceedings of the 2007 Conference on Designing Pleasurable Products and Interfaces, pp. 455–460. ACM (2007)Google Scholar
  26. 26.
    Tsetserukou, D., Neviarouskaya, A.: iFeel IM: augmenting emotions during online communication. IEEE Comput. Graphics Appl. 30, 72–80 (2010)CrossRefGoogle Scholar
  27. 27.
    Grimmer, N.: Heart2Heart. In: Student Design Competition Intel’01. (2001)
  28. 28.
    Wilde, D., Schiphorst, T., Klooster, S.: Move to design/design to move: a conversation about designing for the body. Interactions 18(4), 22–27 (2011)CrossRefGoogle Scholar
  29. 29.
    Ugur, S., Bordegoni, M., Carulli, M., Mangiarotti, R., Wenveen, S.: Embodiment of emotions through wearable technology. In: Proceedings of ASME’10, pp. 28–31. Washington DC (2011)Google Scholar
  30. 30.
    Cacioppo, J.T., Berntson, G.G., Klein, D.J.: What is an emotion? The role of somatovisceral afference, with special emphasis on soma- tovisceral illusions. Rev. Pers. Soc. Psychol. 14, 63–98 (1993)Google Scholar
  31. 31.
    Schachter, S., Singer, J.: Cognitive, social, and physiological determinants of emotional state. Psychol. Rev. 69, 379–399 (1962)CrossRefGoogle Scholar
  32. 32.
    Jääskö, V., Mattelmäki, T.: Observing and probing. In: Proceedings of International Conference on Designing Pleasurable Products and Interfaces, pp. 126–131. ACM Press, New York (2003)Google Scholar
  33. 33.
    Field, T.: Touch. MIT Press, Cambridge (2001)Google Scholar
  34. 34.
    Dunne, A., Raby, F.: Design Noir: The Secret Life of Electronic Objects. Birkhäuser, Basel (2001)Google Scholar

Copyright information

© Springer India 2015

Authors and Affiliations

  • Secil Ugur Yavuz
    • 1
    Email author
  • Monica Bordegoni
    • 1
  • Marina Carulli
    • 1
  1. 1.Department of Mechanical EngineeringPolitecnico Di MilanoMilanItaly

Personalised recommendations