Emotional empathy transition patterns from human brain responses in interactive communication situations
- 309 Downloads
The paper reports our research aiming at utilization of human interactive communication modeling principles in application to a novel interaction paradigm designed for brain–computer/machine-interfacing (BCI/BMI) technologies as well as for socially aware intelligent environments or communication support systems. Automatic procedures for human affective responses or emotional states estimation are still a hot topic of contemporary research. We propose to utilize human brain and bodily physiological responses for affective/emotional as well as communicative interactivity estimation, which potentially could be used in the future for human–machine/environment interaction design. As a test platform for such an intelligent human–machine communication application, an emotional stimuli paradigm was chosen to evaluate brain responses to various affective stimuli in an emotional empathy mode. Videos with moving faces expressing various emotional displays as well as speech stimuli with similarly emotionally articulated sentences are presented to the subjects in order to further analyze different affective responses. From information processing point of view, several challenges with multimodal signal conditioning and stimuli dynamic response extraction in time frequency domain are addressed. Emotions play an important role in human daily life and human-to-human communication. This is why involvement of affective stimuli principles to human–machine communication or machine-mediated communication with utilization of multichannel neurophysiological and periphery physiological signals monitoring techniques, allowing real-time subjective brain responses evaluation, is discussed. We present our preliminary results and discuss potential applications of brain/body affective responses estimation for future interactive/smart environments.
KeywordsEmotional stages from brain responses estimation Communication with emotional stages evaluation Socially aware intelligent environments design Brain–computer or machine interfacing paradigms
Authors would like to thank Prof. Michihiko Minoh and Prof. Koh Kakusho of Kyoto University for their support and fruitful discussions in frame of the project “Intelligent Media Technology for Supporting Natural Communication between People”, which was partially supported by the Ministry of Education, Science, Sports and Culture in Japan, Grant-in-Aid for Creative Scientific Research, 13GS0003, where communicative interactivity approach was first developed. Also, we would like to thank for many discussions Prof. Victor V. Kryssanov of Ritsumeikan University in Kyoto at the beginning stages of presented research, which were very valuable to shape the final approach.
- Adler R, Rodman G (2003) Undestanding human communication. Oxford University Press, OxfordGoogle Scholar
- Baron-Cohen S (2004) Mind reading—the interactive guide to emotions. Jessica Kingsley Publishers, LondonGoogle Scholar
- Murdoch D, Chow ED (2007) Ellipse: functions for drawing ellipses and ellipse-like confidence regions. R package version 0.3–5 (porting to R by Jesus M. Frias Celayeta)Google Scholar
- Niedermeyer E, Da Silva FL (eds) (2004) Electroencephalography: basic principles, clinical applications, and related fields. 5th edn. Lippincott Williams & Wilkins, PhiladelphiaGoogle Scholar
- R Development Core Team (2009) R: a language and environment for statistical computing. R foundation for statistical computing, Vienna, Austria, ISBN 3-900051-07-0Google Scholar
- Rutkowski TM, Mandic D (2007) Artificial intelligence for human computing. In: Lecture notes in artificial intelligence, vol 4451, chap Modelling the communication atmosphere—a human centered multimedia approach to evaluate communicative situations. Springer, Berlin, pp 155–169Google Scholar
- Rutkowski TM, Mandic D (2008) Engineering approaches to conversational informatics, chap Modeling communication atmosphere. Wiley, pp 353–369 (Wiley Series in Agent Technology)Google Scholar
- Rutkowski TM, Yamakata SSY, Kakusho K, Minoh M (2003) Knowledge-based intelligent information and engineering systems. In: Lecture notes in computer science, vol 2774, chap Toward the human communication efficiency monitoring from captured audio and video media in real environments. Springer, Berlin, pp 1093–1100Google Scholar
- Rutkowski TM, Cichocki A, Ralescu AL, Mandic DP (2008) Emotional states estimation from multichannel EEG maps. In: Wang R, Gu F, Shen E (eds) Advances in cognitive neurodynamics ICCN 2007, Proceedings of the international conference on cognitive neurodynamics, Neuroscience. Springer, Berlin, pp 695–698Google Scholar
- Rutkowski TM, Cichocki A, Tanaka T, Mandic DP, Cao J, Ralescu AL (2009) Multichannel spectral pattern separation—an EEG processing application. In: Proceedings of the 2009 IEEE international conference on acoustics, speech, and signal processing (ICASSP2009), IEEE. pp 373–376Google Scholar