Effect of Visual Feedback Caused by Changing Mental States of the Avatar Based on the Operator’s Mental States Using Physiological Indices

  • Yoshimasa OhmotoEmail author
  • Seiji Takeda
  • Toyoaki Nishida
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10498)


Use of a virtual environment allows rehearsal or practice in specialized situations, extraordinary environments, or circumstances in which mistakes are not allowed. However, virtual experiences often do not translate to the real world. We propose to regard a virtual avatar as an agent which mediates between virtual experiences and the real ones. The aim of this study is to investigate how to enhance the effect of virtual experiences. To achieve this, we propose a method of providing feedback on human mental state by using affective avatar expressions based on physiological indices. We conduct experiments to evaluate the effect of the method. As a result, we find that feedback from avatar expressions can increase participants’ physiological responses without reducing concentration on the task. We suggest that this feedback can enhance commitment to virtual-world experiences.


Physiological indices Mental state estimation Affective computing Mental feedback 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Graafland, M., Schraagen, J., Schijven, M.: Systematic review of serious games for medical education and surgical skills training. British Journal of Surgery 99(10), 1322–1330 (2012)CrossRefGoogle Scholar
  2. 2.
    Katmada, A., Mavridis, A., Apostolidis, H., Tsiatsos, T.: Developing an adaptive serious game based on students’ bio-feedback. In: 6th International Conference on Information, Intelligence, Systems and Applications (IISA), pp. 1–6 (2015)Google Scholar
  3. 3.
    Lin, T., Omata, M., Hu, W., Imamiya, A.: Do physiological data relate to traditional usability indexes? In: Proceedings of the 17th Australia Conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future, pp. 1–10 (2005)Google Scholar
  4. 4.
    Mandryk, R.L., Inkpen, K.M.: Physiological indicators for the evaluation of co-located collaborative play. In: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, pp. 102–111 (2004)Google Scholar
  5. 5.
    Mokka, S., Väätänen, A., Heinilä, J., Välkkynen, P.: Fitness computer game with a bodily user interface. In: Proceedings of the Second International Conference on Entertainment Computing, pp. 1–3. Carnegie Mellon University (2003)Google Scholar
  6. 6.
    Ohmoto, Y., Kataoka, M., Nishida, T.: The effect of convergent interaction using subjective opinions in the decision-making process. In: Proc. the 36th Annual Conference of the Cognitive Science Society, pp. 2711–2716 (2014)Google Scholar
  7. 7.
    Ohmoto, Y., Takeda, S., Nishida, T.: Distinction of intrinsic and extrinsic stress in an exercise game by combining multiple physiological indices. In: 2015 7th International Conference on Games and Virtual Worlds for Serious Applications (VS-Games), pp. 1–4 (2015)Google Scholar
  8. 8.
    Rego, P., Moreira, P.M., Reis, L.P.: Serious games for rehabilitation: a survey and a classification towards a taxonomy. In: 5th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–6. IEEE (2010)Google Scholar
  9. 9.
    Rosenberg, R.S., Baughman, S.L., Bailenson, J.N.: Virtual superheroes: Using superpowers in virtual reality to encourage prosocial behavior. PloS One 8(1), e55003 (2013)CrossRefGoogle Scholar
  10. 10.
    Yucha, C., Montgomery, D.: Evidence-based practice in biofeedback and neurofeedback. AAPB, Wheat Ridge (2008)Google Scholar
  11. 11.
    Zotev, V., Phillips, R., Yuan, H., Misaki, M., Bodurka, J.: Self-regulation of human brain activity using simultaneous real-time fmri and eeg neurofeedback. NeuroImage 85, 985–995 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yoshimasa Ohmoto
    • 1
    Email author
  • Seiji Takeda
    • 1
  • Toyoaki Nishida
    • 1
  1. 1.Graduate School of InfomaticsKyoto UniversityKyoto-shiJapan

Personalised recommendations