A Sensory Control System for Adjusting Group Emotion Using Bayesian Networks and Reinforcement Learning

  • Jun-Ho Kim
  • Ki-Hoon Kim
  • Sung-Bae ChoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9648)


The relationship between sensory stimuli and emotion has been actively investigated, but it is relatively undisclosed to determine appropriate stimuli for inducing the target emotion for a group of people in the same space like school, hospital, store, etc. In this paper, we propose a stimuli control system to adjust group emotion in a closed space, especially kindergarten. The proposed system predicts the next emotion of a group of people using modular tree-structured Bayesian networks, and controls the stimuli appropriate to the target emotion using utility table initialized by domain knowledge and adapted by reinforcement learning as the cases of stimuli and emotions are cumulated. To evaluate the proposed system, the real data were collected for five days from a kindergarten where the sensor and stimulus devices were installed. We obtained 84 % of prediction accuracy, and 56.2 % of stimuli control accuracy. Moreover, in the scenario tests on math and music classes, we could control the stimuli to fit the target emotion with 63.2 % and 76.3 % accuracies, respectively.



This work was supported by the Industrial Strategic Technology Development Program, 10044828, Development of Augmenting Multisensory Technology for Enhancing Significant Effects on the Service Industry, funded by the Ministry of Trade, Industry and Energy (MI, Korea).


  1. 1.
    Brave, S., Nass, C.: The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, pp. 81–96. CRC Press, Boca Raton (2003)Google Scholar
  2. 2.
    Mehrabian, A., Russell, J.: An Approach to Environmental Psychology. MIT Press, Cambridge (1974)Google Scholar
  3. 3.
    Donovan, R., Rossiter, J.: Store atmosphere: an environmental psychology approach. J. Retail. 58(1), 34–57 (1982)Google Scholar
  4. 4.
    Black, M., Yacoob, Y.: Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. In: International Conference on Computer Vision, pp. 374–381 (1995)Google Scholar
  5. 5.
    Eyharabide, V., Amandi, A., Courgeon, M., Clavel, C., Zakaria, C., Martin, J.: An ontology for predicting students’ emotions during a quiz. In: IEEE Workshop on Affective Computational Intelligence, pp. 1–8 (2011)Google Scholar
  6. 6.
    Yacoub, S., Simske, S., Lin, X., Burns, J.: Recognition of emotions in interactive voice response systems. In: 8th European Conference on Speech Communication and Technology, pp. 1–4 (2003)Google Scholar
  7. 7.
    Lee, C., Mover, E., Busso, C., Lee, S., Narayanan, S.: Emotion recognition using a hierarchical binary decision tree approach. Speech Commun. 53, 1162–1171 (2011)CrossRefGoogle Scholar
  8. 8.
    Utane, A., Nalbalwar, S.: Emotion recognition through speech using Gaussian mixture model and hidden Markov model. Int. J. Adv. Res. Comput. Sci. Software Eng. 3, 742–746 (2013)Google Scholar
  9. 9.
    Hayamizu, T., Mutsuo, S., Miyawaki, K., Mori, H., Nishiguchi, S., Yamashita, N.: Group emotion estimation using Bayesian network based on facial expression and prosodic information. In: International Conference on Control System, Computing and Engineering, pp. 23−25 (2012)Google Scholar
  10. 10.
    Kim, J., Cho, S.-B.: Predicting group emotion in kindergarten classes by modular Bayesian networks. In: 7th International Conference on Soft Computing and Pattern Recognition, pp. 298–302 (2015)Google Scholar
  11. 11.
    Ivon, A., James, R., Beverly, W.: Using an intelligent tutor and math fluency training to improve math performance. Int. J. Artif. Intell. Educ. 21(1–2), 135–152 (2011)Google Scholar
  12. 12.
    Yang, K., Cho, S.-B.: Modular dynamic Bayesian network based on Markov boundary for emotion prediction in multi-sensory environment. In: 10th International Conference on Natural Computation, pp. 1131–1136 (2014)Google Scholar
  13. 13.
    Park, J., Kim, J., Hwang, S.: The study on emotional classification algorithm using Vibraimage technology. In: ACED Asian Conference on Ergonomics and Design (2014)Google Scholar
  14. 14.
    Romero, C., Ventura, S., Pechenizkiy, M., Baker, RSJd: Handbook of Educational Data Mining, pp. 323–338. CRC Press, Boca Raton (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceYonsei UniversitySeoulKorea

Personalised recommendations