Skip to main content
SpringerLink
Log in

Evaluation of User’s Emotional Experience Through Neurological and Physiological Measures in Playing Serious Games

  • Conference paper
  • First Online:
Intelligent Systems Design and Applications (ISDA 2020)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1351))

Abstract

The importance of evaluating user experience (UX) is increasing gradually and so are the varieties of methods of UX evaluation. In order to keep the interactive computing and entertaining system sustainable in business, satisfying the user needs is a must, while UX evaluation plays a vital role in this respect. Again, gaming experience is largely impacted by users’ emotions. Among various types of games, serious game is a particular type which provides some purpose along with common gaming entertainment. The objective of this research is to show how objective methods (neurological and physiological measures) can be used to infer users’ emotional experience. To attain this objective, a machine learning-based approach considering both the neurological and physiological measures is proposed to evaluate the users’ emotions (UX) while playing serious game. The proposed approach is simulated through an experimental study to evaluate UX of an educational serious game –‘Programming Hero’. The finding of the study indicates that neurological and physiological measures of UX evaluation can infer the users’ emotions in playing serious game.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://play.google.com/store/apps/details?id=com.learnprogramming.codecamp.

  2. 2.

    https://store.neurosky.com/pages/mindwave.

  3. 3.

    https://play.google.com/store/apps/details?id=com.isomerprogramming.application.eegID&hl=en&gl=US.

  4. 4.

    https://www.mi.com/global/mi-smart-band-4/.

  5. 5.

    https://play.google.com/store/apps/details?id=com.xiaomi.hm.health&hl=en&gl=US.

References

  1. ThinkGear measurements (MindSet Pro/TGEM). http://support.neurosky.com/kb/science/thinkgear-measurements-mindset-protgem. Accessed 29 Oct 2020

  2. Understanding Your Target Heart Rate. https://www.hopkinsmedicine.org/health/wellness-and-prevention/understanding-your-target-heart-rate. Accessed 29 Oct 2020

  3. Alarcao, S.M., Fonseca, M.J.: Emotions recognition using EEG signals: a survey. IEEE Trans. Affect. Comput. 10(3), 374–393 (2017)

    Article  Google Scholar 

  4. Anolli, L., Mantovani, F., Confalonieri, L., Ascolese, A., Peveri, L.: Emotions in serious games: from experience to assessment. Int. J. Emerg. Tech. Learn. (iJET) 5(2010), 7–16 (2010)

    Google Scholar 

  5. Baraybar-Fernández, A., Baños-González, M., Barquero-Pérez, Ó., Goya-Esteban, R., De-la Morena-Gómez, A.: Evaluation of emotional responses to television advertising through neuromarketing. Comunicar. Media Educ. Res. J. 25(2) (2017)

    Google Scholar 

  6. Bardzell, J., Bardzell, S., Pace, T., Karnell, J.: Making user engagement visible: a multimodal strategy for interactive media experience research. In: CHI’08 Extended Abstracts on Human Factors in Computing Systems, pp. 3663–3668 (2008)

    Google Scholar 

  7. Benyon, D.: Designing User Experience: A Guide to HCI. UX and Interaction Design. Pearson, London (2019)

    Google Scholar 

  8. Bernhaupt, R. (ed.): Evaluating User Experience in Games: Concepts and Methods. Springer, London (2010)

    Google Scholar 

  9. Breuer, J., Bente, G.: Why so serious? on the relation of serious games and learning. J. Comput. Game Cult. 4(1), 7–24 (2010)

    Google Scholar 

  10. Brooks, P., Hestnes, B.: User measures of quality of experience: why being objective and quantitative is important. IEEE Network 24(2), 8–13 (2010)

    Article  Google Scholar 

  11. Bulagang, A.F., Weng, N.G., Mountstephens, J., Teo, J.: A review of recent approaches for emotion classification using electrocardiography and electrodermography signals. Inform. Med. Unlocked 20, 100363 (2020)

    Article  Google Scholar 

  12. Calderón, A., Ruiz, M.: A systematic literature review on serious games evaluation: an application to software project management. Comput. Educ. 87, 396–422 (2015)

    Article  Google Scholar 

  13. Cusveller, J., Gerritsen, C., de Man, J.: Evoking and measuring arousal in game settings. In: International Conference on Serious Games, pp. 165–174. Springer, Cham (2014)

    Google Scholar 

  14. Dalgleish, T., Power, M. (eds.): Handbook of Cognition and Emotion. John Wiley & Sons, Chichester (2000)

    Google Scholar 

  15. Devi, C.A., KarthikaRenuka, D., Soundarya, S.: A survey based on human emotion identification using machine learning and deep learning. J. Comput. Theor. Nanosci. 15(5), 1662–1665 (2018)

    Article  Google Scholar 

  16. Diya, S.Z., Prorna, R.A., Rahman, I.I., Islam, A.B., Islam, M.N.: Applying brain-computer interface technology for evaluation of user experience in playing games. In: 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE), pp. 1–6. IEEE, Cox’s Bazar, Bangladesh (2019)

    Google Scholar 

  17. Donald, I., MacLeod, K.: Glitchspace: teaching programming through puzzles in cyberspace. In: European Conference on Games Based Learning, pp. 148–154. Academic Conferences International Limited (2017)

    Google Scholar 

  18. Egger, M., Ley, M., Hanke, S.: Emotion recognition from physiological signal analysis: a review. Electron. Notes Theoret. Comput. Sci. 343, 35–55 (2019)

    Article  Google Scholar 

  19. Hornbæk, K.: Current practice in measuring usability: challenges to usability studies and research. Int. J. Hum. Comput. Stud. 64(2), 79–102 (2006)

    Article  Google Scholar 

  20. Ibrahim, R., Yusoff, R.C.M., Mohamed-Omar, H., Jaafar, A.: Students perceptions of using educational games to learn introductory programming. Comput. Inf. Sci. 4(1), 205–216 (2011)

    Google Scholar 

  21. Jankowski, J., Wa̧tróbski, J., Witkowska, K., Ziemba, P.: Eye tracking based experimental evaluation of the parameters of online content affecting the web user behaviour. In: Selected Issues in Experimental Economics, pp. 311–332. Springer, Cham (2016)

    Google Scholar 

  22. Knapp, R.B., Kim, J., André, E.: Physiological signals and their use in augmenting emotion recognition for human–machine interaction. In: Emotion-oriented systems, pp. 133–159. Springer, Berlin (2011)

    Google Scholar 

  23. Laamarti, F., Eid, M., El Saddik, A.: An overview of serious games. Int. J. Comput. Games Tech. 2014, 1–15 (2014)

    Google Scholar 

  24. Lester, J.C., Ha, E.Y., Lee, S.Y., Mott, B.W., Rowe, J.P., Sabourin, J.L.: Serious games get smart: Intelligent game-based learning environments. AI Mag. 34(4), 31–45 (2013)

    Google Scholar 

  25. Meftah, C., Retbi, A., Bennani, S., Idrissi, M.K.: Evaluation of user experience in the context of mobile serious game. In: 2019 International Conference on Intelligent Systems and Advanced Computing Sciences (ISACS), pp. 1–5. IEEE, Taza, Morocco (2019)

    Google Scholar 

  26. Moizer, J., Lean, J., Dell’Aquila, E., Walsh, P., Keary, A.A., O’Byrne, D., Di Ferdinando, A., Miglino, O., Friedrich, R., Asperges, R., et al.: An approach to evaluating the user experience of serious games. Comput. Educ. 136, 141–151 (2019)

    Article  Google Scholar 

  27. Muratet, M., Torguet, P., Viallet, F., Jessel, J.P.: Experimental feedback on Prog&Play: a serious game for programming practice. In: Computer Graphics Forum, vol. 30, pp. 61–73. Wiley Online Library (2011)

    Google Scholar 

  28. Nacke, L.E.: Games user research and physiological game evaluation. In: Game User Experience Evaluation, pp. 63–86. Springer, Cham (2015)

    Google Scholar 

  29. Nacke, L.E., Grimshaw, M.N., Lindley, C.A.: More than a feeling: measurement of sonic user experience and psychophysiology in a first-person shooter game. Interact. Comput. 22(5), 336–343 (2010)

    Article  Google Scholar 

  30. Posner, J., Russell, J.A., Peterson, B.S.: The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 17(3), 715–734 (2005)

    Article  Google Scholar 

  31. Protopsaltis, A., Pannese, L., Pappa, D., Hetzner, S.: Serious games and formal and informal learning. E-Learning Papers, pp. 1887–1542 (2011)

    Google Scholar 

  32. Ray, W.J., Cole, H.W.: EEG alpha activity reflects attentional demands, and beta activity reflects emotional and cognitive processes. Science 228(4700), 750–752 (1985)

    Article  Google Scholar 

  33. Sagi, O., Rokach, L.: Ensemble learning: a survey. Wiley Interdisc. Rev. Data Min. Knowl. Disc. 8(4), e1249 (2018)

    Google Scholar 

  34. Schmidt, A.: The user experience. Libr. J. 135(1), 28–29 (2010)

    Google Scholar 

  35. Shu, L., Xie, J., Yang, M., Li, Z., Li, Z., Liao, D., Xu, X., Yang, X.: A review of emotion recognition using physiological signals. Sensors 18(7), 2074 (2018)

    Article  Google Scholar 

  36. Shu, L., Yu, Y., Chen, W., Hua, H., Li, Q., Jin, J., Xu, X.: Wearable emotion recognition using heart rate data from a smart bracelet. Sensors 20(3), 718 (2020)

    Article  Google Scholar 

  37. Trindade, Y., Rebelo, F., Noriega, P.: Potentialities of a face reading tool to a digital game evaluation and development: a preliminary study. In: International Conference on Applied Human Factors and Ergonomics, pp. 371–381. Springer, Cham (2017)

    Google Scholar 

  38. Tubelo, R.A., Portella, F.F., Gelain, M.A., de Oliveira, M.M.C., de Oliveira, A.E.F., Dahmer, A., Pinto, M.E.B.: Serious game is an effective learning method for primary health care education of medical students: a randomized controlled trial. Int. J. Med. Inform. 130, 103944 (2019)

    Article  Google Scholar 

  39. Wioleta, S.: Using physiological signals for emotion recognition. In: 2013 6th International Conference on Human System Interactions (HSI), pp. 556–561. IEEE, Sopot, Poland (2013)

    Google Scholar 

  40. Yu, Q., Che, X., Ma, S., Pan, S., Yang, Y., Xing, W., Wang, X.: A hybrid user experience evaluation method for mobile games. IEEE Access 6, 49067–49079 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Military Institute of Science and Technology (MIST), Mirpur Cantonment, Dhaka, 1216, Bangladesh

    Tarannum Zaki, Nafiz Imtiaz Khan & Muhammad Nazrul Islam

Authors
  1. Tarannum Zaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nafiz Imtiaz Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Nazrul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tarannum Zaki .

Editor information

Editors and Affiliations

  1. Scientific Network for Innovation and Research Excellence, Machine Intelligence Research Labs (MIR Labs), Auburn, WA, USA

    Ajith Abraham

  2. Department of Computer Science, Università degli Studi di Milano, Milan, Milano, Italy

    Vincenzo Piuri

  3. Machine Intelligence Research Labs (MIR Labs), Auburn, WA, USA

    Niketa Gandhi

  4. Campus Centre de Créteil, Université Paris-Est Créteil, Créteil, France

    Patrick Siarry

  5. Department of Construction Management and Real Estate, Vilnius Gediminas Technical University, Vilnius, Lithuania

    Arturas Kaklauskas

  6. School of Engineering, Instituto Superior de Engenharia do Porto, Porto, Portugal

    Ana Madureira

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zaki, T., Khan, N.I., Islam, M.N. (2021). Evaluation of User’s Emotional Experience Through Neurological and Physiological Measures in Playing Serious Games. In: Abraham, A., Piuri, V., Gandhi, N., Siarry, P., Kaklauskas, A., Madureira, A. (eds) Intelligent Systems Design and Applications. ISDA 2020. Advances in Intelligent Systems and Computing, vol 1351. Springer, Cham. https://doi.org/10.1007/978-3-030-71187-0_96

Download citation

Publish with us

Policies and ethics

Search

Navigation